[jira] [Commented] (HBASE-15338) Add a option to disable the data block cache for testing the performance of underlying file system
[ https://issues.apache.org/jira/browse/HBASE-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177412#comment-15177412 ] Anoop Sam John commented on HBASE-15338: I was against adding one more config. But only in cache on read area we dont have (Other like cache on write and all we have global configs) one, I changed my mind.. A user can not set global level as false and enable it on one HCD. It wont honor that. And what abt enabled on Scan level? It is not set to true but Scan sets as cache we wont cache still right? Just for confirmation asked. pls add it clearly in release notes. > Add a option to disable the data block cache for testing the performance of > underlying file system > -- > > Key: HBASE-15338 > URL: https://issues.apache.org/jira/browse/HBASE-15338 > Project: HBase > Issue Type: Improvement > Components: integration tests >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15338-trunk-v1.diff, HBASE-15338-trunk-v2.diff, > HBASE-15338-trunk-v3.diff, HBASE-15338-trunk-v4.diff, > HBASE-15338-trunk-v5.diff, HBASE-15338-trunk-v6.diff > > > When testing and comparing the performance of different file systems(HDFS, > Azure blob storage, AWS S3 and so on) for HBase, it's better to avoid the > affect of the HBase BlockCache and get the actually random read latency when > data block is read from underlying file system. (Usually, the index block and > meta block should be cached in memory in the testing). > So we add a option in CacheConfig to disable the data block cache. > Suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177404#comment-15177404
]
Anoop Sam John commented on HBASE-15325:
{code}
// In every RPC response there should be at most a single partial result.
Furthermore, if
// there is a partial result, it is guaranteed to be in the last position
of the array.
Result last = resultsFromServer[resultsFromServer.length - 1];
Result partial = last.isPartial() ? last : null;
{code}
Seeing the comment. That also is no longer true if we change the meaning of
partial. There can be more than one batch in a single RPC response. Any other
places we have such consideration?
> ResultScanner allowing partial result will miss the rest of the row if the
> region is moved between two rpc requests
> ---
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt,
> HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt,
> HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt,
> HBASE-15325-v6.5.txt, HBASE-15325-v6.txt
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for
> one rpc request. And client can setAllowPartial or setBatch to get several
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get
> the next part if there is a partial result before. If we move the region to
> another RS, client will get a NotServingRegionException and open a new
> scanner to the new RS which will be regarded as a new scan from the end of
> this row. So the rest cells of the row of last result will be missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15338) Add a option to disable the data block cache for testing the performance of underlying file system
[ https://issues.apache.org/jira/browse/HBASE-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177399#comment-15177399 ] Liu Shaohui commented on HBASE-15338: - [~anoop.hbase] [~stack] [~jingcheng...@intel.com] Any other suggestions? I will commit it tomorrow of no objection. Thanks~ > Add a option to disable the data block cache for testing the performance of > underlying file system > -- > > Key: HBASE-15338 > URL: https://issues.apache.org/jira/browse/HBASE-15338 > Project: HBase > Issue Type: Improvement > Components: integration tests >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15338-trunk-v1.diff, HBASE-15338-trunk-v2.diff, > HBASE-15338-trunk-v3.diff, HBASE-15338-trunk-v4.diff, > HBASE-15338-trunk-v5.diff, HBASE-15338-trunk-v6.diff > > > When testing and comparing the performance of different file systems(HDFS, > Azure blob storage, AWS S3 and so on) for HBase, it's better to avoid the > affect of the HBase BlockCache and get the actually random read latency when > data block is read from underlying file system. (Usually, the index block and > meta block should be cached in memory in the testing). > So we add a option in CacheConfig to disable the data block cache. > Suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177354#comment-15177354
]
stack commented on HBASE-15376:
---
Sounds good by me. If it just 'broke', no need to bring it along . Just release
note it.
> ScanNext metric is size-based while every other per-operation metric is time
> based
> --
>
> Key: HBASE-15376
> URL: https://issues.apache.org/jira/browse/HBASE-15376
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
> Attachments: HBASE-15376.patch, HBASE-15376_v1.patch
>
>
> We have per-operation metrics for {{Get}}, {{Mutate}}, {{Delete}},
> {{Increment}}, and {{ScanNext}}.
> The metrics are emitted like:
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> ...
> "ScanNext_num_ops" : 194705,
> "ScanNext_min" : 0,
> "ScanNext_max" : 18441,
> "ScanNext_mean" : 7468.274651395701,
> "ScanNext_median" : 583.0,
> "ScanNext_75th_percentile" : 583.0,
> "ScanNext_95th_percentile" : 13481.0,
> "ScanNext_99th_percentile" : 13481.0,
> {code}
> The problem is that all of Get,Mutate,Delete,Increment,Append,Replay are time
> based tracking how long the operation ran, while ScanNext is tracking
> returned response sizes (returned cell-sizes to be exact). Obviously, this is
> very confusing and you would only know this subtlety if you read the metrics
> collection code.
> Not sure how useful is the ScanNext metric as it is today. We can deprecate
> it, and introduce a time based one to keep track of scan request latencies.
> ps. Shamelessly using the parent jira (since these seem relavant).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177343#comment-15177343
]
stack commented on HBASE-15329:
---
Thanks lads.
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177320#comment-15177320
]
Hadoop QA commented on HBASE-15325:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 37s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 36s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 23s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 25s {color}
| {color:red} hbase-client in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 33m 7s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 18s {color}
| {color:red}
[jira] [Updated] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15368: -- Resolution: Later Status: Resolved (was: Patch Available) > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9393) Hbase does not closing a closed socket resulting in many CLOSE_WAIT
[ https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177302#comment-15177302 ] Ashish Singhi commented on HBASE-9393: -- Test failures are not related to patch. > Hbase does not closing a closed socket resulting in many CLOSE_WAIT > > > Key: HBASE-9393 > URL: https://issues.apache.org/jira/browse/HBASE-9393 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.2, 0.98.0 > Environment: Centos 6.4 - 7 regionservers/datanodes, 8 TB per node, > 7279 regions >Reporter: Avi Zrachya >Assignee: Ashish Singhi >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-9393.patch, HBASE-9393.v1.patch, > HBASE-9393.v10.patch, HBASE-9393.v11.patch, HBASE-9393.v12.patch, > HBASE-9393.v13.patch, HBASE-9393.v14.patch, HBASE-9393.v2.patch, > HBASE-9393.v3.patch, HBASE-9393.v4.patch, HBASE-9393.v5.patch, > HBASE-9393.v5.patch, HBASE-9393.v5.patch, HBASE-9393.v6.patch, > HBASE-9393.v6.patch, HBASE-9393.v6.patch, HBASE-9393.v7.patch, > HBASE-9393.v8.patch, HBASE-9393.v9.patch > > > HBase dose not close a dead connection with the datanode. > This resulting in over 60K CLOSE_WAIT and at some point HBase can not connect > to the datanode because too many mapped sockets from one host to another on > the same port. > The example below is with low CLOSE_WAIT count because we had to restart > hbase to solve the porblem, later in time it will incease to 60-100K sockets > on CLOSE_WAIT > [root@hd2-region3 ~]# netstat -nap |grep CLOSE_WAIT |grep 21592 |wc -l > 13156 > [root@hd2-region3 ~]# ps -ef |grep 21592 > root 17255 17219 0 12:26 pts/000:00:00 grep 21592 > hbase21592 1 17 Aug29 ?03:29:06 > /usr/java/jdk1.6.0_26/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m > -ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode > -Dhbase.log.dir=/var/log/hbase > -Dhbase.log.file=hbase-hbase-regionserver-hd2-region3.swnet.corp.log ... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Status: Open (was: Patch Available) > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch, HBASE-15137.v5.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Status: Patch Available (was: Open) > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch, HBASE-15137.v5.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Attachment: HBASE-15137.v5.patch addressed checkstyles > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch, HBASE-15137.v5.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177262#comment-15177262
]
Hadoop QA commented on HBASE-15376:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 12s
{color} | {color:red} hbase-hadoop2-compat in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 15s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-server
(total was 140, now 140). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 31s
{color} | {color:green} hbase-hadoop2-compat in the patch passed with JDK
v1.8.0. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-hadoop-compat in the patch passed with JDK
v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 29m 38s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hbase-hadoop2-compat in the patch passed with JDK
v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-hadoop-compat in the patch passed with JDK
v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 30m 47s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 118m
[jira] [Commented] (HBASE-15295) MutateTableAccess.multiMutate() does not get high priority causing a deadlock
[
https://issues.apache.org/jira/browse/HBASE-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177232#comment-15177232
]
Hadoop QA commented on HBASE-15295:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 45s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 46s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 36s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 19s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 22s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-client
(total was 532, now 465). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 38s
{color} | {color:green} hbase-it in the patch passed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 115m 20s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 38s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s
{color} | {color:green} hbase-it in the patch passed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 138m 7s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
25s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 346m 54s {color}
| {color:black} {color} |
\\
\\
||
[jira] [Commented] (HBASE-15379) Fake cells created in read path not implementing SettableSequenceId
[
https://issues.apache.org/jira/browse/HBASE-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177201#comment-15177201
]
ramkrishna.s.vasudevan commented on HBASE-15379:
Ya okie to commit without test also.
> Fake cells created in read path not implementing SettableSequenceId
> ---
>
> Key: HBASE-15379
> URL: https://issues.apache.org/jira/browse/HBASE-15379
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Assignee: Amal Joshy
> Fix For: 2.0.0
>
> Attachments: HBASE-15379-v2.patch, HBASE-15379.patch
>
>
> This issue found by [~appy]. In HBASE-14099 he says,
> I was doing some testing when I hit a weird issue, seems related to this, so
> re-opening it (apologies in advance if it's not). Here's the stack trace
> {noformat}
> java.io.IOException: java.lang.UnsupportedOperationException: Cell is not of
> type org.apache.hadoop.hbase.SettableSequenceId
> at org.apache.hadoop.hbase.CellUtil.setSequenceId(CellUtil.java:923)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.setCurrentCell(StoreFileScanner.java:231)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:389)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:348)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:212)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:1873)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5487)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2577)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2563)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2544)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2534)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6659)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6624)
> at
> org.apache.hadoop.hbase.regionserver.TestWithSingleHRegion.test(TestWithSingleHRegion.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {noformat}
> I think it's because of using changing from KeyValue to a different sub-class
> of {{Cell}}l which doesn't implement {{SettableSequenceId}}
> {noformat}
> -this.startKey =
> KeyValueUtil.createFirstDeleteFamilyOnRow(scan.getStartRow(),
> +this.startKey =
> CellUtil.createFirstDeleteFamilyCellOnRow(scan.getStartRow(),
> {noformat}
> To replicate it, download the attached hfiles somewhere, copy the
> TestWithSingleHRegion class to regionserver tests, change the ROOT_DIR
> appropriately and run it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15379) Fake cells created in read path not implementing SettableSequenceId
[
https://issues.apache.org/jira/browse/HBASE-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177179#comment-15177179
]
Anoop Sam John commented on HBASE-15379:
Looks like.. Ya even the seqId is of no use.. Actually we can avoid the call
to setSeqId on these fake cell at that time.. But as said above, it is added
as a safer side.. Later also we may land in this I fear. IMHO with out test
also we can get this in.
> Fake cells created in read path not implementing SettableSequenceId
> ---
>
> Key: HBASE-15379
> URL: https://issues.apache.org/jira/browse/HBASE-15379
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Assignee: Amal Joshy
> Fix For: 2.0.0
>
> Attachments: HBASE-15379-v2.patch, HBASE-15379.patch
>
>
> This issue found by [~appy]. In HBASE-14099 he says,
> I was doing some testing when I hit a weird issue, seems related to this, so
> re-opening it (apologies in advance if it's not). Here's the stack trace
> {noformat}
> java.io.IOException: java.lang.UnsupportedOperationException: Cell is not of
> type org.apache.hadoop.hbase.SettableSequenceId
> at org.apache.hadoop.hbase.CellUtil.setSequenceId(CellUtil.java:923)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.setCurrentCell(StoreFileScanner.java:231)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:389)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:348)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:212)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:1873)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5487)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2577)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2563)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2544)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2534)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6659)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6624)
> at
> org.apache.hadoop.hbase.regionserver.TestWithSingleHRegion.test(TestWithSingleHRegion.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {noformat}
> I think it's because of using changing from KeyValue to a different sub-class
> of {{Cell}}l which doesn't implement {{SettableSequenceId}}
> {noformat}
> -this.startKey =
> KeyValueUtil.createFirstDeleteFamilyOnRow(scan.getStartRow(),
> +this.startKey =
> CellUtil.createFirstDeleteFamilyCellOnRow(scan.getStartRow(),
> {noformat}
> To replicate it, download the attached hfiles somewhere, copy the
> TestWithSingleHRegion class to
[jira] [Commented] (HBASE-15378) Scanner can not handle a heartbeat message with no results
[ https://issues.apache.org/jira/browse/HBASE-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177173#comment-15177173 ] Phil Yang commented on HBASE-15378: --- The patch can be committed to branch-1.1/1.2/1.3/1 and master. Any ideas? Thanks :) > Scanner can not handle a heartbeat message with no results > -- > > Key: HBASE-15378 > URL: https://issues.apache.org/jira/browse/HBASE-15378 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: HBASE-15378-v1.txt, HBASE-15378-v2.txt, > HBASE-15378-v3.txt > > > When a RS scanner get a TIME_LIMIT_REACHED_MID_ROW state, they will stop > scanning and send back what it has read to client and mark the message as a > heartbeat message. If there is no cell has been read, it will be an empty > response. > However, ClientScanner only handles the situation that the client gets an > empty heartbeat and its cache is not empty. If the cache is empty too, it > will be regarded as end-of-region and open a new scanner for next region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177157#comment-15177157 ] Phil Yang commented on HBASE-15325: --- Thanks Ted, the last patch is to test Yetus environment only so it must fail. I'll draft a new patch after code review done and HBASE-15378 committed to master because two patches may conflict in one line. > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.5.txt, HBASE-15325-v6.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15379) Fake cells created in read path not implementing SettableSequenceId
[
https://issues.apache.org/jira/browse/HBASE-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177155#comment-15177155
]
ramkrishna.s.vasudevan commented on HBASE-15379:
+1 on patch. If possible we can add test case. Better. Not sure on the exact
scenario. Does this happen only during bulk load?
> Fake cells created in read path not implementing SettableSequenceId
> ---
>
> Key: HBASE-15379
> URL: https://issues.apache.org/jira/browse/HBASE-15379
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Assignee: Amal Joshy
> Fix For: 2.0.0
>
> Attachments: HBASE-15379-v2.patch, HBASE-15379.patch
>
>
> This issue found by [~appy]. In HBASE-14099 he says,
> I was doing some testing when I hit a weird issue, seems related to this, so
> re-opening it (apologies in advance if it's not). Here's the stack trace
> {noformat}
> java.io.IOException: java.lang.UnsupportedOperationException: Cell is not of
> type org.apache.hadoop.hbase.SettableSequenceId
> at org.apache.hadoop.hbase.CellUtil.setSequenceId(CellUtil.java:923)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.setCurrentCell(StoreFileScanner.java:231)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:389)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:348)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:212)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:1873)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5487)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2577)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2563)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2544)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2534)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6659)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6624)
> at
> org.apache.hadoop.hbase.regionserver.TestWithSingleHRegion.test(TestWithSingleHRegion.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {noformat}
> I think it's because of using changing from KeyValue to a different sub-class
> of {{Cell}}l which doesn't implement {{SettableSequenceId}}
> {noformat}
> -this.startKey =
> KeyValueUtil.createFirstDeleteFamilyOnRow(scan.getStartRow(),
> +this.startKey =
> CellUtil.createFirstDeleteFamilyCellOnRow(scan.getStartRow(),
> {noformat}
> To replicate it, download the attached hfiles somewhere, copy the
> TestWithSingleHRegion class to regionserver tests, change the ROOT_DIR
> appropriately and run it.
--
This message was sent by Atlassian JIRA
[jira] [Resolved] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15329.
---
Resolution: Fixed
Hadoop Flags: Reviewed
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177136#comment-15177136
]
Heng Chen commented on HBASE-15329:
---
Actually, it is not a problem. If your url parameter 'name' is not a valid
table name in cluster, it will return 500 like below. And the table's name is
only valid with [a-zA-Z_0-9-.]. But i will still push it just for unified
code with XSS dealing.
{code}
HTTP ERROR 500
Problem accessing /table.jsp. Reason:
Illegal character code:38, <&> at 0. User-space table qualifiers can only
contain 'alphanumeric characters': i.e. [a-zA-Z_0-9-.]:
scriptcluster_test
Caused by:
java.lang.IllegalArgumentException: Illegal character code:38, <&> at 0.
User-space table qualifiers can only contain 'alphanumeric characters': i.e.
[a-zA-Z_0-9-.]: scriptcluster_test
at
org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:201)
at
org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:149)
at org.apache.hadoop.hbase.TableName.(TableName.java:322)
at
org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:358)
at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:445)
at
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:117)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1354)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{code}
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177120#comment-15177120 ] Ted Yu commented on HBASE-15325: Phil: I started a test run using latest Yetus build. This should resolve module dependency issue. > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.5.txt, HBASE-15325-v6.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177117#comment-15177117
]
Heng Chen commented on HBASE-15329:
---
Let me check it locally, and push it to master.
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15329:
--
Fix Version/s: 2.0.0
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177115#comment-15177115
]
Heng Chen commented on HBASE-15329:
---
LGTM. +1
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15329:
--
Assignee: Samir Ahmic
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15376:
--
Attachment: HBASE-15376_v1.patch
> ScanNext metric is size-based while every other per-operation metric is time
> based
> --
>
> Key: HBASE-15376
> URL: https://issues.apache.org/jira/browse/HBASE-15376
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
> Attachments: HBASE-15376.patch, HBASE-15376_v1.patch
>
>
> We have per-operation metrics for {{Get}}, {{Mutate}}, {{Delete}},
> {{Increment}}, and {{ScanNext}}.
> The metrics are emitted like:
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> ...
> "ScanNext_num_ops" : 194705,
> "ScanNext_min" : 0,
> "ScanNext_max" : 18441,
> "ScanNext_mean" : 7468.274651395701,
> "ScanNext_median" : 583.0,
> "ScanNext_75th_percentile" : 583.0,
> "ScanNext_95th_percentile" : 13481.0,
> "ScanNext_99th_percentile" : 13481.0,
> {code}
> The problem is that all of Get,Mutate,Delete,Increment,Append,Replay are time
> based tracking how long the operation ran, while ScanNext is tracking
> returned response sizes (returned cell-sizes to be exact). Obviously, this is
> very confusing and you would only know this subtlety if you read the metrics
> collection code.
> Not sure how useful is the ScanNext metric as it is today. We can deprecate
> it, and introduce a time based one to keep track of scan request latencies.
> ps. Shamelessly using the parent jira (since these seem relavant).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15325: -- Attachment: HBASE-15325-v6.5.txt > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.5.txt, HBASE-15325-v6.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15384) Avoid using '/tmp' directory in TestBulkLoad
Heng Chen created HBASE-15384: - Summary: Avoid using '/tmp' directory in TestBulkLoad Key: HBASE-15384 URL: https://issues.apache.org/jira/browse/HBASE-15384 Project: HBase Issue Type: Sub-task Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177087#comment-15177087 ] Duo Zhang commented on HBASE-15368: --- Fine, the code is based on our version. I have to move to use your DateTieredCompactionPolicy before upload it here. One or two days. To write multiple files is simple, we can follow the stripe compaction way. One problem is we will write different StoreFiles with same seqId, so we need to add the timestamp to StoreFile comparator. > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177086#comment-15177086 ] Phil Yang commented on HBASE-15325: --- [~tedyu] I add some logs in my last patch, some in hbase-server module and some in hbase-client module. But I can only find logs in hbase-server and can't find any logs belongs to hbase-client. Is it possible that the build ran the original hbase-client codes, not the patched one since TestPartialResultsFromClientSide is in hbase-server module? Not only logs but the behavior of client is not as expected. Thanks. > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177083#comment-15177083 ] Ted Yu commented on HBASE-14801: [~jmhsieh]: Looks like Zhan has addressed the compatibility issue. Mind taking another look ? > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch, HBASE-14801-5.patch, > HBASE-14801-6.patch, HBASE-14801-7.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177078#comment-15177078 ] Francis Liu commented on HBASE-6721: I forgot to rebase when I did rebase had a minor conflict. So I uploaded a rebased patch. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, immediateAssignments Sequence Diagram.svg, > randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, > roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177075#comment-15177075 ] Francis Liu commented on HBASE-6721: Thanks [~eclark]! [~enis] doesn't the activation property actually only tests for the presence of a propery and ignores the value: http://maven.apache.org/guides/introduction/introduction-to-profiles.html So in your case 'mvn install -Drsgroup=false' would still enable the rsgroup profile? > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, immediateAssignments Sequence Diagram.svg, > randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, > roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177071#comment-15177071 ] Clara Xiong commented on HBASE-15368: - Again, would you mind opening a new jira for major compactions with tiered multiple output and share your prototype? It is on our design spec and TODO list for Date Tiered compaction. > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-6721: --- Attachment: hbase-6721-v27.patch > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, immediateAssignments Sequence Diagram.svg, > randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, > roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177066#comment-15177066 ] Clara Xiong commented on HBASE-15368: - I think we should wait until we have an alternative implementation people actually use to have an abstraction. I also went through both http://www.datastax.com/dev/blog/datetieredcompactionstrategy and https://labs.spotify.com/2014/12/18/date-tiered-compaction/ to understand Cassandra's design and code. There is no practical way to do relative windows as Dave pointed out. > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[
https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177062#comment-15177062
]
Hadoop QA commented on HBASE-15137:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 29s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 30s
{color} | {color:red} Patch generated 18 new checkstyle issues in hbase-client
(total was 40, now 29). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 32s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 37s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 29s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 0s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 112m 12s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
3s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 319m 4s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| |
[jira] [Commented] (HBASE-15379) Fake cells created in read path not implementing SettableSequenceId
[
https://issues.apache.org/jira/browse/HBASE-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177061#comment-15177061
]
Heng Chen commented on HBASE-15379:
---
Could we add a testcase for this situation?
> Fake cells created in read path not implementing SettableSequenceId
> ---
>
> Key: HBASE-15379
> URL: https://issues.apache.org/jira/browse/HBASE-15379
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Assignee: Amal Joshy
> Fix For: 2.0.0
>
> Attachments: HBASE-15379-v2.patch, HBASE-15379.patch
>
>
> This issue found by [~appy]. In HBASE-14099 he says,
> I was doing some testing when I hit a weird issue, seems related to this, so
> re-opening it (apologies in advance if it's not). Here's the stack trace
> {noformat}
> java.io.IOException: java.lang.UnsupportedOperationException: Cell is not of
> type org.apache.hadoop.hbase.SettableSequenceId
> at org.apache.hadoop.hbase.CellUtil.setSequenceId(CellUtil.java:923)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.setCurrentCell(StoreFileScanner.java:231)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:389)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:348)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:212)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:1873)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5487)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2577)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2563)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2544)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2534)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6659)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6624)
> at
> org.apache.hadoop.hbase.regionserver.TestWithSingleHRegion.test(TestWithSingleHRegion.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> {noformat}
> I think it's because of using changing from KeyValue to a different sub-class
> of {{Cell}}l which doesn't implement {{SettableSequenceId}}
> {noformat}
> -this.startKey =
> KeyValueUtil.createFirstDeleteFamilyOnRow(scan.getStartRow(),
> +this.startKey =
> CellUtil.createFirstDeleteFamilyCellOnRow(scan.getStartRow(),
> {noformat}
> To replicate it, download the attached hfiles somewhere, copy the
> TestWithSingleHRegion class to regionserver tests, change the ROOT_DIR
> appropriately and run it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15378) Scanner can not handle a heartbeat message with no results
[ https://issues.apache.org/jira/browse/HBASE-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177047#comment-15177047 ] Phil Yang commented on HBASE-15378: --- I think these fails are unrelated(seems environment issue?) > Scanner can not handle a heartbeat message with no results > -- > > Key: HBASE-15378 > URL: https://issues.apache.org/jira/browse/HBASE-15378 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: HBASE-15378-v1.txt, HBASE-15378-v2.txt, > HBASE-15378-v3.txt > > > When a RS scanner get a TIME_LIMIT_REACHED_MID_ROW state, they will stop > scanning and send back what it has read to client and mark the message as a > heartbeat message. If there is no cell has been read, it will be an empty > response. > However, ClientScanner only handles the situation that the client gets an > empty heartbeat and its cache is not empty. If the cache is empty too, it > will be regarded as end-of-region and open a new scanner for next region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177043#comment-15177043 ] Duo Zhang commented on HBASE-15368: --- OK, I checked cassandra's code, it is using a fixed window :) http://www.datastax.com/dev/blog/datetieredcompactionstrategy I was misleading by the pictures here. So retain the WindowFactory and commit it? Or wait for someone implement a calendar based window factory? > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177041#comment-15177041
]
Heng Chen commented on HBASE-15376:
---
{code}
The https://hbase.apache.org/book.html#hbase.versioning page says that there is
no operational compatibility between minor versions, so according to the
previously agreed on guidelines, we can (but do not need to) in fact change the
name of the metrics in 1.3+.
{code}
You are right. So we have no plan to backport it to branch-1.0+ (just add
metrics for scan time) ?
> ScanNext metric is size-based while every other per-operation metric is time
> based
> --
>
> Key: HBASE-15376
> URL: https://issues.apache.org/jira/browse/HBASE-15376
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
> Attachments: HBASE-15376.patch
>
>
> We have per-operation metrics for {{Get}}, {{Mutate}}, {{Delete}},
> {{Increment}}, and {{ScanNext}}.
> The metrics are emitted like:
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> ...
> "ScanNext_num_ops" : 194705,
> "ScanNext_min" : 0,
> "ScanNext_max" : 18441,
> "ScanNext_mean" : 7468.274651395701,
> "ScanNext_median" : 583.0,
> "ScanNext_75th_percentile" : 583.0,
> "ScanNext_95th_percentile" : 13481.0,
> "ScanNext_99th_percentile" : 13481.0,
> {code}
> The problem is that all of Get,Mutate,Delete,Increment,Append,Replay are time
> based tracking how long the operation ran, while ScanNext is tracking
> returned response sizes (returned cell-sizes to be exact). Obviously, this is
> very confusing and you would only know this subtlety if you read the metrics
> collection code.
> Not sure how useful is the ScanNext metric as it is today. We can deprecate
> it, and introduce a time based one to keep track of scan request latencies.
> ps. Shamelessly using the parent jira (since these seem relavant).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15345) add branch-1.3 post-commit builds
[ https://issues.apache.org/jira/browse/HBASE-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov resolved HBASE-15345. - Resolution: Fixed > add branch-1.3 post-commit builds > - > > Key: HBASE-15345 > URL: https://issues.apache.org/jira/browse/HBASE-15345 > Project: HBase > Issue Type: Sub-task > Components: build >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Ted Yu > Fix For: 1.3.0 > > > point hbase-1.3 build to branch-1.3, add hbase-1.4 and point it to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15345) add branch-1.3 post-commit builds
[ https://issues.apache.org/jira/browse/HBASE-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15345: Description: point hbase-1.3 build to branch-1.3, add hbase-1.4 and point it to branch-1. > add branch-1.3 post-commit builds > - > > Key: HBASE-15345 > URL: https://issues.apache.org/jira/browse/HBASE-15345 > Project: HBase > Issue Type: Sub-task > Components: build >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Ted Yu > Fix For: 1.3.0 > > > point hbase-1.3 build to branch-1.3, add hbase-1.4 and point it to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177028#comment-15177028 ] Sean Busbey commented on HBASE-15370: - This looks like a fine email for starting the thread. > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch, > HBASE-15370-branch-1.v2.patch, mob-cmmits.txt > > > MOB feature was integrated to master branch half a year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15345) add branch-1.3 post-commit builds
[ https://issues.apache.org/jira/browse/HBASE-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-15345: --- Assignee: Ted Yu (was: Mikhail Antonov) Was done by [~te...@apache.org] (thanks!) > add branch-1.3 post-commit builds > - > > Key: HBASE-15345 > URL: https://issues.apache.org/jira/browse/HBASE-15345 > Project: HBase > Issue Type: Sub-task > Components: build >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Ted Yu > Fix For: 1.3.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177025#comment-15177025 ] Dave Latham commented on HBASE-15368: - Agreed, no need for an abstraction unless we expect an alternate implementation. I still think a calendar based alternative would be reasonable if anyone wanted it. Cassandra's DTCS also uses fixed windows, not relative ones (see for example the secton "How DTCS works" at https://labs.spotify.com/2014/12/18/date-tiered-compaction/ ). > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15374) Avoid using '/tmp' directory in our unit tests
[ https://issues.apache.org/jira/browse/HBASE-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176984#comment-15176984 ] Duo Zhang commented on HBASE-15374: --- Yes, but TestBulkLoad does not write data, it only uses "/tmp/does_not_exist" as a place holder. But yes we could fix it anyway. Can you please open a issue and preapre a patch for it [~chenheng]? Thanks. > Avoid using '/tmp' directory in our unit tests > -- > > Key: HBASE-15374 > URL: https://issues.apache.org/jira/browse/HBASE-15374 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: Duo Zhang > > I'm used to run unit tests on a remote server before summit a large patch > because sometimes a testcase failure can not be reproduced locally. > Recently [~yangzhe1991] and me share a remote server with a different > account. We found that some of the unit tests write to '/tmp' with a fixed > name and do not clean the file after test. This cause that the unit test can > only be succeeded for one person... > Open an umbrella issue to address this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176955#comment-15176955 ] Duo Zhang commented on HBASE-15368: --- The important thing is the abstraction I think... But it is strange to abstract a factory but only has one implementation, so I add a implementation that just follow cassandra's DTCS. It is released in a production and the compaction logic seems much simpler(compact if a window contains more than 4 files). I think we could optimize it later when we collect some performance data? We have RatioBasedCompactionPolicy, then ExploringCompactionPolicy, and may still introduce new compaction policies. What do you think? Thanks. > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15383) Load distribute across secondary read replicas for meta
Ashu Pachauri created HBASE-15383: - Summary: Load distribute across secondary read replicas for meta Key: HBASE-15383 URL: https://issues.apache.org/jira/browse/HBASE-15383 Project: HBase Issue Type: Improvement Components: Client Reporter: Ashu Pachauri Assignee: Ashu Pachauri Fix For: 2.0.0, 1.3.0, 1.2.0 Right now, we always hit the primary replica for meta and fallback to the secondary replicas in case of a timeout. This can hamper performance in scenarios where meta becomes a hot region e.g. cluster ramp up..clients dropping connections etc. It's good to have a load distribution approach on meta's secondary replicas with fallback to primary if we read stale data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176938#comment-15176938
]
Enis Soztutar commented on HBASE-6721:
--
I skimmed the patch looking around the changes, looks good. One minor nit is
that instead of {{mvn install -Dskip-rsgroup=false}} which is double negation,
we can do {{mvn install -Drsgroup=true}}.
Let's get a hadoopqa run. After we address the reported issues, I'll do the
commit.
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch,
> immediateAssignments Sequence Diagram.svg, randomAssignment Sequence
> Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment
> Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[
https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176932#comment-15176932
]
Hadoop QA commented on HBASE-14801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 8s
{color} | {color:red} hbase-spark in master has 66 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 19s
{color} | {color:red} hbase-spark introduced 7 new FindBugs issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 52s
{color} | {color:red} hbase-spark-jdk1.8.0 with JDK v1.8.0 generated 1 new
issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 2m 11s
{color} | {color:red} hbase-spark-jdk1.7.0_79 with JDK v1.7.0_79 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} |
[jira] [Commented] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176876#comment-15176876
]
Enis Soztutar commented on HBASE-15376:
---
Can you also change the ScanNext histograms to be size-based by calling
newSizeHistogram()
{code}
scanNextHisto = getMetricsRegistry().newTimeHistogram(SCAN_NEXT_KEY);
{code}
> ScanNext metric is size-based while every other per-operation metric is time
> based
> --
>
> Key: HBASE-15376
> URL: https://issues.apache.org/jira/browse/HBASE-15376
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
> Attachments: HBASE-15376.patch
>
>
> We have per-operation metrics for {{Get}}, {{Mutate}}, {{Delete}},
> {{Increment}}, and {{ScanNext}}.
> The metrics are emitted like:
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> ...
> "ScanNext_num_ops" : 194705,
> "ScanNext_min" : 0,
> "ScanNext_max" : 18441,
> "ScanNext_mean" : 7468.274651395701,
> "ScanNext_median" : 583.0,
> "ScanNext_75th_percentile" : 583.0,
> "ScanNext_95th_percentile" : 13481.0,
> "ScanNext_99th_percentile" : 13481.0,
> {code}
> The problem is that all of Get,Mutate,Delete,Increment,Append,Replay are time
> based tracking how long the operation ran, while ScanNext is tracking
> returned response sizes (returned cell-sizes to be exact). Obviously, this is
> very confusing and you would only know this subtlety if you read the metrics
> collection code.
> Not sure how useful is the ScanNext metric as it is today. We can deprecate
> it, and introduce a time based one to keep track of scan request latencies.
> ps. Shamelessly using the parent jira (since these seem relavant).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176862#comment-15176862 ] Elliott Clark commented on HBASE-6721: -- Oh I was looking at the draft rather than the last version. I see that lots of stuff has been fixed. Thanks. Ok i'm officially -0. this patch. Not minus one. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176842#comment-15176842
]
Francis Liu commented on HBASE-6721:
{quote}
So then the last part is getting better naming
{quote}
Are you referring to the rename from Group -> RSGroup? Then the just uploaded
patch should've addressed it. Let me know if I missed anything. If not let me
know what I'm missing?
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch,
> immediateAssignments Sequence Diagram.svg, randomAssignment Sequence
> Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment
> Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176837#comment-15176837
]
Francis Liu commented on HBASE-6721:
Attached patch which is pretty much done except for the negative port concern.
{quote}
Yeah, address the groups naming thing.
{quote}
Done. Change classes, packages, etc to RSGroups.
{quote}
Address the ServerName questions, breaking compat is not an option.
{quote}
It seems clear now the serverName form I fixed in TestMasterStatusServlet is
indeed a typo. So the current patch can still support the old form. Tho it
cannot support negative port numbers due to port restrictions. Let me know if
this is not acceptable.
{quote}
Don't go around the Metrics to get to jmx
{quote}
Removed JMX this code from the base patch and filed HBASE-15382
{quote}
Don't create a new protobuf class when not needed
{quote}
Done
{quote}
Move it into a different module with a profile that doesn't include it.
{quote}
Done. Tho RSGroup protos are still in hbase-protocol and there's one POJO
(RSGroupInfo) that had to stay in hbase-common as it is needed by the RSGroup
related MasterObserver CP methods. RSGroup shell is also still in hbase-shell.
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch,
> immediateAssignments Sequence Diagram.svg, randomAssignment Sequence
> Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment
> Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176824#comment-15176824 ] Elliott Clark commented on HBASE-6721: -- The negative port numbers are unlikely to cause any errors so I would say we can ignore that. So then the last part is getting better naming for things. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15382) Expose regionserver metadata (ie groups, tables, servers) via JMX
Francis Liu created HBASE-15382: --- Summary: Expose regionserver metadata (ie groups, tables, servers) via JMX Key: HBASE-15382 URL: https://issues.apache.org/jira/browse/HBASE-15382 Project: HBase Issue Type: Sub-task Reporter: Francis Liu Assignee: Francis Liu This feature was removed from the base patch. So we can come up with a proper interface for other components to use as well, as directly accessing jmx is not an option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14801: --- Attachment: HBASE-14801-7.patch > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch, HBASE-14801-5.patch, > HBASE-14801-6.patch, HBASE-14801-7.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-6721: --- Attachment: hbase-6721-v26.patch > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176798#comment-15176798
]
Francis Liu commented on HBASE-6721:
{quote}
HBASE-1502 is committed around 0.92 timeframe, so I am not sure whether we will
have any such server name strings around anymore.
{quote}
Thanks for pointing that out [~enis]. This is a clear indication that the
ServerNameform in TestMasterStatusServlet is a typo 'rootserver:123,12345' as
it is neither the old form or the new form. Then that leaves wether negative
port numbers are a breaking change or not? As HostAndPort doesn't allow it.
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26_draft1.patch, immediateAssignments
> Sequence Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment
> Sequence Diagram.svg, roundRobinAssignment Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15376) ScanNext metric is size-based while every other per-operation metric is time based
[
https://issues.apache.org/jira/browse/HBASE-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176780#comment-15176780
]
Enis Soztutar commented on HBASE-15376:
---
I was checking whether we can change the metric names and/or remove metrics in
minor or major versions. The
https://hbase.apache.org/book.html#hbase.versioning page says that there is no
operational compatibility between minor versions, so according to the
previously agreed on guidelines, we can (but do not need to) in fact change the
name of the metrics in 1.3+.
Coming to this patch, I think we should rename the metric name "ScanNext" to
"ScanSize" and add "ScanTime" as the new metric. We should do this for the
per-region metric as well. cc [~eclark], [~saint@gmail.com].
> ScanNext metric is size-based while every other per-operation metric is time
> based
> --
>
> Key: HBASE-15376
> URL: https://issues.apache.org/jira/browse/HBASE-15376
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
> Attachments: HBASE-15376.patch
>
>
> We have per-operation metrics for {{Get}}, {{Mutate}}, {{Delete}},
> {{Increment}}, and {{ScanNext}}.
> The metrics are emitted like:
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> ...
> "ScanNext_num_ops" : 194705,
> "ScanNext_min" : 0,
> "ScanNext_max" : 18441,
> "ScanNext_mean" : 7468.274651395701,
> "ScanNext_median" : 583.0,
> "ScanNext_75th_percentile" : 583.0,
> "ScanNext_95th_percentile" : 13481.0,
> "ScanNext_99th_percentile" : 13481.0,
> {code}
> The problem is that all of Get,Mutate,Delete,Increment,Append,Replay are time
> based tracking how long the operation ran, while ScanNext is tracking
> returned response sizes (returned cell-sizes to be exact). Obviously, this is
> very confusing and you would only know this subtlety if you read the metrics
> collection code.
> Not sure how useful is the ScanNext metric as it is today. We can deprecate
> it, and introduce a time based one to keep track of scan request latencies.
> ps. Shamelessly using the parent jira (since these seem relavant).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[
https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176752#comment-15176752
]
Hadoop QA commented on HBASE-15321:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 38s
{color} | {color:red} Patch generated 2 new checkstyle issues in hbase-server
(total was 253, now 255). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 57s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 42s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 125m 59s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 288m 17s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests |
hadoop.hbase.security.access.TestScanEarlyTermination |
| JDK v1.8.0_72 Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| |
[jira] [Commented] (HBASE-13683) Doc HBase and G1GC
[ https://issues.apache.org/jira/browse/HBASE-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176732#comment-15176732 ] Andrew Purtell commented on HBASE-13683: It went well. We are making the switch. Let me ask the guys on our perf team if they're interested in blogging about it. > Doc HBase and G1GC > -- > > Key: HBASE-13683 > URL: https://issues.apache.org/jira/browse/HBASE-13683 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack > > Add section to refguide on running G1GC with HBase. There is the intel talk > at hbasecon with nice pictures and healthy recommendations, there is our > [~esteban]'s recent experience running G1GC, and the mighty [~bbeaudreault] > dumped a bunch of helpful advice in the mailing list just now: > http://search-hadoop.com/m/YGbbupEDoKTrDo/%2522How+to+know+the+root+reason+to+cause+RegionServer+OOM%253F%2522=Re+How+to+know+the+root+reason+to+cause+RegionServer+OOM+ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15295) MutateTableAccess.multiMutate() does not get high priority causing a deadlock
[
https://issues.apache.org/jira/browse/HBASE-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-15295:
--
Attachment: hbase-15295_v5.patch
reattach for qa.
> MutateTableAccess.multiMutate() does not get high priority causing a deadlock
> -
>
> Key: HBASE-15295
> URL: https://issues.apache.org/jira/browse/HBASE-15295
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4
>
> Attachments: hbase-15295_v1.patch, hbase-15295_v1.patch,
> hbase-15295_v2.patch, hbase-15295_v3.patch, hbase-15295_v4.patch,
> hbase-15295_v5.patch, hbase-15295_v5.patch
>
>
> We have seen this in a cluster with Phoenix secondary indexes leading to a
> deadlock. All handlers are busy waiting on the index updates to finish:
> {code}
> "B.defaultRpcServer.handler=50,queue=0,port=16020" #91 daemon prio=5
> os_prio=0 tid=0x7f29f64ba000 nid=0xab51 waiting on condition
> [0x7f29a8762000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000124f1d5c8> (a
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:66)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
> at
> org.apache.phoenix.hbase.index.write.ParallelWriterIndexCommitter.write(ParallelWriterIndexCommitter.java:194)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.write(IndexWriter.java:179)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.writeAndKillYourselfOnFailure(IndexWriter.java:144)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.writeAndKillYourselfOnFailure(IndexWriter.java:134)
> at
> org.apache.phoenix.hbase.index.Indexer.doPostWithExceptions(Indexer.java:457)
> at org.apache.phoenix.hbase.index.Indexer.doPost(Indexer.java:406)
> at
> org.apache.phoenix.hbase.index.Indexer.postBatchMutate(Indexer.java:401)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$36.call(RegionCoprocessorHost.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1705)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postBatchMutate(RegionCoprocessorHost.java:1002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3162)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2801)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2743)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2031)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> And the index region is trying to split, and is trying to do a meta update:
> {code}
> "regionserver//10.132.70.191:16020-splits-1454693389669" #1779
> prio=5 os_prio=0 tid=0x7f29e149c000 nid=0x5107 in Object.wait()
> [0x7f1f136d6000]
>java.lang.Thread.State: TIMED_WAITING (on object
[jira] [Commented] (HBASE-15160) Put back HFile's HDFS op latency sampling code and add metrics for monitoring
[
https://issues.apache.org/jira/browse/HBASE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176718#comment-15176718
]
Enis Soztutar commented on HBASE-15160:
---
bq. Yes, already made the change in the latest patch.
Ok, I was looking at the following for why we are not using a histogram for
this:
{code}
+ private static final BlockingQueue fsReadLatenciesNanos =
+ new ArrayBlockingQueue(LATENCY_BUFFER_SIZE);
+ private static final BlockingQueue fsWriteLatenciesNanos =
+ new ArrayBlockingQueue(LATENCY_BUFFER_SIZE);
{code}
For every RPC and for every operation (get, etc), we already increment counters
or histograms directly inline, rather than keeping track of individual points
like the one in the patch and bulk updating the histograms frequently. Since
num gets > num fs operations in theory, doing the counter updates inline should
not be a perf regression. This is of course to be verified if possible.
One other thing is that instead of using the histogram inline (which is based
on FastLongHistogram / Counters and high perf counters) we are using a
BlockingQueue which is using a RWLock and in-theory more costly. So doing this
indirect way maybe even worse than doing inline updates.
> Put back HFile's HDFS op latency sampling code and add metrics for monitoring
> -
>
> Key: HBASE-15160
> URL: https://issues.apache.org/jira/browse/HBASE-15160
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.1.2
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15160.patch, HBASE-15160_v2.patch,
> HBASE-15160_v3.patch
>
>
> In HBASE-11586 all HDFS op latency sampling code, including fsReadLatency,
> fsPreadLatency and fsWriteLatency, have been removed. There was some
> discussion about putting them back in a new JIRA but never happened.
> According to our experience, these metrics are useful to judge whether issue
> lies on HDFS when slow request occurs, so we propose to put them back in this
> JIRA, and add the metrics for monitoring as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[
https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176673#comment-15176673
]
Hadoop QA commented on HBASE-14801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 41s
{color} | {color:red} hbase-spark in master has 66 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 13s
{color} | {color:red} Patch generated 2 new checkstyle issues in hbase-spark
(total was 2, now 4). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 55s
{color} | {color:red} hbase-spark introduced 7 new FindBugs issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 9s
{color} | {color:red} hbase-spark-jdk1.8.0_74 with JDK v1.8.0_74 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 24s
{color} | {color:red} hbase-spark-jdk1.7.0_95 with JDK v1.7.0_95 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color}
[jira] [Commented] (HBASE-13683) Doc HBase and G1GC
[ https://issues.apache.org/jira/browse/HBASE-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176657#comment-15176657 ] Bryan Beaudreault commented on HBASE-13683: --- We will be publishing a blog post very soon, which has some lessons learned from some very thorough G1 testing/tuning we've recently done (based on our use case at least). I'll update this ticket when we have that available, which you guys can feel free to use as you will > Doc HBase and G1GC > -- > > Key: HBASE-13683 > URL: https://issues.apache.org/jira/browse/HBASE-13683 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack > > Add section to refguide on running G1GC with HBase. There is the intel talk > at hbasecon with nice pictures and healthy recommendations, there is our > [~esteban]'s recent experience running G1GC, and the mighty [~bbeaudreault] > dumped a bunch of helpful advice in the mailing list just now: > http://search-hadoop.com/m/YGbbupEDoKTrDo/%2522How+to+know+the+root+reason+to+cause+RegionServer+OOM%253F%2522=Re+How+to+know+the+root+reason+to+cause+RegionServer+OOM+ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15370: --- Attachment: mob-cmmits.txt mob-cmmits.txt contains the commit hashes related to MOB with most recent commit first. > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch, > HBASE-15370-branch-1.v2.patch, mob-cmmits.txt > > > MOB feature was integrated to master branch half a year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13683) Doc HBase and G1GC
[ https://issues.apache.org/jira/browse/HBASE-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176615#comment-15176615 ] Nick Dimiduk commented on HBASE-13683: -- How'd the evaluation go [~apurtell]? Anyone else have some pearls of wisdom to distill here? Maybe [~vrodionov] or [~eclark]? > Doc HBase and G1GC > -- > > Key: HBASE-13683 > URL: https://issues.apache.org/jira/browse/HBASE-13683 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack > > Add section to refguide on running G1GC with HBase. There is the intel talk > at hbasecon with nice pictures and healthy recommendations, there is our > [~esteban]'s recent experience running G1GC, and the mighty [~bbeaudreault] > dumped a bunch of helpful advice in the mailing list just now: > http://search-hadoop.com/m/YGbbupEDoKTrDo/%2522How+to+know+the+root+reason+to+cause+RegionServer+OOM%253F%2522=Re+How+to+know+the+root+reason+to+cause+RegionServer+OOM+ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15130) Backport 0.98 Scan different TimeRange for each column family
[ https://issues.apache.org/jira/browse/HBASE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176612#comment-15176612 ] Andrew Purtell commented on HBASE-15130: KeyValueScanner is annotated Private. Worth considering moving away from this on the Phoenix side somehow? Do we need to promote this to LimitedPrivate? [~giacomotaylor] > Backport 0.98 Scan different TimeRange for each column family > -- > > Key: HBASE-15130 > URL: https://issues.apache.org/jira/browse/HBASE-15130 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 0.98.17 >Reporter: churro morales >Assignee: churro morales > Fix For: 0.98.18 > > Attachments: HBASE-15130-0.98.patch, HBASE-15130-0.98.v1.patch, > HBASE-15130-0.98.v1.patch, HBASE-15130-0.98.v2.patch, > HBASE-15130-0.98.v3.patch > > > branch 98 version backport for HBASE-14355 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-15130) Backport 0.98 Scan different TimeRange for each column family
[
https://issues.apache.org/jira/browse/HBASE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-15130:
Crap, looking at this again I realize this breaks the 0.98 version of Phoenix.
{noformat}
[ERROR]
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/covered/data/IndexMemStore.java:[310,5]
method does not override or implement a method from a supertype
[ERROR]
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/FilteredKeyValueScanner.java:[37,8]
org.apache.phoenix.hbase.index.scanner.FilteredKeyValueScanner is not abstract
and does not override abstract method
shouldUseScanner(org.apache.hadoop.hbase.client.Scan,org.apache.hadoop.hbase.regionserver.Store,long)
in org.apache.hadoop.hbase.regionserver.KeyValueScanner
[ERROR]
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/FilteredKeyValueScanner.java:[126,5]
method does not override or implement a method from a supertype
{noformat}
We need to commit and addendum or revert and redo.
> Backport 0.98 Scan different TimeRange for each column family
> --
>
> Key: HBASE-15130
> URL: https://issues.apache.org/jira/browse/HBASE-15130
> Project: HBase
> Issue Type: Bug
> Components: Client, regionserver, Scanners
>Affects Versions: 0.98.17
>Reporter: churro morales
>Assignee: churro morales
> Fix For: 0.98.18
>
> Attachments: HBASE-15130-0.98.patch, HBASE-15130-0.98.v1.patch,
> HBASE-15130-0.98.v1.patch, HBASE-15130-0.98.v2.patch,
> HBASE-15130-0.98.v3.patch
>
>
> branch 98 version backport for HBASE-14355
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15370: --- Description: MOB feature was integrated to master branch half a year ago. Since then there have been bug fixes which stabilize the feature. Some customers have been using it at PB scale. Here is discussion thread on dev mailing list: http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ This issue is to backport MOB feature to branch-1. was: MOB feature was integrated to master branch 1 year ago. Since then there have been bug fixes which stabilize the feature. Some customers have been using it at PB scale. Here is discussion thread on dev mailing list: http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ This issue is to backport MOB feature to branch-1. > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch, > HBASE-15370-branch-1.v2.patch > > > MOB feature was integrated to master branch half a year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14801: --- Attachment: HBASE-14801-6.patch > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch, HBASE-14801-5.patch, > HBASE-14801-6.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Status: Open (was: Patch Available) > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Attachment: HBASE-15137.v4.patch moved exception handling to client exceptions util, reformatted PFFE class. > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15137: Status: Patch Available (was: Open) > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch, HBASE-15137.v4.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176495#comment-15176495
]
Hadoop QA commented on HBASE-15370:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 53 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 37s
{color} | {color:green} branch-1 passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 26s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
38s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
49s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 9m 3s
{color} | {color:red} branch/. no findbugs output file
(./target/findbugsXml.xml) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 13s
{color} | {color:red} branch/hbase-it no findbugs output file
(hbase-it/target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 50s
{color} | {color:green} branch-1 passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 5s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 11s
{color} | {color:red} root-jdk1.8.0_72 with JDK v1.8.0_72 generated 6 new
issues (was 17, now 17). {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 11s
{color} | {color:red} hbase-server-jdk1.8.0_72 with JDK v1.8.0_72 generated 6
new issues (was 6, now 6). {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 59s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 20m 11s
{color} | {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 6 new
issues (was 17, now 17). {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 20m 11s
{color} | {color:red} hbase-server-jdk1.7.0_95 with JDK v1.7.0_95 generated 6
new issues (was 6, now 6). {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 59s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 52s
{color} | {color:red} Patch generated 60 new checkstyle issues in root (total
was 251, now 305). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 10s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-client
(total was 7, now 8). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 7s
{color} | {color:red} Patch generated 2 new checkstyle issues in
hbase-hadoop2-compat (total was 1, now 3). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 16s
{color} | {color:red} Patch generated 57 new checkstyle issues in hbase-server
(total was 240, now 291). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
1s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 12s
{color} | {color:red} The applied patch generated 39 new rubocop issues (total
was 678, now 708). {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m 5s
{color} | {color:red} The applied patch generated 70 new ruby-lint issues
(total was 495, now
[jira] [Updated] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15370: --- Attachment: HBASE-15370-branch-1.v2.patch Patch v2 tries to fix checkstyle warnings. > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch, > HBASE-15370-branch-1.v2.patch > > > MOB feature was integrated to master branch 1 year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176454#comment-15176454 ] Ted Yu commented on HBASE-15325: In the JDK 7 run: Caused by: java.lang.RuntimeException: Error while running command to get file permissions : ExitCodeException exitCode=127: /bin/ls: error while loading shared libraries: libdl.so.2: failed to map segment from shared object: Permission denied Environment may have something to do with test failures. > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt, HBASE-15325-v1.txt, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15137) CallTimeoutException and CallQueueTooBigException should trigger PFFE
[ https://issues.apache.org/jira/browse/HBASE-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176385#comment-15176385 ] Elliott Clark commented on HBASE-15137: --- Should the exception type stuff be moved into the ClientExceptionsUtil? Can some of the checkstyles be cleaned up? > CallTimeoutException and CallQueueTooBigException should trigger PFFE > - > > Key: HBASE-15137 > URL: https://issues.apache.org/jira/browse/HBASE-15137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15137.mantonov.patch, HBASE-15137.patch, > HBASE-15137.v3.patch > > > If a region server is backed up enough that lots of calls are timing out then > we should think about treating a server as failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction
[ https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176373#comment-15176373 ] Clara Xiong commented on HBASE-15181: - Results from our production is added at https://docs.google.com/document/d/1GqRtQZMMkTEWOijZc8UCTqhACNmdxBSjtAQSYIWsmGU/edit# > A simple implementation of date based tiered compaction > --- > > Key: HBASE-15181 > URL: https://issues.apache.org/jira/browse/HBASE-15181 > Project: HBase > Issue Type: New Feature > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15181-0.98-ADD.patch, HBASE-15181-0.98.patch, > HBASE-15181-0.98.v4.patch, HBASE-15181-98.patch, HBASE-15181-ADD.patch, > HBASE-15181-branch-1.patch, HBASE-15181-master-v1.patch, > HBASE-15181-master-v2.patch, HBASE-15181-master-v3.patch, > HBASE-15181-master-v4.patch, HBASE-15181-v1.patch, HBASE-15181-v2.patch > > > This is a simple implementation of date-based tiered compaction similar to > Cassandra's for the following benefits: > 1. Improve date-range-based scan by structuring store files in date-based > tiered layout. > 2. Reduce compaction overhead. > 3. Improve TTL efficiency. > Perfect fit for the use cases that: > 1. has mostly date-based date write and scan and a focus on the most recent > data. > 2. never or rarely deletes data. > Out-of-order writes are handled gracefully. Time range overlapping among > store files is tolerated and the performance impact is minimized. > Configuration can be set at hbase-site.xml or overriden at per-table or > per-column-famly level by hbase shell. > Design spec is at > https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing > Results in our production is at > https://docs.google.com/document/d/1GqRtQZMMkTEWOijZc8UCTqhACNmdxBSjtAQSYIWsmGU/edit# -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15181) A simple implementation of date based tiered compaction
[ https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong updated HBASE-15181: Description: This is a simple implementation of date-based tiered compaction similar to Cassandra's for the following benefits: 1. Improve date-range-based scan by structuring store files in date-based tiered layout. 2. Reduce compaction overhead. 3. Improve TTL efficiency. Perfect fit for the use cases that: 1. has mostly date-based date write and scan and a focus on the most recent data. 2. never or rarely deletes data. Out-of-order writes are handled gracefully. Time range overlapping among store files is tolerated and the performance impact is minimized. Configuration can be set at hbase-site.xml or overriden at per-table or per-column-famly level by hbase shell. Design spec is at https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing Results in our production is at https://docs.google.com/document/d/1GqRtQZMMkTEWOijZc8UCTqhACNmdxBSjtAQSYIWsmGU/edit# was: This is a simple implementation of date-based tiered compaction similar to Cassandra's for the following benefits: 1. Improve date-range-based scan by structuring store files in date-based tiered layout. 2. Reduce compaction overhead. 3. Improve TTL efficiency. Perfect fit for the use cases that: 1. has mostly date-based date write and scan and a focus on the most recent data. 2. never or rarely deletes data. Out-of-order writes are handled gracefully. Time range overlapping among store files is tolerated and the performance impact is minimized. Configuration can be set at hbase-site.xml or overriden at per-table or per-column-famly level by hbase shell. Design spec is at https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing > A simple implementation of date based tiered compaction > --- > > Key: HBASE-15181 > URL: https://issues.apache.org/jira/browse/HBASE-15181 > Project: HBase > Issue Type: New Feature > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15181-0.98-ADD.patch, HBASE-15181-0.98.patch, > HBASE-15181-0.98.v4.patch, HBASE-15181-98.patch, HBASE-15181-ADD.patch, > HBASE-15181-branch-1.patch, HBASE-15181-master-v1.patch, > HBASE-15181-master-v2.patch, HBASE-15181-master-v3.patch, > HBASE-15181-master-v4.patch, HBASE-15181-v1.patch, HBASE-15181-v2.patch > > > This is a simple implementation of date-based tiered compaction similar to > Cassandra's for the following benefits: > 1. Improve date-range-based scan by structuring store files in date-based > tiered layout. > 2. Reduce compaction overhead. > 3. Improve TTL efficiency. > Perfect fit for the use cases that: > 1. has mostly date-based date write and scan and a focus on the most recent > data. > 2. never or rarely deletes data. > Out-of-order writes are handled gracefully. Time range overlapping among > store files is tolerated and the performance impact is minimized. > Configuration can be set at hbase-site.xml or overriden at per-table or > per-column-famly level by hbase shell. > Design spec is at > https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing > Results in our production is at > https://docs.google.com/document/d/1GqRtQZMMkTEWOijZc8UCTqhACNmdxBSjtAQSYIWsmGU/edit# -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction
[
https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176320#comment-15176320
]
Vladimir Rodionov commented on HBASE-15181:
---
[~claraxiong]
{quote}
When it doesn't perform as well:
random gets without a time range
{quote}
I think we should educate users how to read data efficiently in case of
multiple store files in a region:
*io.storefile.bloom.error.rate* must be decreased from default 0.01 to 0.001 or
less (I would recommend 0.0001). This will reduce false positives in rowkey
lookups accordingly but will increase space occupied by bloom filter in memory
(of course).
> A simple implementation of date based tiered compaction
> ---
>
> Key: HBASE-15181
> URL: https://issues.apache.org/jira/browse/HBASE-15181
> Project: HBase
> Issue Type: New Feature
> Components: Compaction
>Reporter: Clara Xiong
>Assignee: Clara Xiong
> Fix For: 2.0.0, 1.3.0, 0.98.19
>
> Attachments: HBASE-15181-0.98-ADD.patch, HBASE-15181-0.98.patch,
> HBASE-15181-0.98.v4.patch, HBASE-15181-98.patch, HBASE-15181-ADD.patch,
> HBASE-15181-branch-1.patch, HBASE-15181-master-v1.patch,
> HBASE-15181-master-v2.patch, HBASE-15181-master-v3.patch,
> HBASE-15181-master-v4.patch, HBASE-15181-v1.patch, HBASE-15181-v2.patch
>
>
> This is a simple implementation of date-based tiered compaction similar to
> Cassandra's for the following benefits:
> 1. Improve date-range-based scan by structuring store files in date-based
> tiered layout.
> 2. Reduce compaction overhead.
> 3. Improve TTL efficiency.
> Perfect fit for the use cases that:
> 1. has mostly date-based date write and scan and a focus on the most recent
> data.
> 2. never or rarely deletes data.
> Out-of-order writes are handled gracefully. Time range overlapping among
> store files is tolerated and the performance impact is minimized.
> Configuration can be set at hbase-site.xml or overriden at per-table or
> per-column-famly level by hbase shell.
> Design spec is at
> https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15291) FileSystem not closed in secure bulkLoad
[
https://issues.apache.org/jira/browse/HBASE-15291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176297#comment-15176297
]
Hadoop QA commented on HBASE-15291:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 46s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 107m 52s
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 111m 25s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 269m 0s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_95 Failed junit tests |
hadoop.hbase.security.token.TestGenerateDelegationToken |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-03-02 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176294#comment-15176294
]
Hadoop QA commented on HBASE-15325:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 10s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/latest/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 23s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 19s
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 12s
{color} | {color:red} Patch generated 2 new checkstyle issues in hbase-client
(total was 216, now 216). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 25s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-server
(total was 150, now 150). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 13s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 15s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 33s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 34s
{color} | {color:green} hbase-client in the patch passed
[jira] [Commented] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176282#comment-15176282 ] Mikhail Antonov commented on HBASE-15136: - [~jmspaggi] it's not really "expected", but that's just a new thing which didn't run in prod under different workloads etc. Additional reviews are always welcome ;) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176263#comment-15176263 ] stack commented on HBASE-15381: --- [~jingcheng...@intel.com] " might impact the running of HMaster..." Can you say more? > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15378) Scanner can not handle a heartbeat message with no results
[
https://issues.apache.org/jira/browse/HBASE-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176260#comment-15176260
]
Hadoop QA commented on HBASE-15378:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 8s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/latest/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 56s
{color} | {color:red} Patch generated 2 new checkstyle issues in hbase-client
(total was 209, now 210). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 19s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 13s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 50s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176245#comment-15176245
]
Ted Yu commented on HBASE-15370:
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/785/artifact/patchprocess/diff-patch-ruby-lint.txt
> :
{code}
/testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:69:26: undefined
method org
/testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:70:11: undefined
constant ::(send nil :org)::(send
/testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:70:11: undefined
constant ::(send nil :org)::(send
/testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:70:11: undefined
method org
/testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:73:26: undefined
method org
{code}
Here is relevant code in admin.rb around line 69:
{code}
@admin.compact(org.apache.hadoop.hbase.TableName.valueOf(table_or_region_name),
org.apache.hadoop.hbase.client.Admin::CompactType::MOB)
{code}
'org' is not method name.
Not sure how ruby lint deduces the warning.
> Backport Moderate Object Storage (MOB) to branch-1
> --
>
> Key: HBASE-15370
> URL: https://issues.apache.org/jira/browse/HBASE-15370
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.4.0
>
> Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch
>
>
> MOB feature was integrated to master branch 1 year ago.
> Since then there have been bug fixes which stabilize the feature.
> Some customers have been using it at PB scale.
> Here is discussion thread on dev mailing list:
> http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+
> This issue is to backport MOB feature to branch-1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15321: --- Attachment: HBASE-15321-v4.patch Thanks [~anoop.hbase] and [~enis]. I took out the additional state and just made a helper method. Ensuring that when you initialize a hdfs snapshot region the replicaId is set such that you don't do any FS mutations. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > Attachments: HBASE-15321-v1.patch, HBASE-15321-v2.patch, > HBASE-15321-v3.patch, HBASE-15321-v4.patch, HBASE-15321.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176172#comment-15176172 ] Ted Yu commented on HBASE-15370: How about the following as a starting point for thread on dev@hbase ? [DISCUSS] Criteria for including MOB feature backport in branch-1 This is to solicit discussion on the criteria for including MOB feature backport in branch-1. I can think of the following: 1. whether there is customer interest There is. See Jonathan's response here: http://search-hadoop.com/m/YGbbDSqxD1PYXK62 2. whether unit test stability can be maintained in branch-1 Inclusion of the backport should keep unit tests (both existing and new) in branch-1 green. Preliminary test runs showed that MOB / snapshot related tests consistently pass. https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094 Comments are welcome > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch > > > MOB feature was integrated to master branch 1 year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176143#comment-15176143 ] Ted Yu commented on HBASE-15370: I looked for [DISCUSS] thread in my Inbox and using search-hadoop.com - haven't found one. [~busbey]: Mind refreshing my memory so that I can use it as a template ? Thanks > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch > > > MOB feature was integrated to master branch 1 year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176121#comment-15176121 ] Sean Busbey commented on HBASE-15370: - I would also like a properly labeled [DISCUSS] thread for what gates the backport getting applied to branch-1, similar to how we agreed to set a threshold on the hbase-spark work. > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 1.4.0 > > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch > > > MOB feature was integrated to master branch 1 year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176094#comment-15176094
]
Hadoop QA commented on HBASE-15370:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 53 new or modified test
files. {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 0m 53s
{color} | {color:red} branch's . dependency:list failed {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 1m 19s
{color} | {color:red} branch's hbase-client dependency:list failed {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 1m 33s
{color} | {color:red} branch's hbase-common dependency:list failed {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 1m 42s
{color} | {color:red} branch's hbase-hadoop2-compat dependency:list failed
{color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 1m 46s
{color} | {color:red} branch's hbase-hadoop-compat dependency:list failed
{color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 1m 59s
{color} | {color:red} branch's hbase-it dependency:list failed {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 2m 42s
{color} | {color:red} branch's hbase-server dependency:list failed {color} |
| {color:red}-1{color} | {color:red} mvndep {color} | {color:red} 2m 51s
{color} | {color:red} branch's hbase-shell dependency:list failed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 51s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 38s
{color} | {color:green} branch-1 passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 50s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
8s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
57s {color} | {color:green} branch-1 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped branch modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 12s
{color} | {color:red} branch/hbase-it no findbugs output file
(hbase-it/target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 55s
{color} | {color:green} branch-1 passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 54s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 2s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 2s {color}
| {color:red} hbase-server-jdk1.7.0_95 with JDK v1.7.0_95 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 2s {color}
| {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 2 new + 15
unchanged - 2 fixed = 17 total (was 17) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} |
[jira] [Commented] (HBASE-15291) FileSystem not closed in secure bulkLoad
[ https://issues.apache.org/jira/browse/HBASE-15291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176079#comment-15176079 ] Hudson commented on HBASE-15291: ABORTED: Integrated in HBase-0.98-matrix #305 (See [https://builds.apache.org/job/HBase-0.98-matrix/305/]) HBASE-15291 Revert due to the bug Haitao discovered (tedyu: rev fe7bcf58d9148df81280d83cb6f0d8e8eb174605) * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java > FileSystem not closed in secure bulkLoad > > > Key: HBASE-15291 > URL: https://issues.apache.org/jira/browse/HBASE-15291 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.2, 0.98.16.1 >Reporter: Yong Zhang >Assignee: Yong Zhang > Fix For: 2.0.0, 1.3.0, 1.2.1, 0.98.18 > > Attachments: HBASE-15291.001.patch, HBASE-15291.002.patch, > HBASE-15291.003.patch, HBASE-15291.addendum, patch > > > FileSystem not closed in secure bulkLoad after bulkLoad finish, it will > cause memory used more and more if too many bulkLoad . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15181) A simple implementation of date based tiered compaction
[ https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Latham updated HBASE-15181: Fix Version/s: (was: 1.4.0) > A simple implementation of date based tiered compaction > --- > > Key: HBASE-15181 > URL: https://issues.apache.org/jira/browse/HBASE-15181 > Project: HBase > Issue Type: New Feature > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15181-0.98-ADD.patch, HBASE-15181-0.98.patch, > HBASE-15181-0.98.v4.patch, HBASE-15181-98.patch, HBASE-15181-ADD.patch, > HBASE-15181-branch-1.patch, HBASE-15181-master-v1.patch, > HBASE-15181-master-v2.patch, HBASE-15181-master-v3.patch, > HBASE-15181-master-v4.patch, HBASE-15181-v1.patch, HBASE-15181-v2.patch > > > This is a simple implementation of date-based tiered compaction similar to > Cassandra's for the following benefits: > 1. Improve date-range-based scan by structuring store files in date-based > tiered layout. > 2. Reduce compaction overhead. > 3. Improve TTL efficiency. > Perfect fit for the use cases that: > 1. has mostly date-based date write and scan and a focus on the most recent > data. > 2. never or rarely deletes data. > Out-of-order writes are handled gracefully. Time range overlapping among > store files is tolerated and the performance impact is minimized. > Configuration can be set at hbase-site.xml or overriden at per-table or > per-column-famly level by hbase shell. > Design spec is at > https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction
[ https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176080#comment-15176080 ] Hudson commented on HBASE-15181: ABORTED: Integrated in HBase-0.98-matrix #305 (See [https://builds.apache.org/job/HBase-0.98-matrix/305/]) HBASE-15181 Addendum fixes findbugs warning (Clara Xiong) (tedyu: rev 41c04ee685b07321efe570fa91416ba90f8eeaa9) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java > A simple implementation of date based tiered compaction > --- > > Key: HBASE-15181 > URL: https://issues.apache.org/jira/browse/HBASE-15181 > Project: HBase > Issue Type: New Feature > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15181-0.98-ADD.patch, HBASE-15181-0.98.patch, > HBASE-15181-0.98.v4.patch, HBASE-15181-98.patch, HBASE-15181-ADD.patch, > HBASE-15181-branch-1.patch, HBASE-15181-master-v1.patch, > HBASE-15181-master-v2.patch, HBASE-15181-master-v3.patch, > HBASE-15181-master-v4.patch, HBASE-15181-v1.patch, HBASE-15181-v2.patch > > > This is a simple implementation of date-based tiered compaction similar to > Cassandra's for the following benefits: > 1. Improve date-range-based scan by structuring store files in date-based > tiered layout. > 2. Reduce compaction overhead. > 3. Improve TTL efficiency. > Perfect fit for the use cases that: > 1. has mostly date-based date write and scan and a focus on the most recent > data. > 2. never or rarely deletes data. > Out-of-order writes are handled gracefully. Time range overlapping among > store files is tolerated and the performance impact is minimized. > Configuration can be set at hbase-site.xml or overriden at per-table or > per-column-famly level by hbase shell. > Design spec is at > https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration
[ https://issues.apache.org/jira/browse/HBASE-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176078#comment-15176078 ] Hudson commented on HBASE-15365: ABORTED: Integrated in HBase-0.98-matrix #305 (See [https://builds.apache.org/job/HBase-0.98-matrix/305/]) HBASE-15365 Do not write to '/tmp' in TestHBaseConfiguration (zhangduo: rev 26c114c20f03e879cbc8353ec3d2c48472c3417e) * hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java > Do not write to '/tmp' in TestHBaseConfiguration > > > Key: HBASE-15365 > URL: https://issues.apache.org/jira/browse/HBASE-15365 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 1.0.4, 0.98.18, 1.4.0 > > Attachments: HBASE-15365-v1.patch, HBASE-15365.patch > > > In testGetPassword, we create a key file at /tmp/foo.jks and set its > permissions to 600. This will cause testcase failure on a shared machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15368) Add relative window support
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176063#comment-15176063 ] Dave Latham commented on HBASE-15368: - This still is not going to work well. Using the min timestamp does not fix it. With relative windows, there's nothing to prevent a fully compacted HFile for one window from sliding across the window boundary, being compacted into that window, and sliding quickly to the next again. There would be no reliable behavior of actual data staying partitioned in a well defined way. For example, suppose you have windows of .2 days, 1 day, 5 days, 25 days, each with a single HFile containing well patterned data spread across most of their windows. A few seconds later the .2 day HFile could be in the 1 day window, then be compacted with it, then that HFile slide to the 5 day window, be compacted, then to the 25 day window. Now quickly all of your data winds up in a single window and HFile. This is just one example of how the relative windows could go wrong. To make it work you would probably need to more rigidly control which files within a window could be allowed to be compacted together - I'm not sure a reasonable scheme exists that doesn't wind up approximating the fixed windows anyway. > Add relative window support > --- > > Key: HBASE-15368 > URL: https://issues.apache.org/jira/browse/HBASE-15368 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15368-v1.patch, HBASE-15368.patch > > > To better determine 'hot' data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176055#comment-15176055
]
Ted Yu commented on HBASE-15370:
Among the checkstyle warnings:
{code}
./hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/MobCompactor.java:42:24:
Variable 'fs' must be private and have accessor methods.
./hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/MobCompactor.java:43:27:
Variable 'conf' must be private and have accessor methods.
{code}
I plan to hold off addressing such warnings partly because this would create
some discrepancy between the backport and existing code in master branch.
> Backport Moderate Object Storage (MOB) to branch-1
> --
>
> Key: HBASE-15370
> URL: https://issues.apache.org/jira/browse/HBASE-15370
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.4.0
>
> Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch
>
>
> MOB feature was integrated to master branch 1 year ago.
> Since then there have been bug fixes which stabilize the feature.
> Some customers have been using it at PB scale.
> Here is discussion thread on dev mailing list:
> http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+
> This issue is to backport MOB feature to branch-1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
