[jira] [Updated] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
[ https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-17757: --- Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) Pushed to branch-1 also.. Thanks for the patch Allan. > Unify blocksize after encoding to decrease memory fragment > --- > > Key: HBASE-17757 > URL: https://issues.apache.org/jira/browse/HBASE-17757 > Project: HBase > Issue Type: New Feature >Reporter: Allan Yang >Assignee: Allan Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17757-branch-1.v5.patch, HBASE-17757.patch, > HBASE-17757v2.patch, HBASE-17757v3.patch, HBASE-17757v4-branch-1.patch, > HBASE-17757v4.patch > > > Usually, we store encoded block(uncompressed) in blockcache/bucketCache. > Though we have set the blocksize, after encoding, blocksize is varied. Varied > blocksize will cause memory fragment problem, which will result in more FGC > finally.In order to relief the memory fragment, This issue adjusts the > encoded block to a unified size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17924) Consider sorting the row order when processing multi() ops before taking rowlocks
[
https://issues.apache.org/jira/browse/HBASE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992324#comment-15992324
]
Hadoop QA commented on HBASE-17924:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 59s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 7s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 190m 44s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient |
| | org.apache.hadoop.hbase.client.TestSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865857/HBASE-17924.v5.patch |
| JIRA Issue | HBASE-17924 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d3a1929f0ba0 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 13b6fdf |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6655/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6655/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6655/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Issue Comment Deleted] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-17343: Comment: was deleted (was: Based on the email that [~anoop.hbase] has sent recently the default change is acceptable. The email is copy-pasted below. We assume the senior committer ([~stack]?) will notify the community. Thanks everyone for the long and hard work! I am going to commit tomorrow. bq. Hi All bq.Tests are done.. Even with CMS algo also. As some of our big customers still use it in production, I thought will test that also once.. Till now it looks good.. No where we perform bad compared to default memstore. (None of my tests do in memory merge btw. Only it does flattening) bq. BTW one Q Stack.. Am not sure how many of dev/user track the jira and comments. Should we send a notice to dev@/user@ before changing the default type of memstore? bq. Anoop ) > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Anastasia Braginsky >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch, > HBASE-17343-V04.patch, HBASE-17343-V05.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14925) Develop HBase shell command/tool to list table's region info through command line
[ https://issues.apache.org/jira/browse/HBASE-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992317#comment-15992317 ] Ashish Singhi commented on HBASE-14925: --- bq. 4. I am not sure how release notes are handled since I am new to this, but I am happy to learn. Please provide me with relevant info. If you click on Edit button to edit an issue you will see a section for Release Note there you can add the information about this command. > Develop HBase shell command/tool to list table's region info through command > line > - > > Key: HBASE-14925 > URL: https://issues.apache.org/jira/browse/HBASE-14925 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Romil Choksi >Assignee: Karan Mehta > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-14925.002.patch, HBASE-14925.003.patch, > HBASE-14925.patch > > > I am going through the hbase shell commands to see if there is anything I can > use to get all the regions info just for a particular table. I don’t see any > such command that provides me that information. > It would be better to have a command that provides region info, start key, > end key etc taking a table name as the input parameter. This is available > through HBase UI on clicking on a particular table's link > A tool/shell command to get a list of regions for a table or all tables in a > tabular structured output (that is machine readable) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17124) HFileInputFormat can help user read hbase table with HFile way,more effective
[ https://issues.apache.org/jira/browse/HBASE-17124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992304#comment-15992304 ] Yuan Kang commented on HBASE-17124: --- I have see HBASE-14123 ,it's used for Backup/Restore,and I can't find HFileInputFormat in the patchs.HFileInputFormat can be used for analyze. simplly,people can flush a table,then run a mapreduce job with HFileInputFormat ,this can analyze data online just like the function TableMapper do > HFileInputFormat can help user read hbase table with HFile way,more effective > - > > Key: HBASE-17124 > URL: https://issues.apache.org/jira/browse/HBASE-17124 > Project: HBase > Issue Type: New Feature > Components: HFile >Reporter: Yuan Kang >Assignee: Yuan Kang > Attachments: HFileInputFormat.java > > > When we need to read large amount of data from hbase,usually we use > tableinputformat to query hbase table. but hfileInputFormat way is more > effective -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17979) HBase Shell 'list' Command Help Doc Improvements
[ https://issues.apache.org/jira/browse/HBASE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992300#comment-15992300 ] Hudson commented on HBASE-17979: FAILURE: Integrated in Jenkins build HBase-1.4 #713 (See [https://builds.apache.org/job/HBase-1.4/713/]) HBASE-17979 HBase Shell 'list' Command Help Doc Improvements (Hugo (enis: rev 961bb7325cbcac2a5cf9064e404e2c5141c543c7) * (edit) hbase-shell/src/main/ruby/shell/commands/list.rb > HBase Shell 'list' Command Help Doc Improvements > > > Key: HBASE-17979 > URL: https://issues.apache.org/jira/browse/HBASE-17979 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Hugo Louro >Assignee: Hugo Louro >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17979.patch > > > In HBase shell: > 'help list' currently prints "List all tables in hbase." > Since the 'list' command only lists the user tables, and not the system > tables, it is more accurate the help command to print: > "List all **user** tables in hbase." -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17862) Condition that always returns true
[
https://issues.apache.org/jira/browse/HBASE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17862:
---
Fix Version/s: 1.1.11
1.3.2
1.2.6
1.4.0
2.0.0
> Condition that always returns true
> --
>
> Key: HBASE-17862
> URL: https://issues.apache.org/jira/browse/HBASE-17862
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: JC
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11
>
> Attachments:
> 0001-HBASE-17862-Fix-a-condition-that-always-returns-true.patch
>
>
> Hi
> In recent github mirror of hbase, I've found the following code smell.
> Path:
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
> {code}
> 209
> 210 ColumnPaginationFilter other = (ColumnPaginationFilter)o;
> 211 if (this.columnOffset != null) {
> 212 return this.getLimit() == this.getLimit() &&
> 213 Bytes.equals(this.getColumnOffset(), other.getColumnOffset());
> 214 }
> {code}
> It should be?
> {code}
> 212 return this.getLimit() == other.getLimit() &&
> {code}
> This might be just a code smell as Bytes.equals can be enough for the return
> value but wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17862) Condition that always returns true
[
https://issues.apache.org/jira/browse/HBASE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992285#comment-15992285
]
Chia-Ping Tsai commented on HBASE-17862:
LGTM. +1 if QA is fine.
[~busbey] Could you assign this issue to [~lifove] ? Thanks.
> Condition that always returns true
> --
>
> Key: HBASE-17862
> URL: https://issues.apache.org/jira/browse/HBASE-17862
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: JC
>Priority: Trivial
> Attachments:
> 0001-HBASE-17862-Fix-a-condition-that-always-returns-true.patch
>
>
> Hi
> In recent github mirror of hbase, I've found the following code smell.
> Path:
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
> {code}
> 209
> 210 ColumnPaginationFilter other = (ColumnPaginationFilter)o;
> 211 if (this.columnOffset != null) {
> 212 return this.getLimit() == this.getLimit() &&
> 213 Bytes.equals(this.getColumnOffset(), other.getColumnOffset());
> 214 }
> {code}
> It should be?
> {code}
> 212 return this.getLimit() == other.getLimit() &&
> {code}
> This might be just a code smell as Bytes.equals can be enough for the return
> value but wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17862) Condition that always returns true
[
https://issues.apache.org/jira/browse/HBASE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992280#comment-15992280
]
JC commented on HBASE-17862:
I've just attached the patch. (This is the first time to submit a patch in this
way. I have no idea if I submitted correctly here.) Thanks!
> Condition that always returns true
> --
>
> Key: HBASE-17862
> URL: https://issues.apache.org/jira/browse/HBASE-17862
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: JC
>Priority: Trivial
> Attachments:
> 0001-HBASE-17862-Fix-a-condition-that-always-returns-true.patch
>
>
> Hi
> In recent github mirror of hbase, I've found the following code smell.
> Path:
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
> {code}
> 209
> 210 ColumnPaginationFilter other = (ColumnPaginationFilter)o;
> 211 if (this.columnOffset != null) {
> 212 return this.getLimit() == this.getLimit() &&
> 213 Bytes.equals(this.getColumnOffset(), other.getColumnOffset());
> 214 }
> {code}
> It should be?
> {code}
> 212 return this.getLimit() == other.getLimit() &&
> {code}
> This might be just a code smell as Bytes.equals can be enough for the return
> value but wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17862) Condition that always returns true
[
https://issues.apache.org/jira/browse/HBASE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JC updated HBASE-17862:
---
Attachment: 0001-HBASE-17862-Fix-a-condition-that-always-returns-true.patch
> Condition that always returns true
> --
>
> Key: HBASE-17862
> URL: https://issues.apache.org/jira/browse/HBASE-17862
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: JC
>Priority: Trivial
> Attachments:
> 0001-HBASE-17862-Fix-a-condition-that-always-returns-true.patch
>
>
> Hi
> In recent github mirror of hbase, I've found the following code smell.
> Path:
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
> {code}
> 209
> 210 ColumnPaginationFilter other = (ColumnPaginationFilter)o;
> 211 if (this.columnOffset != null) {
> 212 return this.getLimit() == this.getLimit() &&
> 213 Bytes.equals(this.getColumnOffset(), other.getColumnOffset());
> 214 }
> {code}
> It should be?
> {code}
> 212 return this.getLimit() == other.getLimit() &&
> {code}
> This might be just a code smell as Bytes.equals can be enough for the return
> value but wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17958) Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP
[
https://issues.apache.org/jira/browse/HBASE-17958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992149#comment-15992149
]
Guanghao Zhang commented on HBASE-17958:
When the heap changed the current storefile scanner, the new storefile maybe
have lots of cell for current column. And the new scanner has a new
nextIndexedKey, it may be not suitable to optimize SEEK to SKIP. So we should
decide whether to optimize after call heap.next() every time. Any more concerns
about v6 patch? [~lhofhansl] [~Apache9]
> Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP
>
>
> Key: HBASE-17958
> URL: https://issues.apache.org/jira/browse/HBASE-17958
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: 0001-add-one-ut-testWithColumnCountGetFilter.patch,
> HBASE-17958-v1.patch, HBASE-17958-v2.patch, HBASE-17958-v3.patch,
> HBASE-17958-v4.patch, HBASE-17958-v5.patch, HBASE-17958-v6.patch
>
>
> {code}
> ScanQueryMatcher.MatchCode qcode = matcher.match(cell);
> qcode = optimize(qcode, cell);
> {code}
> The optimize method may change the MatchCode from SEEK_NEXT_COL/SEEK_NEXT_ROW
> to SKIP. But it still pass the next cell to ScanQueryMatcher. It will get
> wrong result when use some filter, etc. ColumnCountGetFilter. It just count
> the columns's number. If pass a same column to this filter, the count result
> will be wrong. So we should avoid passing cell to ScanQueryMatcher when
> optimize SEEK to SKIP.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17969) Balance by table using SimpleLoadBalancer could end up imbalance
[ https://issues.apache.org/jira/browse/HBASE-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992122#comment-15992122 ] Allan Yang commented on HBASE-17969: Ping [~carp84], since HBASE-17110 is a work of his team. > Balance by table using SimpleLoadBalancer could end up imbalance > > > Key: HBASE-17969 > URL: https://issues.apache.org/jira/browse/HBASE-17969 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.10 >Reporter: Allan Yang >Assignee: Allan Yang > Attachments: HBASE-17969-branch-1.patch, > HBASE-17969-branch-1.v2.patch, HBASE-17969-branch-1.v3.patch, > HBASE-17969.patch > > > This really happens in our production env. > Here is a example: > Say we have three RS named r1, r2, r3. A table named table1 with 3 regions > distributes on these rs like this: > r1 1 > r2 1 > r3 1 > Each rs have one region, it means table1 is balanced. So balancer will not > run. > If the region on r3 splits, then it becomes: > r1 1 > r2 1 > r3 2 > For table1, in average, each rs will have min=1, max=2 regions. So still it > is balanced, balancer will not run. > Then a region on r3 splits again, the distribution becomes: > r1 1 > r2 1 > r3 3 > In average, each rs will have min=1, max=2 regions. So balancer will run. > For r1 and r2, they have already have min=1 regions. Balancer won't do any > operation on them. > But for r3, it exceed max=3, so balancer will remove one region from r3 and > choose one rs from r1, r2 to move to. > But r1 and r2 have the same load, so balancer will always choose r1 since > servername r1 < r2(alphabet order, sorted by ServerAndLoad's compareTo > method). It is OK for table1 itself. But if every table in the cluster have > similar situations like table1, then the load in the cluster will always be > like r1 > r2 > r3. > So, the solution here is when each rs reach min regions (min=total region / > servers), but there still some region need to move, shuffle the regionservers > before move. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17969) Balance by table using SimpleLoadBalancer could end up imbalance
[ https://issues.apache.org/jira/browse/HBASE-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992115#comment-15992115 ] Phil Yang commented on HBASE-17969: --- If HBASE-17110 resolved this issue, we can just wait [~aoxiang] porting it to branch-1 and close this as duplicated? Does this patch solve problem other than HBASE-17110 resolved? > Balance by table using SimpleLoadBalancer could end up imbalance > > > Key: HBASE-17969 > URL: https://issues.apache.org/jira/browse/HBASE-17969 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.10 >Reporter: Allan Yang >Assignee: Allan Yang > Attachments: HBASE-17969-branch-1.patch, > HBASE-17969-branch-1.v2.patch, HBASE-17969-branch-1.v3.patch, > HBASE-17969.patch > > > This really happens in our production env. > Here is a example: > Say we have three RS named r1, r2, r3. A table named table1 with 3 regions > distributes on these rs like this: > r1 1 > r2 1 > r3 1 > Each rs have one region, it means table1 is balanced. So balancer will not > run. > If the region on r3 splits, then it becomes: > r1 1 > r2 1 > r3 2 > For table1, in average, each rs will have min=1, max=2 regions. So still it > is balanced, balancer will not run. > Then a region on r3 splits again, the distribution becomes: > r1 1 > r2 1 > r3 3 > In average, each rs will have min=1, max=2 regions. So balancer will run. > For r1 and r2, they have already have min=1 regions. Balancer won't do any > operation on them. > But for r3, it exceed max=3, so balancer will remove one region from r3 and > choose one rs from r1, r2 to move to. > But r1 and r2 have the same load, so balancer will always choose r1 since > servername r1 < r2(alphabet order, sorted by ServerAndLoad's compareTo > method). It is OK for table1 itself. But if every table in the cluster have > similar situations like table1, then the load in the cluster will always be > like r1 > r2 > r3. > So, the solution here is when each rs reach min regions (min=total region / > servers), but there still some region need to move, shuffle the regionservers > before move. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17862) Condition that always returns true
[
https://issues.apache.org/jira/browse/HBASE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992013#comment-15992013
]
Chia-Ping Tsai commented on HBASE-17862:
ping [~lifove].
Would you please submit a patch here?
> Condition that always returns true
> --
>
> Key: HBASE-17862
> URL: https://issues.apache.org/jira/browse/HBASE-17862
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: JC
>Priority: Trivial
>
> Hi
> In recent github mirror of hbase, I've found the following code smell.
> Path:
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
> {code}
> 209
> 210 ColumnPaginationFilter other = (ColumnPaginationFilter)o;
> 211 if (this.columnOffset != null) {
> 212 return this.getLimit() == this.getLimit() &&
> 213 Bytes.equals(this.getColumnOffset(), other.getColumnOffset());
> 214 }
> {code}
> It should be?
> {code}
> 212 return this.getLimit() == other.getLimit() &&
> {code}
> This might be just a code smell as Bytes.equals can be enough for the return
> value but wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (HBASE-15804) Some links in documentation are 404
[
https://issues.apache.org/jira/browse/HBASE-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-15804.
Resolution: Duplicate
see HBASE-15461
> Some links in documentation are 404
> ---
>
> Key: HBASE-15804
> URL: https://issues.apache.org/jira/browse/HBASE-15804
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Heng Chen
> Attachments: HBASE-15804.patch
>
>
> http://hbase.apache.org/book.html#security
> The link to {{Understanding User Authentication and Authorization in Apache
> HBase}} return 404
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14286) Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-14286:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the patching. [~gliptak]
> Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
> ---
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch,
> HBASE-14286.3.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14286) Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-14286:
---
Summary: Correct typo in argument name for
WALSplitter.writeRegionSequenceIdFile (was: Fix spelling error in
"safetyBumper" parameter in WALSplitter)
> Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
> ---
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch,
> HBASE-14286.3.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14286) Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-14286:
---
Fix Version/s: 2.0.0
> Correct typo in argument name for WALSplitter.writeRegionSequenceIdFile
> ---
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch,
> HBASE-14286.3.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17979) HBase Shell 'list' Command Help Doc Improvements
[ https://issues.apache.org/jira/browse/HBASE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-17979: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this. Thanks [~hmclouro] for the patch. > HBase Shell 'list' Command Help Doc Improvements > > > Key: HBASE-17979 > URL: https://issues.apache.org/jira/browse/HBASE-17979 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Hugo Louro >Assignee: Hugo Louro >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17979.patch > > > In HBase shell: > 'help list' currently prints "List all tables in hbase." > Since the 'list' command only lists the user tables, and not the system > tables, it is more accurate the help command to print: > "List all **user** tables in hbase." -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991957#comment-15991957 ] Ashu Pachauri commented on HBASE-16138: --- Yeah [~sukuna...@gmail.com], I have been bit busy to work on it. Feel free to pick it up, thanks! > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri >Priority: Critical > Attachments: HBASE-16138.patch, HBASE-16138-v1.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17924) Consider sorting the row order when processing multi() ops before taking rowlocks
[
https://issues.apache.org/jira/browse/HBASE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991944#comment-15991944
]
Allan Yang commented on HBASE-17924:
Sorry, I messed up some code, some line is missing, v5 patch is attached, it
should be the right version. Thanks, [~apurtell] for reviewing!
> Consider sorting the row order when processing multi() ops before taking
> rowlocks
> -
>
> Key: HBASE-17924
> URL: https://issues.apache.org/jira/browse/HBASE-17924
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.1.8
>Reporter: Andrew Purtell
>Assignee: Allan Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-17924.patch, HBASE-17924.v0.patch,
> HBASE-17924.v2.patch, HBASE-17924.v3.patch, HBASE-17924.v4.patch,
> HBASE-17924.v5.patch
>
>
> When processing a batch mutation, we take row locks in whatever order the
> mutations were added to the multi op by the client.
>
> {noformat}
> RSRpcServices#multi -> RSRpcServices#mutateRows -> HRegion#mutateRow ->
> HRegion#mutateRowsWithLocks -> HRegion#processRowsWithLocks
> {noformat}
> Or
> {noformat}
> RSRpcServices#multi -> RSRpcServices#doNonAtomicRegionMutation ->
> HRegion#get
> | HRegion#append
> | HRegion#increment
> | HRegionServer#doBatchOp -> HRegion#batchMutate ->
> HRegion#doMiniBatchMutation
> {noformat}
>
> multi() is fed by client APIs that accept a RowMutations object containing
> actions for multiple rows. The container for ops inside RowMutations is an
> ArrayList, which doesn't change the ordering of objects added to it. The
> protobuf implementation of the messages for multi ops do not reorder the list
> of actions. When processing multi ops we iterate over the actions in the
> order rehydrated from protobuf.
> We should discuss sorting the order of ops by row key when processing multi()
> ops before taking row locks. Does this make lock ordering more predictable
> for server side operations? Yes, but potentially surprising for the client,
> right? Is there any legitimate reason we should take locks out of row key
> sorted order because the client has structured the request as such?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17924) Consider sorting the row order when processing multi() ops before taking rowlocks
[
https://issues.apache.org/jira/browse/HBASE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-17924:
---
Attachment: HBASE-17924.v5.patch
> Consider sorting the row order when processing multi() ops before taking
> rowlocks
> -
>
> Key: HBASE-17924
> URL: https://issues.apache.org/jira/browse/HBASE-17924
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.1.8
>Reporter: Andrew Purtell
>Assignee: Allan Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-17924.patch, HBASE-17924.v0.patch,
> HBASE-17924.v2.patch, HBASE-17924.v3.patch, HBASE-17924.v4.patch,
> HBASE-17924.v5.patch
>
>
> When processing a batch mutation, we take row locks in whatever order the
> mutations were added to the multi op by the client.
>
> {noformat}
> RSRpcServices#multi -> RSRpcServices#mutateRows -> HRegion#mutateRow ->
> HRegion#mutateRowsWithLocks -> HRegion#processRowsWithLocks
> {noformat}
> Or
> {noformat}
> RSRpcServices#multi -> RSRpcServices#doNonAtomicRegionMutation ->
> HRegion#get
> | HRegion#append
> | HRegion#increment
> | HRegionServer#doBatchOp -> HRegion#batchMutate ->
> HRegion#doMiniBatchMutation
> {noformat}
>
> multi() is fed by client APIs that accept a RowMutations object containing
> actions for multiple rows. The container for ops inside RowMutations is an
> ArrayList, which doesn't change the ordering of objects added to it. The
> protobuf implementation of the messages for multi ops do not reorder the list
> of actions. When processing multi ops we iterate over the actions in the
> order rehydrated from protobuf.
> We should discuss sorting the order of ops by row key when processing multi()
> ops before taking row locks. Does this make lock ordering more predictable
> for server side operations? Yes, but potentially surprising for the client,
> right? Is there any legitimate reason we should take locks out of row key
> sorted order because the client has structured the request as such?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
[
https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991905#comment-15991905
]
Allan Yang edited comment on HBASE-17757 at 5/2/17 12:46 AM:
-
{{HBASE-17757v4-branch-1.patch}} is the patch generated by 'git format-patch',
It is the same patch as {{HBASE-17757-branch-1.v5.patch}}, you can submit
{{HBASE-17757v4-branch-1.patch}} directly, [~anoopsamjohn], thanks!
was (Author: allan163):
HBASE-17757v4-branch-1.patch is the patch generated by 'git format-patch', It
is the same patch as HBASE-17757-branch-1.v5.patch, you can submit
HBASE-17757v4-branch-1.patch directly, [~anoopsamjohn], thanks!
> Unify blocksize after encoding to decrease memory fragment
> ---
>
> Key: HBASE-17757
> URL: https://issues.apache.org/jira/browse/HBASE-17757
> Project: HBase
> Issue Type: New Feature
>Reporter: Allan Yang
>Assignee: Allan Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-17757-branch-1.v5.patch, HBASE-17757.patch,
> HBASE-17757v2.patch, HBASE-17757v3.patch, HBASE-17757v4-branch-1.patch,
> HBASE-17757v4.patch
>
>
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache.
> Though we have set the blocksize, after encoding, blocksize is varied. Varied
> blocksize will cause memory fragment problem, which will result in more FGC
> finally.In order to relief the memory fragment, This issue adjusts the
> encoded block to a unified size.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
[ https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991905#comment-15991905 ] Allan Yang commented on HBASE-17757: HBASE-17757v4-branch-1.patch is the patch generated by 'git format-patch', It is the same patch as HBASE-17757-branch-1.v5.patch, you can submit HBASE-17757v4-branch-1.patch directly, [~anoopsamjohn], thanks! > Unify blocksize after encoding to decrease memory fragment > --- > > Key: HBASE-17757 > URL: https://issues.apache.org/jira/browse/HBASE-17757 > Project: HBase > Issue Type: New Feature >Reporter: Allan Yang >Assignee: Allan Yang > Fix For: 2.0.0 > > Attachments: HBASE-17757-branch-1.v5.patch, HBASE-17757.patch, > HBASE-17757v2.patch, HBASE-17757v3.patch, HBASE-17757v4-branch-1.patch, > HBASE-17757v4.patch > > > Usually, we store encoded block(uncompressed) in blockcache/bucketCache. > Though we have set the blocksize, after encoding, blocksize is varied. Varied > blocksize will cause memory fragment problem, which will result in more FGC > finally.In order to relief the memory fragment, This issue adjusts the > encoded block to a unified size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14286) Fix spelling error in "safetyBumper" parameter in WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991900#comment-15991900
]
Hadoop QA commented on HBASE-14286:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 43s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 109m 27s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
1s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 192m 40s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionReplicaFailover |
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.TestPartialResultsFromClientSide |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865820/HBASE-14286.3.patch |
| JIRA Issue | HBASE-14286 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux fb0f02c4b631 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 13b6fdf |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6652/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6652/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test
[jira] [Commented] (HBASE-17938) General fault - tolerance framework for backup/restore operations
[ https://issues.apache.org/jira/browse/HBASE-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991883#comment-15991883 ] Devaraj Das commented on HBASE-17938: - I agree.. Let's keep it simple for now. Once we get more experience, we can enhance as needed. > General fault - tolerance framework for backup/restore operations > - > > Key: HBASE-17938 > URL: https://issues.apache.org/jira/browse/HBASE-17938 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-17938-v1.patch, HBASE-17938-v2.patch, > HBASE-17938-v3.patch > > > The framework must take care of all general types of failures during backup/ > restore and restore system to the original state in case of a failure. > That won't solve all the possible issues but we have a separate JIRAs for > them as a sub-tasks of HBASE-15277 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991875#comment-15991875
]
Hudson commented on HBASE-17973:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2933 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2933/])
HBASE-17973 Expand list_regions to filter on data locality (elserj: rev
13b6fdf8ad81e236632a2dc99e6c4a317213858e)
* (edit) hbase-shell/src/main/ruby/shell/commands/list_regions.rb
* (add) hbase-shell/src/test/ruby/hbase/list_regions_test_no_cluster.rb
* (edit) hbase-shell/src/main/ruby/hbase_constants.rb
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17938) General fault - tolerance framework for backup/restore operations
[
https://issues.apache.org/jira/browse/HBASE-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991862#comment-15991862
]
Vladimir Rodionov commented on HBASE-17938:
---
{quote}
Currently fault handling is coarse grained (see cleanupAndRestoreBackupSystem).
I suggest investigating fine grained approach where last successful sub-step is
recorded so that subsequent run can omit unnecessary work.
{quote}
The same as above. Complex algorithms are proved to be fragile. There is no
need for additional complexity here. Repair operation is very fast (deletes
some files and restores table from snapshot)
> General fault - tolerance framework for backup/restore operations
> -
>
> Key: HBASE-17938
> URL: https://issues.apache.org/jira/browse/HBASE-17938
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-17938-v1.patch, HBASE-17938-v2.patch,
> HBASE-17938-v3.patch
>
>
> The framework must take care of all general types of failures during backup/
> restore and restore system to the original state in case of a failure.
> That won't solve all the possible issues but we have a separate JIRAs for
> them as a sub-tasks of HBASE-15277
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17938) General fault - tolerance framework for backup/restore operations
[
https://issues.apache.org/jira/browse/HBASE-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991857#comment-15991857
]
Vladimir Rodionov commented on HBASE-17938:
---
{quote}
Assuming IOE may come out of each of the calls above, shouldn't a state machine
be designed for more robustness ?
{quote}
No needs to overcomplicate the feature. If system repair fails in the middle,
user will get notified and will have a chance to fix everything by running
repair tool manually.
Repair operation is idempotent and can be run as many times as we need.
Did I address your concerns, [~tedyu]?
> General fault - tolerance framework for backup/restore operations
> -
>
> Key: HBASE-17938
> URL: https://issues.apache.org/jira/browse/HBASE-17938
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-17938-v1.patch, HBASE-17938-v2.patch,
> HBASE-17938-v3.patch
>
>
> The framework must take care of all general types of failures during backup/
> restore and restore system to the original state in case of a failure.
> That won't solve all the possible issues but we have a separate JIRAs for
> them as a sub-tasks of HBASE-15277
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991856#comment-15991856 ] Appy commented on HBASE-16549: -- Looks great [~uagashe]. +1. > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch, > HBASE-16549-hbase-14614.v3.patch, HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17938) General fault - tolerance framework for backup/restore operations
[ https://issues.apache.org/jira/browse/HBASE-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991816#comment-15991816 ] Ted Yu commented on HBASE-17938: I did look at the review board 5 days ago - my latest comments were not addressed. Currently fault handling is coarse grained (see cleanupAndRestoreBackupSystem). I suggest investigating fine grained approach where last successful sub-step is recorded so that subsequent run can omit unnecessary work. > General fault - tolerance framework for backup/restore operations > - > > Key: HBASE-17938 > URL: https://issues.apache.org/jira/browse/HBASE-17938 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-17938-v1.patch, HBASE-17938-v2.patch, > HBASE-17938-v3.patch > > > The framework must take care of all general types of failures during backup/ > restore and restore system to the original state in case of a failure. > That won't solve all the possible issues but we have a separate JIRAs for > them as a sub-tasks of HBASE-15277 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16466) HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
[
https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991793#comment-15991793
]
Andrew Purtell commented on HBASE-16466:
The change to TableMapReduceUtil is unrelated. Please remove it. If you want
this this change in the code, file a new JIRA for it and we can get that
committed separately. It's a nontrivial behavioral change.
Sorry to pick on nits but this patch does not conform to our coding standards.
See chahttps://hbase.apache.org/book.html#common.patch.feedback which refers
https://www.oracle.com/technetwork/java/index-135089.html . That's a lot to
read, so let me specifically address what bothers the eye:
Braces on the same line, please.
{code}
if () {
} else {
}
{code}
not
{code}
if ()
{
}
else
{
}
{code}
Spaces between keywords and parentheses please.
Don't comment out code
{code}
//deleteDirectories(conf, snapshotTempPath);
{code}
If you don't want it in there, remove it.
If anything, the changes made to a file should look the same as the code above
and below.
> HBase snapshots support in VerifyReplication tool to reduce load on live
> HBase cluster with large tables
>
>
> Key: HBASE-16466
> URL: https://issues.apache.org/jira/browse/HBASE-16466
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 0.98.21
>Reporter: Sukumar Maddineni
>Assignee: Maddineni Sukumar
> Fix For: 2.0.0
>
> Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch,
> HBASE-16466.v2.patch, HBASE-16466.v3.patch
>
>
> As of now VerifyReplicatin tool is running using normal HBase scanners. If
> you want to run VerifyReplication multiple times on a production live
> cluster with large tables then it creates extra load on HBase layer. So if we
> implement snapshot based support then both in source and target we can read
> data from snapshots which reduces load on HBase
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17924) Consider sorting the row order when processing multi() ops before taking rowlocks
[
https://issues.apache.org/jira/browse/HBASE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991767#comment-15991767
]
Andrew Purtell commented on HBASE-17924:
[~allan163] Apologies for the delay. I'm looking at the V4 patch. I don't see
where you add anything to mutationActionMap, which is initialized as empty. Is
this some kind of incremental diff against earlier patches? Please attach a
full diff from the base source. That can also explain a failed QA run.
> Consider sorting the row order when processing multi() ops before taking
> rowlocks
> -
>
> Key: HBASE-17924
> URL: https://issues.apache.org/jira/browse/HBASE-17924
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.1.8
>Reporter: Andrew Purtell
>Assignee: Allan Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-17924.patch, HBASE-17924.v0.patch,
> HBASE-17924.v2.patch, HBASE-17924.v3.patch, HBASE-17924.v4.patch
>
>
> When processing a batch mutation, we take row locks in whatever order the
> mutations were added to the multi op by the client.
>
> {noformat}
> RSRpcServices#multi -> RSRpcServices#mutateRows -> HRegion#mutateRow ->
> HRegion#mutateRowsWithLocks -> HRegion#processRowsWithLocks
> {noformat}
> Or
> {noformat}
> RSRpcServices#multi -> RSRpcServices#doNonAtomicRegionMutation ->
> HRegion#get
> | HRegion#append
> | HRegion#increment
> | HRegionServer#doBatchOp -> HRegion#batchMutate ->
> HRegion#doMiniBatchMutation
> {noformat}
>
> multi() is fed by client APIs that accept a RowMutations object containing
> actions for multiple rows. The container for ops inside RowMutations is an
> ArrayList, which doesn't change the ordering of objects added to it. The
> protobuf implementation of the messages for multi ops do not reorder the list
> of actions. When processing multi ops we iterate over the actions in the
> order rehydrated from protobuf.
> We should discuss sorting the order of ops by row key when processing multi()
> ops before taking row locks. Does this make lock ordering more predictable
> for server side operations? Yes, but potentially surprising for the client,
> right? Is there any legitimate reason we should take locks out of row key
> sorted order because the client has structured the request as such?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991708#comment-15991708 ] Elliott Clark commented on HBASE-16138: --- I don't think that anyone is currently working on this. Feel free to pick up where they left off if you have free cycles. > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri >Priority: Critical > Attachments: HBASE-16138.patch, HBASE-16138-v1.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16549) Procedure v2 - Add new AM metrics
[
https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991653#comment-15991653
]
Hadoop QA commented on HBASE-16549:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s {color}
| {color:red} HBASE-16549 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865826/HBASE-16549-hbase-14614.v3.patch
|
| JIRA Issue | HBASE-16549 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6654/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Add new AM metrics
> -
>
> Key: HBASE-16549
> URL: https://issues.apache.org/jira/browse/HBASE-16549
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Umesh Agashe
> Fix For: 2.0.0
>
> Attachments: HBASE-16549-hbase-14614.v1.patch,
> HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch,
> HBASE-16549-hbase-14614.v3.patch, HBASE-16549-hbase-14614.v3.patch
>
>
> With the new AM we can add a bunch of metrics
> - assign/unassign time
> - server crash time
> - grouping related metrics? (how many batch we do, and similar?)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-16549: - Attachment: HBASE-16549-hbase-14614.v3.patch > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch, > HBASE-16549-hbase-14614.v3.patch, HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser reopened HBASE-17973:
Apparently I'm terrible at writing Ruby tests for HBase.
Basic syntax error.. will resubmit with some more robust tests..
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17954) Switch findbugs implementation to spotbugs
[
https://issues.apache.org/jira/browse/HBASE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991603#comment-15991603
]
Hadoop QA commented on HBASE-17954:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 54s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 20s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
69m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 125m 43s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 243m 17s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.TestPartialResultsFromClientSide |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865790/HBASE-17954.master.001.patch
|
| JIRA Issue | HBASE-17954 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux 3e62df4cff76 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 94c14ad |
| Default Java | 1.8.0_121 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6649/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6649/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6649/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6649/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Switch findbugs implementation to spotbugs
> --
>
> Key: HBASE-17954
> URL: https://issues.apache.org/jira/browse/HBASE-17954
> Project: HBase
> Issue Type: Task
> Components: build, community, test
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
> Fix
[jira] [Commented] (HBASE-16549) Procedure v2 - Add new AM metrics
[
https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991599#comment-15991599
]
Hadoop QA commented on HBASE-16549:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color}
| {color:red} HBASE-16549 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865824/HBASE-16549-hbase-14614.v2-v3.patch
|
| JIRA Issue | HBASE-16549 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6653/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Add new AM metrics
> -
>
> Key: HBASE-16549
> URL: https://issues.apache.org/jira/browse/HBASE-16549
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Umesh Agashe
> Fix For: 2.0.0
>
> Attachments: HBASE-16549-hbase-14614.v1.patch,
> HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch,
> HBASE-16549-hbase-14614.v3.patch
>
>
> With the new AM we can add a bunch of metrics
> - assign/unassign time
> - server crash time
> - grouping related metrics? (how many batch we do, and similar?)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-16549: - Attachment: HBASE-16549-hbase-14614.v2-v3.patch > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch, > HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-16549: - Attachment: (was: HBASE-16549-hbase-14614.v2-v3.diff) > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.patch, > HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-16549: - Attachment: HBASE-16549-hbase-14614.v2-v3.diff Diff between v2.patch and v3.patch > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v2-v3.diff, > HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14286) Fix spelling error in "safetyBumper" parameter in WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Liptak updated HBASE-14286:
-
Attachment: HBASE-14286.3.patch
> Fix spelling error in "safetyBumper" parameter in WALSplitter
> -
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch,
> HBASE-14286.3.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16549) Procedure v2 - Add new AM metrics
[
https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991565#comment-15991565
]
Hadoop QA commented on HBASE-16549:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 12s {color}
| {color:red} HBASE-16549 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865817/HBASE-16549-hbase-14614.v3.patch
|
| JIRA Issue | HBASE-16549 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6651/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Add new AM metrics
> -
>
> Key: HBASE-16549
> URL: https://issues.apache.org/jira/browse/HBASE-16549
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Umesh Agashe
> Fix For: 2.0.0
>
> Attachments: HBASE-16549-hbase-14614.v1.patch,
> HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v3.patch
>
>
> With the new AM we can add a bunch of metrics
> - assign/unassign time
> - server crash time
> - grouping related metrics? (how many batch we do, and similar?)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991564#comment-15991564
]
Yi Liang commented on HBASE-16179:
--
Hi,
I just review the patch, and have some thoughts on this jira, I think 6
hbase-sparkXXX folders is too many, and how about the structure as below, it is
more clear and simple
{quote}
hbase-spark
- hbase-spark-1.6 (diff code for spark1.6)
-pom.xml
-src
- hbase-spark-2.0 (diff code for spark2.0)
- src (common code)
- pom.xml
{quote}
We still have a hbase-spark as parent structure, and add two sub-folder, one is
for spark2.0 and one is for spark1.6; and I also check the code, for both scala
2.10 and 2.11, there are no code difference between the two sub-folder, we can
make hbase-spark-1.6 default support scala 2.10 and hbase-spark-2.0 default
support scala 2.11, and if user want to change scala version, they can modify
pom file under those two sub folder or use some mvn option like
-Psparkscalaversion = xxx.xxx
> Fix compilation errors when building hbase-spark against Spark 2.0
> --
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
> Issue Type: Bug
> Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Labels: build
> Fix For: 2.0.0
>
> Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt,
> 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt,
> 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt,
> 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt,
> 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt,
> 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16549) Procedure v2 - Add new AM metrics
[
https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991553#comment-15991553
]
Umesh Agashe commented on HBASE-16549:
--
Hi [~appy],
Thanks for your review feedback. I have changed the code as per your review
comments. Regarding having member {{static ProcedureMetric metric}} in
AssignProcedure etc., there is a threading issue as we discussed on Friday.
Thanks,
Umesh
> Procedure v2 - Add new AM metrics
> -
>
> Key: HBASE-16549
> URL: https://issues.apache.org/jira/browse/HBASE-16549
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Umesh Agashe
> Fix For: 2.0.0
>
> Attachments: HBASE-16549-hbase-14614.v1.patch,
> HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v3.patch
>
>
> With the new AM we can add a bunch of metrics
> - assign/unassign time
> - server crash time
> - grouping related metrics? (how many batch we do, and similar?)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16549) Procedure v2 - Add new AM metrics
[ https://issues.apache.org/jira/browse/HBASE-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-16549: - Attachment: HBASE-16549-hbase-14614.v3.patch Addresses most of [~appy]'s review comments. > Procedure v2 - Add new AM metrics > - > > Key: HBASE-16549 > URL: https://issues.apache.org/jira/browse/HBASE-16549 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-16549-hbase-14614.v1.patch, > HBASE-16549-hbase-14614.v2.patch, HBASE-16549-hbase-14614.v3.patch > > > With the new AM we can add a bunch of metrics > - assign/unassign time > - server crash time > - grouping related metrics? (how many batch we do, and similar?) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17957) Custom metrics of replicate endpoints don't prepend "source." to global metrics
[
https://issues.apache.org/jira/browse/HBASE-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991428#comment-15991428
]
Hudson commented on HBASE-17957:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2932 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2932/])
HBASE-17957 Custom metrics of replicate endpoints don't prepend (tedyu: rev
ef94de3eb7ecbf2bb45ce1a3cbfb861ef4c1a417)
* (edit)
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationGlobalSourceSource.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java
> Custom metrics of replicate endpoints don't prepend "source." to global
> metrics
>
>
> Key: HBASE-17957
> URL: https://issues.apache.org/jira/browse/HBASE-17957
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.4.0, 0.98.22
>Reporter: Shinya Yoshida
>Assignee: Shinya Yoshida
>Priority: Minor
> Labels: incompatibleChange
> Attachments: HBASE-17957.master.001.patch,
> HBASE-17957.master.002.patch
>
>
> Custom metrics for custom replication endpoints is introduced by
> [HBASE-16448].
> The name of local custom metrics follows the "source.id.metricsName" format,
> but the name of global custom metrics doesn't follow the "source.metricsName"
> format.
> Ex:
> {code}
> // default metrics
> "source.2.shippedOps" : 1234, // peer local
> "source.shippedOps" : 12345, // global
> // custom metrics
> "source.1.failed.start" : 1, // peer local
> "failed.start" : 1, // global
> {code}
> When we consider that default metrics do so, it should be
> "source.metricsName" like:
> {code}
> "source.1.failed.start" : 1,
> "source.failed.start" : 1,
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991429#comment-15991429 ] Hudson commented on HBASE-17968: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2932 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2932/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 94c14ad0f692f60b53c091c1ae705ca0ca4a35a6) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17887) TestAcidGuarantees fails frequently
[
https://issues.apache.org/jira/browse/HBASE-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991383#comment-15991383
]
Hadoop QA commented on HBASE-17887:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 32s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
58m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 154m 52s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
4s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 242m 8s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.client.TestAsyncTableBatch |
| | org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream |
| | org.apache.hadoop.hbase.client.TestMetaWithReplicas |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient |
| | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
| | org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865769/HBASE-17887.v0.patch |
| JIRA Issue | HBASE-17887 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c988aaf47afe 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 94c14ad |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6646/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6646/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-16466) HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
[ https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991344#comment-15991344 ] Maddineni Sukumar commented on HBASE-16466: --- And I fixed below two minor issues found during this in this same patch(I hope its fine, if not i will put diff patch) #HBASE-17900 - VerifyReplication - input param variables declared as static causing issues to run VerifyReplication multiple times in single JVM(mainly for unit tests) #HBASE-17899 - VerifyReplication not exiting for invalid arguments even though we have check for it. > HBase snapshots support in VerifyReplication tool to reduce load on live > HBase cluster with large tables > > > Key: HBASE-16466 > URL: https://issues.apache.org/jira/browse/HBASE-16466 > Project: HBase > Issue Type: Improvement > Components: hbase >Affects Versions: 0.98.21 >Reporter: Sukumar Maddineni >Assignee: Maddineni Sukumar > Fix For: 2.0.0 > > Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch, > HBASE-16466.v2.patch, HBASE-16466.v3.patch > > > As of now VerifyReplicatin tool is running using normal HBase scanners. If > you want to run VerifyReplication multiple times on a production live > cluster with large tables then it creates extra load on HBase layer. So if we > implement snapshot based support then both in source and target we can read > data from snapshots which reduces load on HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991340#comment-15991340
]
Sean Busbey commented on HBASE-15199:
-
General approach looks good.
{quote}
./bin/hbase:313:14: error: Double quote array expansions to avoid re-splitting
elements. [SC2068]
./bin/hbase:314:19: warning: Quote the rhs of == in [[ ]] to prevent glob
matching. [SC2053]
./bin/hbase:321:29: warning: Brace expansions and globs are literal in
assignments. Quote it or use an array. [SC2125]
{quote}
Could you correct these shellcheck errors please?
{code}
+ # add JRUBY_PACKAGED_WITH_HBASE to CLASSPATH when jruby is needed
+ JRUBY_PACKAGED_WITH_HBASE=$HBASE_HOME/lib/ruby/*
{code}
Shouldn't this be just the jruby jar, rather than e.g. also the ruby sources.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15199: Status: In Progress (was: Patch Available) > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14286) Fix spelling error in "safetyBumper" parameter in WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991290#comment-15991290
]
Chia-Ping Tsai commented on HBASE-14286:
Would you please rebase it? Thanks. [~gliptak]
> Fix spelling error in "safetyBumper" parameter in WALSplitter
> -
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (HBASE-17636) Fix speling [sic] error in enable replication script output
[
https://issues.apache.org/jira/browse/HBASE-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-17636.
Resolution: Duplicate
duplicate, as [~Jan Hentschel] said.
> Fix speling [sic] error in enable replication script output
> ---
>
> Key: HBASE-17636
> URL: https://issues.apache.org/jira/browse/HBASE-17636
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.3.1
>Reporter: Lars George
>
> When enabling the replication for a table:
> {noformat}
> hbase(main):012:0> enable_table_replication 'repltest'
> 0 row(s) in 7.6080 seconds
> The replication swith of table 'repltest' successfully enabled
> {noformat}
> See {{swith}} as opposed to {{switch}}. Also, that sentence is somewhat too
> complicated. Better is maybe {{Replication for table successfully
> enabled.}}?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17981) Roll list_quota_violations into list_quota_snapshots
Josh Elser created HBASE-17981: -- Summary: Roll list_quota_violations into list_quota_snapshots Key: HBASE-17981 URL: https://issues.apache.org/jira/browse/HBASE-17981 Project: HBase Issue Type: Sub-task Reporter: Josh Elser Assignee: Josh Elser Fix For: HBASE-16961 [~apurtell] had the good suggestion on the parent issue to consolidate the functionality of these two commands into one, with uniform output and some filtering options. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16961) FileSystem Quotas
[
https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991278#comment-15991278
]
Josh Elser commented on HBASE-16961:
bq. Regardless of the usefulness, the extra columns missing on
list_quota_violations might be due to an internal change (RS's used to only see
the violation policy or nothing – they see the full snapshot now).
bq. Suggestion for filtering options:
Actually, thinking about this more, I like this idea. Can also add a
{{REGIONSERVER}} option to differentiate getting the state from the quota table
and asking a RS for its view. One less command, more functionality on the one
command. Thanks, Andrew!
> FileSystem Quotas
> -
>
> Key: HBASE-16961
> URL: https://issues.apache.org/jira/browse/HBASE-16961
> Project: HBase
> Issue Type: New Feature
>Reporter: Josh Elser
>Assignee: Josh Elser
> Attachments: hbase-quota-test.sh
>
>
> Umbrella issue for tracking the filesystem utilization of HBase data,
> defining quotas on that utilization, and enforcement when utilization exceeds
> the limits of the quota.
> At a high level: we can define quotas on tables and namespaces. Region size
> is computed by RegionServers and sent to the Master. The Master inspects the
> sizes of Regions, rolling up to table and namespace sizes. Defined quotas in
> the quota table are evaluated given the computed sizes, and, for those
> tables/namespaces violating the quota, RegionServers are informed to take
> some action to limit any further filesystem growth by that table/namespace.
> Discuss:
> https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E
> Design Doc:
> http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or
> https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16466) HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
[ https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991274#comment-15991274 ] Maddineni Sukumar commented on HBASE-16466: --- FYI [~lhofhansl] / [~churromorales] > HBase snapshots support in VerifyReplication tool to reduce load on live > HBase cluster with large tables > > > Key: HBASE-16466 > URL: https://issues.apache.org/jira/browse/HBASE-16466 > Project: HBase > Issue Type: Improvement > Components: hbase >Affects Versions: 0.98.21 >Reporter: Sukumar Maddineni >Assignee: Maddineni Sukumar > Fix For: 2.0.0 > > Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch, > HBASE-16466.v2.patch, HBASE-16466.v3.patch > > > As of now VerifyReplicatin tool is running using normal HBase scanners. If > you want to run VerifyReplication multiple times on a production live > cluster with large tables then it creates extra load on HBase layer. So if we > implement snapshot based support then both in source and target we can read > data from snapshots which reduces load on HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16961) FileSystem Quotas
[ https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991268#comment-15991268 ] Josh Elser commented on HBASE-16961: bq. Well one thing that immediately comes to mind is list_quota_snapshots and list_quota_violations should have the same output format. The difference is filtering applied to the latter. Not really, actually. The former is polling the quota table to list the recorded quota "snapshot" objects. This snapshot includes the space usage, the space limit, and the quota policy (if the use is greater than the limit). The latter command (list_quota_violations) is asking a specific regionserver for the list of quotas that are in violation on that regionserver. The more I think about it, the latter is just not generally useful for operators (only useful when there is a bug and we are investigating why some regionserver is not rejecting requests). Regardless of the usefulness, the extra columns missing on list_quota_violations might be due to an internal change (RS's used to only see the violation policy or nothing -- they see the full snapshot now). > FileSystem Quotas > - > > Key: HBASE-16961 > URL: https://issues.apache.org/jira/browse/HBASE-16961 > Project: HBase > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser > Attachments: hbase-quota-test.sh > > > Umbrella issue for tracking the filesystem utilization of HBase data, > defining quotas on that utilization, and enforcement when utilization exceeds > the limits of the quota. > At a high level: we can define quotas on tables and namespaces. Region size > is computed by RegionServers and sent to the Master. The Master inspects the > sizes of Regions, rolling up to table and namespace sizes. Defined quotas in > the quota table are evaluated given the computed sizes, and, for those > tables/namespaces violating the quota, RegionServers are informed to take > some action to limit any further filesystem growth by that table/namespace. > Discuss: > https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E > Design Doc: > http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or > https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-13860) Remove units from ServerMetricsTmpl.jamon since values are formatted human readable
[ https://issues.apache.org/jira/browse/HBASE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-13860: --- Resolution: Duplicate Status: Resolved (was: Patch Available) fixed by HBASE-15586 > Remove units from ServerMetricsTmpl.jamon since values are formatted human > readable > --- > > Key: HBASE-13860 > URL: https://issues.apache.org/jira/browse/HBASE-13860 > Project: HBase > Issue Type: Bug > Components: regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Gabor Liptak >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.2 > > Attachments: HBASE-13860.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14286) Fix spelling error in "safetyBumper" parameter in WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991254#comment-15991254
]
Hadoop QA commented on HBASE-14286:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color}
| {color:red} HBASE-14286 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12799165/HBASE-14286.2.patch |
| JIRA Issue | HBASE-14286 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6650/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix spelling error in "safetyBumper" parameter in WALSplitter
> -
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991248#comment-15991248 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #154 (See [https://builds.apache.org/job/HBase-1.3-JDK7/154/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 12415f8bde2402ce96ba1450bc73104e62e2df7c) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14286) Fix spelling error in "safetyBumper" parameter in WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991234#comment-15991234
]
Chia-Ping Tsai commented on HBASE-14286:
+1
> Fix spelling error in "safetyBumper" parameter in WALSplitter
> -
>
> Key: HBASE-14286
> URL: https://issues.apache.org/jira/browse/HBASE-14286
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Attachments: HBASE-14286.1.patch, HBASE-14286.2.patch
>
>
> In {{WALSplitter]] we have this code:
> {code}
> public static long writeRegionSequenceIdFile(final FileSystem fs, final
> Path regiondir,
> long newSeqId, long saftyBumper) throws IOException {
> {code}
> We should fix the parameter name to be {{safetyBumper}}. Same for the JavaDoc
> above the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17938) General fault - tolerance framework for backup/restore operations
[ https://issues.apache.org/jira/browse/HBASE-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991233#comment-15991233 ] Vladimir Rodionov commented on HBASE-17938: --- [~tedyu], when you have a time please take a look at the above RB submission. > General fault - tolerance framework for backup/restore operations > - > > Key: HBASE-17938 > URL: https://issues.apache.org/jira/browse/HBASE-17938 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-17938-v1.patch, HBASE-17938-v2.patch, > HBASE-17938-v3.patch > > > The framework must take care of all general types of failures during backup/ > restore and restore system to the original state in case of a failure. > That won't solve all the possible issues but we have a separate JIRAs for > them as a sub-tasks of HBASE-15277 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991216#comment-15991216 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #168 (See [https://builds.apache.org/job/HBase-1.3-JDK8/168/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 12415f8bde2402ce96ba1450bc73104e62e2df7c) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991204#comment-15991204 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #128 (See [https://builds.apache.org/job/HBase-1.2-JDK7/128/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 66e2aa9d9d561f08513db498e839d7f85a520342) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991195#comment-15991195 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1947 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1947/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 8436292eb24b62d6b0d32b0051295a4725245b46) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991190#comment-15991190 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #124 (See [https://builds.apache.org/job/HBase-1.2-JDK8/124/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 66e2aa9d9d561f08513db498e839d7f85a520342) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991179#comment-15991179 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1864 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1864/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 8436292eb24b62d6b0d32b0051295a4725245b46) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17973:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Sean, and the input, Ashish!
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991157#comment-15991157 ] Hudson commented on HBASE-17968: FAILURE: Integrated in Jenkins build HBase-1.4 #712 (See [https://builds.apache.org/job/HBase-1.4/712/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 30c8b380208cbdbde2fb28cd07f4a5fe1901109a) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991156#comment-15991156
]
Sean Busbey commented on HBASE-17973:
-
+1
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17954) Switch findbugs implementation to spotbugs
[ https://issues.apache.org/jira/browse/HBASE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel updated HBASE-17954: -- Status: Patch Available (was: In Progress) > Switch findbugs implementation to spotbugs > -- > > Key: HBASE-17954 > URL: https://issues.apache.org/jira/browse/HBASE-17954 > Project: HBase > Issue Type: Task > Components: build, community, test >Reporter: Sean Busbey >Assignee: Jan Hentschel > Fix For: 2.0.0 > > Attachments: HBASE-17954.master.001.patch > > > The Hadoop folks got some nice finds out of switching their findbugs plugin > to use the new spotbugs fork instead, let's try it out too. (ref > HADOOP-14316 for details) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17954) Switch findbugs implementation to spotbugs
[ https://issues.apache.org/jira/browse/HBASE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel updated HBASE-17954: -- Attachment: HBASE-17954.master.001.patch > Switch findbugs implementation to spotbugs > -- > > Key: HBASE-17954 > URL: https://issues.apache.org/jira/browse/HBASE-17954 > Project: HBase > Issue Type: Task > Components: build, community, test >Reporter: Sean Busbey >Assignee: Jan Hentschel > Fix For: 2.0.0 > > Attachments: HBASE-17954.master.001.patch > > > The Hadoop folks got some nice finds out of switching their findbugs plugin > to use the new spotbugs fork instead, let's try it out too. (ref > HADOOP-14316 for details) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17976) Remove stability annotation from public audience-marked classes
[ https://issues.apache.org/jira/browse/HBASE-17976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17976: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks, Ted. Pushed to the branch. > Remove stability annotation from public audience-marked classes > --- > > Key: HBASE-17976 > URL: https://issues.apache.org/jira/browse/HBASE-17976 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17976.001.HBASE-16961.patch > > > Looks like I grabbed HBASE-17857 in the last rebase, but missed this test > failing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991125#comment-15991125 ] Sean Busbey commented on HBASE-16196: - No replies on the user@hbase thread ( https://s.apache.org/DXQK ) ever happened. I'm +1 on closing HBASE-13338 as a duplicate of this issue if no one else has objections. [~mmullins], I'd like to get someone else to move this forward for 2.0 if you don't mind. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Matt Mullins >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[
https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991124#comment-15991124
]
Hadoop QA commented on HBASE-17534:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 14s
{color} | {color:red} Docker failed to build yetus/hbase:e01ee2f. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865779/HBASE-17534.004.branch-1.patch
|
| JIRA Issue | HBASE-17534 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6648/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
>
>
> Key: HBASE-17534
> URL: https://issues.apache.org/jira/browse/HBASE-17534
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11
>
> Attachments: HBASE-17534.001.branch-1.patch,
> HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch,
> HBASE-17534.004.branch-1.patch
>
>
> While writing some tests against 1.x, I noticed that what should have been a
> DoNotRetryIOException sent to the client from a RegionServer was getting
> retried until it reached the hbase client retries limit.
> Upon inspection, I found that the SecureBulkLoadClient was wrapping all
> Exceptions from the RPC as an IOException. I believe this is creating a case
> where the RPC system doesn't notice that there's a DNRIOException wrapped
> beneath it, thinking it's a transient error.
> This results in clients having to wait for the retry limit to be reached
> before they get acknowledgement that something failed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HBASE-17954) Switch findbugs implementation to spotbugs
[ https://issues.apache.org/jira/browse/HBASE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel reassigned HBASE-17954: - Assignee: Jan Hentschel > Switch findbugs implementation to spotbugs > -- > > Key: HBASE-17954 > URL: https://issues.apache.org/jira/browse/HBASE-17954 > Project: HBase > Issue Type: Task > Components: build, community, test >Reporter: Sean Busbey >Assignee: Jan Hentschel > Fix For: 2.0.0 > > > The Hadoop folks got some nice finds out of switching their findbugs plugin > to use the new spotbugs fork instead, let's try it out too. (ref > HADOOP-14316 for details) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Work started] (HBASE-17954) Switch findbugs implementation to spotbugs
[ https://issues.apache.org/jira/browse/HBASE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-17954 started by Jan Hentschel. - > Switch findbugs implementation to spotbugs > -- > > Key: HBASE-17954 > URL: https://issues.apache.org/jira/browse/HBASE-17954 > Project: HBase > Issue Type: Task > Components: build, community, test >Reporter: Sean Busbey >Assignee: Jan Hentschel > Fix For: 2.0.0 > > > The Hadoop folks got some nice finds out of switching their findbugs plugin > to use the new spotbugs fork instead, let's try it out too. (ref > HADOOP-14316 for details) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[
https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991098#comment-15991098
]
Hadoop QA commented on HBASE-17534:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 14s
{color} | {color:red} Docker failed to build yetus/hbase:e01ee2f. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865779/HBASE-17534.004.branch-1.patch
|
| JIRA Issue | HBASE-17534 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6647/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
>
>
> Key: HBASE-17534
> URL: https://issues.apache.org/jira/browse/HBASE-17534
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11
>
> Attachments: HBASE-17534.001.branch-1.patch,
> HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch,
> HBASE-17534.004.branch-1.patch
>
>
> While writing some tests against 1.x, I noticed that what should have been a
> DoNotRetryIOException sent to the client from a RegionServer was getting
> retried until it reached the hbase client retries limit.
> Upon inspection, I found that the SecureBulkLoadClient was wrapping all
> Exceptions from the RPC as an IOException. I believe this is creating a case
> where the RPC system doesn't notice that there's a DNRIOException wrapped
> beneath it, thinking it's a transient error.
> This results in clients having to wait for the retry limit to be reached
> before they get acknowledgement that something failed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[ https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17534: --- Status: Patch Available (was: Open) > SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server > > > Key: HBASE-17534 > URL: https://issues.apache.org/jira/browse/HBASE-17534 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17534.001.branch-1.patch, > HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch, > HBASE-17534.004.branch-1.patch > > > While writing some tests against 1.x, I noticed that what should have been a > DoNotRetryIOException sent to the client from a RegionServer was getting > retried until it reached the hbase client retries limit. > Upon inspection, I found that the SecureBulkLoadClient was wrapping all > Exceptions from the RPC as an IOException. I believe this is creating a case > where the RPC system doesn't notice that there's a DNRIOException wrapped > beneath it, thinking it's a transient error. > This results in clients having to wait for the retry limit to be reached > before they get acknowledgement that something failed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[ https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17534: --- Attachment: HBASE-17534.004.branch-1.patch .004 Same as .003, just retriggering hadoop-qa since it's been 2months > SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server > > > Key: HBASE-17534 > URL: https://issues.apache.org/jira/browse/HBASE-17534 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17534.001.branch-1.patch, > HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch, > HBASE-17534.004.branch-1.patch > > > While writing some tests against 1.x, I noticed that what should have been a > DoNotRetryIOException sent to the client from a RegionServer was getting > retried until it reached the hbase client retries limit. > Upon inspection, I found that the SecureBulkLoadClient was wrapping all > Exceptions from the RPC as an IOException. I believe this is creating a case > where the RPC system doesn't notice that there's a DNRIOException wrapped > beneath it, thinking it's a transient error. > This results in clients having to wait for the retry limit to be reached > before they get acknowledgement that something failed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[ https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17534: --- Fix Version/s: (was: 2.0.0) > SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server > > > Key: HBASE-17534 > URL: https://issues.apache.org/jira/browse/HBASE-17534 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17534.001.branch-1.patch, > HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch > > > While writing some tests against 1.x, I noticed that what should have been a > DoNotRetryIOException sent to the client from a RegionServer was getting > retried until it reached the hbase client retries limit. > Upon inspection, I found that the SecureBulkLoadClient was wrapping all > Exceptions from the RPC as an IOException. I believe this is creating a case > where the RPC system doesn't notice that there's a DNRIOException wrapped > beneath it, thinking it's a transient error. > This results in clients having to wait for the retry limit to be reached > before they get acknowledgement that something failed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17534) SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server
[ https://issues.apache.org/jira/browse/HBASE-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991069#comment-15991069 ] Josh Elser commented on HBASE-17534: bq. It looks like the same issue is present in master, could you make a version of the patch for that branch? Actually, I don't think this is an issue for master. Since the secure bulk load implementation is now using the standard client RPC model in Master (e.g. RegionServerCallable, RpcRetryingFactory) instead of the coprocessor RPC implementation, this works correctly. Let me just work on rebasing the patch for the 1.x lines. > SecureBulkLoadClient squashes DoNotRetryIOExceptions from the server > > > Key: HBASE-17534 > URL: https://issues.apache.org/jira/browse/HBASE-17534 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17534.001.branch-1.patch, > HBASE-17534.002.branch-1.patch, HBASE-17534.003.branch-1.patch > > > While writing some tests against 1.x, I noticed that what should have been a > DoNotRetryIOException sent to the client from a RegionServer was getting > retried until it reached the hbase client retries limit. > Upon inspection, I found that the SecureBulkLoadClient was wrapping all > Exceptions from the RPC as an IOException. I believe this is creating a case > where the RPC system doesn't notice that there's a DNRIOException wrapped > beneath it, thinking it's a transient error. > This results in clients having to wait for the retry limit to be reached > before they get acknowledgement that something failed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991063#comment-15991063
]
Hadoop QA commented on HBASE-17973:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
60m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m 11s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 46s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865768/HBASE-17973.002.patch
|
| JIRA Issue | HBASE-17973 |
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux da4a2e21296c 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 94c14ad |
| Default Java | 1.8.0_121 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6645/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6645/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17976) Remove stability annotation from public audience-marked classes
[ https://issues.apache.org/jira/browse/HBASE-17976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990989#comment-15990989 ] Ted Yu commented on HBASE-17976: +1 > Remove stability annotation from public audience-marked classes > --- > > Key: HBASE-17976 > URL: https://issues.apache.org/jira/browse/HBASE-17976 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17976.001.HBASE-16961.patch > > > Looks like I grabbed HBASE-17857 in the last rebase, but missed this test > failing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17980) Any HRegionInfo we give out should be immutable
Chia-Ping Tsai created HBASE-17980: -- Summary: Any HRegionInfo we give out should be immutable Key: HBASE-17980 URL: https://issues.apache.org/jira/browse/HBASE-17980 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Chia-Ping Tsai Fix For: 2.0.0 This is similar to HBASE-15583. # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. # Deprecate HRegionInfo to be removed in 3.0 # RegionInfo contain all of the read-only methods of HRegionInfo # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990988#comment-15990988 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #35 (See [https://builds.apache.org/job/HBase-1.3-IT/35/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 12415f8bde2402ce96ba1450bc73104e62e2df7c) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990986#comment-15990986 ] Hudson commented on HBASE-17968: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #863 (See [https://builds.apache.org/job/HBase-1.2-IT/863/]) HBASE-17968 Fix NOTICE.txt for src-release (elserj: rev 66e2aa9d9d561f08513db498e839d7f85a520342) * (edit) NOTICE.txt > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17887) TestAcidGuarantees fails frequently
[ https://issues.apache.org/jira/browse/HBASE-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-17887: --- Status: Patch Available (was: Open) > TestAcidGuarantees fails frequently > --- > > Key: HBASE-17887 > URL: https://issues.apache.org/jira/browse/HBASE-17887 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.4.1 > > Attachments: HBASE-17887.branch-1.v0.patch, > HBASE-17887.branch-1.v1.patch, HBASE-17887.branch-1.v1.patch, > HBASE-17887.v0.patch > > > As per the flaky tests dashboard here: > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html, > It fails 30% of the time. > While working on HBASE-17863, a few verification builds on patch failed due > to TestAcidGuarantees didn't pass. IMHO, the changes for HBASE-17863 are > unlikely to affect get/ put path. > I ran the test with and without the patch several times locally and found > that TestAcidGuarantees fails without the patch similar number of times. > Opening blocker, considering acid guarantees are critical to HBase. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17887) TestAcidGuarantees fails frequently
[ https://issues.apache.org/jira/browse/HBASE-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-17887: --- Attachment: HBASE-17887.v0.patch I run the TestAcidGuarantees 50 times with v0 patch on 2.0. All pass. > TestAcidGuarantees fails frequently > --- > > Key: HBASE-17887 > URL: https://issues.apache.org/jira/browse/HBASE-17887 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.4.1 > > Attachments: HBASE-17887.branch-1.v0.patch, > HBASE-17887.branch-1.v1.patch, HBASE-17887.branch-1.v1.patch, > HBASE-17887.v0.patch > > > As per the flaky tests dashboard here: > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html, > It fails 30% of the time. > While working on HBASE-17863, a few verification builds on patch failed due > to TestAcidGuarantees didn't pass. IMHO, the changes for HBASE-17863 are > unlikely to affect get/ put path. > I ran the test with and without the patch several times locally and found > that TestAcidGuarantees fails without the patch similar number of times. > Opening blocker, considering acid guarantees are critical to HBase. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17968) Update copyright year in NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17968: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) bq. (lol, we missed all of 2016? jeez.) :) Thanks, Nick and Sean. Pushed to all branches. > Update copyright year in NOTICE file > > > Key: HBASE-17968 > URL: https://issues.apache.org/jira/browse/HBASE-17968 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.1.10 >Reporter: Nick Dimiduk >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-17968.001.patch > > > From the VOTE thread on 1.1.10RC0, in [~elserj]'s vote: > bq. NOTICE file needs an update (copyright year). The notes about "Licensed > under Apache License, version 2.0" also appear unnecessary to me (but are > only "bad" in that it unnecessarily bloats our NOTICE). > Should do an audit on all branches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17973) Create shell command to identify regions with poor locality
[
https://issues.apache.org/jira/browse/HBASE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17973:
---
Attachment: HBASE-17973.002.patch
.002 Avoids some extra getters/local-variables that aren't needed if we're
ignoring a region due to the locality filter.
Thanks for the feedback, Sean.
> Create shell command to identify regions with poor locality
> ---
>
> Key: HBASE-17973
> URL: https://issues.apache.org/jira/browse/HBASE-17973
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17973.001.patch, HBASE-17973.002.patch
>
>
> The data locality of regions often plays a large role in the efficiency of
> HBase. Compactions are also expensive to execute, especially on very large
> tables. The balancer can do a good job trying to maintain locality (when
> tuned properly), but it is not perfect.
> This creates a less-than-desirable situation where it's a costly operation to
> take a cluster with spotty poor locality (e.g. a small percentage of
> regionservers with poor locality).
> We already have this information available via the {{ClusterStatus}} proto.
> We can easily write a shell command that can present regions which are
> lacking a certain percentage of locality.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17957) Custom metrics of replicate endpoints don't prepend "source." to global metrics
[
https://issues.apache.org/jira/browse/HBASE-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17957:
---
Labels: incompatibleChange (was: )
Please fill in release note
> Custom metrics of replicate endpoints don't prepend "source." to global
> metrics
>
>
> Key: HBASE-17957
> URL: https://issues.apache.org/jira/browse/HBASE-17957
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.4.0, 0.98.22
>Reporter: Shinya Yoshida
>Assignee: Shinya Yoshida
>Priority: Minor
> Labels: incompatibleChange
> Attachments: HBASE-17957.master.001.patch,
> HBASE-17957.master.002.patch
>
>
> Custom metrics for custom replication endpoints is introduced by
> [HBASE-16448].
> The name of local custom metrics follows the "source.id.metricsName" format,
> but the name of global custom metrics doesn't follow the "source.metricsName"
> format.
> Ex:
> {code}
> // default metrics
> "source.2.shippedOps" : 1234, // peer local
> "source.shippedOps" : 12345, // global
> // custom metrics
> "source.1.failed.start" : 1, // peer local
> "failed.start" : 1, // global
> {code}
> When we consider that default metrics do so, it should be
> "source.metricsName" like:
> {code}
> "source.1.failed.start" : 1,
> "source.failed.start" : 1,
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990906#comment-15990906
]
Hadoop QA commented on HBASE-15199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 10s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 12s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
5s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m 8s
{color} | {color:red} The patch generated 3 new + 498 unchanged - 0 fixed = 501
total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
55m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 124m 26s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
38s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 211m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865755/HBASE-15199.master.002.patch
|
| JIRA Issue | HBASE-15199 |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
xml |
| uname | Linux dce17c242ef5 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1848353 |
| Default Java | 1.8.0_121 |
| shellcheck | v0.4.6 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/diff-patch-shellcheck.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/testReport/ |
| modules | C: hbase-assembly . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/console |
| Powered by |
[jira] [Commented] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[
https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990904#comment-15990904
]
Hadoop QA commented on HBASE-16138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s {color}
| {color:red} HBASE-16138 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822361/HBASE-16138-v1.patch |
| JIRA Issue | HBASE-16138 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6644/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Cannot open regions after non-graceful shutdown due to deadlock with
> Replication Table
> --
>
> Key: HBASE-16138
> URL: https://issues.apache.org/jira/browse/HBASE-16138
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
>Reporter: Joseph
>Assignee: Ashu Pachauri
>Priority: Critical
> Attachments: HBASE-16138.patch, HBASE-16138-v1.patch
>
>
> If we shutdown an entire HBase cluster and attempt to start it back up, we
> have to run the WAL pre-log roll that occurs before opening up a region. Yet
> this pre-log roll must record the new WAL inside of ReplicationQueues. This
> method call ends up blocking on
> TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the
> Replication Table is not up yet. And we cannot assign the Replication Table
> because we cannot open any regions. This ends up deadlocking the entire
> cluster whenever we lose Replication Table availability.
> There are a few options that we can do, but none of them seem very good:
> 1. Depend on Zookeeper-based Replication until the Replication Table becomes
> available
> 2. Have a separate WAL for System Tables that does not perform any
> replication (see discussion at HBASE-14623)
> Or just have a seperate WAL for non-replicated vs replicated
> regions
> 3. Record the WAL log in the ReplicationQueue asynchronously (don't block
> opening a region on this event), which could lead to inconsistent Replication
> state
> The stacktrace:
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376)
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348)
>
> org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370)
>
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637)
>
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701)
>
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600)
>
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533)
>
> org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132)
>
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186)
>
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197)
> org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240)
>
> org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883)
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
>
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> Does anyone have any suggestions/ideas/feedback?
> Attached a review board at: https://reviews.apache.org/r/50546/
> It is still pretty rough, would just like some feedback on it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990882#comment-15990882 ] Maddineni Sukumar edited comment on HBASE-16138 at 5/1/17 2:41 PM: --- Hi [~Vegetable26] / [~ashu210890], Are you still working on this? I see some pending work in review board. If not then I am interested in this and I can spend some time on this. And if you can give me some pointers on pending/done tasks that would be great. Thanks Sukumar was (Author: sukuna...@gmail.com): Hi [~Vegetable26] , Are you still working on this? I see some pending work in review board. If not then I am interested in this and I can spend some time on this. And if you can give me some pointers on pending/done tasks that would be great. Thanks Sukumar > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri >Priority: Critical > Attachments: HBASE-16138.patch, HBASE-16138-v1.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990882#comment-15990882 ] Maddineni Sukumar commented on HBASE-16138: --- Hi [~Vegetable26] , Are you still working on this? I see some pending work in review board. If not then I am interested in this and I can spend some time on this. And if you can give me some pointers on pending/done tasks that would be great. Thanks Sukumar > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri >Priority: Critical > Attachments: HBASE-16138.patch, HBASE-16138-v1.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16466) HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
[ https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990837#comment-15990837 ] Maddineni Sukumar commented on HBASE-16466: --- [~apurtell] , could you please take a look at this please. > HBase snapshots support in VerifyReplication tool to reduce load on live > HBase cluster with large tables > > > Key: HBASE-16466 > URL: https://issues.apache.org/jira/browse/HBASE-16466 > Project: HBase > Issue Type: Improvement > Components: hbase >Affects Versions: 0.98.21 >Reporter: Sukumar Maddineni >Assignee: Maddineni Sukumar > Fix For: 2.0.0 > > Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch, > HBASE-16466.v2.patch, HBASE-16466.v3.patch > > > As of now VerifyReplicatin tool is running using normal HBase scanners. If > you want to run VerifyReplication multiple times on a production live > cluster with large tables then it creates extra load on HBase layer. So if we > implement snapshot based support then both in source and target we can read > data from snapshots which reduces load on HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990735#comment-15990735 ] Xiang Li edited comment on HBASE-15199 at 5/1/17 1:47 PM: -- Updated patch 002 for master branch to address [~busbey]'s comments. The logic with the patch is (1) if JRUBY_HOME is defined, CLASSPATH is updated according to JRUBY_HOME defined (2) if JRUBY_HOME is not defined, check if the command issued belongs to a pre-defined command set (currently it contains shell and org.jruby.Main). A. if yes, add jruby jar packaged with HBase into CLASSPATH B. if no, do nothing There is a behavior change when comparing with original logic(without the patch) (1) without the patch, JRUBY_HOME and JRUBY_OPTS only takes effect when "hbase shell", that is, it does not take effect when "hbase org.jruby.Main xxx.rb“. It is a bug I believe (2) with the patch, JRUBY_HOME takes effect, for all commands, as long as it is specified. When JRUBY_HOME is specified, the jruby-complete jar packaged with HBase will be ignored. Make any sense to you? [~busbey], [~stack], [~jinghe] was (Author: water): Updated patch 002 for master branch to address [~busbey]'s comments! The logic with the patch is (1) if JRUBY_HOME is defined, CLASSPATH is updated according to JRUBY_HOME defined (2) if JRUBY_HOME is not defined, check if the command issued belongs to a pre-defined command set (currently it contains shell and org.jruby.Main). A. if yes, add jruby jar packaged with HBase into CLASSPATH B. if no, do nothing There is a behavior change when comparing with original logic(without the patch) (1) without the patch, JRUBY_HOME and JRUBY_OPTS only takes effect when "hbase shell", that is, it does not take effect when "hbase org.jruby.Main xxx.rb“. It is a bug I believe (2) with the patch, JRUBY_HOME takes effect, for all commands, as long as it is specified. When JRUBY_HOME is specified, the jruby-complete jar packaged with HBase will be ignored. Make any sense to you? [~busbey], [~stack], [~jinghe] > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
