[jira] [Commented] (HBASE-20968) list_procedures_test fails due to no matching regex
[
https://issues.apache.org/jira/browse/HBASE-20968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594553#comment-16594553
]
Duo Zhang commented on HBASE-20968:
---
If no objections let's finish this issue.
> list_procedures_test fails due to no matching regex
> ---
>
> Key: HBASE-20968
> URL: https://issues.apache.org/jira/browse/HBASE-20968
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Major
> Attachments: 20968.v2.txt, HBASE-20968-branch-2.1.patch,
> HBASE-20968.patch, org.apache.hadoop.hbase.client.TestShell-output.txt
>
>
> From test output against hadoop3:
> {code}
> 2018-07-28 12:04:24,838 DEBUG [Time-limited test]
> procedure2.ProcedureExecutor(948): Stored pid=12, state=RUNNABLE,
> hasLock=false; org.apache.hadoop.hbase.client.procedure.
> ShellTestProcedure
> 2018-07-28 12:04:24,864 INFO [RS-EventLoopGroup-1-3]
> ipc.ServerRpcConnection(556): Connection from 172.18.128.12:46918,
> version=3.0.0-SNAPSHOT, sasl=false, ugi=hbase (auth: SIMPLE),
> service=MasterService
> 2018-07-28 12:04:24,900 DEBUG [Thread-114] master.MasterRpcServices(1157):
> Checking to see if procedure is done pid=11
> ^[[38;5;196mF^[[0m
> ===
> Failure:
> ^[[48;5;124;38;5;231;1mtest_list_procedures(Hbase::ListProceduresTest)^[[0m
> src/test/ruby/shell/list_procedures_test.rb:65:in `block in
> test_list_procedures'
> 62: end
> 63: end
> 64:
> ^[[48;5;124;38;5;231;1m => 65: assert_equal(1, matching_lines)^[[0m
> 66: end
> 67: end
> 68: end
> <^[[48;5;34;38;5;231;1m1^[[0m> expected but was
> <^[[48;5;124;38;5;231;1m0^[[0m>
> ===
> ...
> 2018-07-28 12:04:25,374 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1316): Finished pid=12, state=SUCCESS,
> hasLock=false; org.apache.hadoop.hbase.client.procedure.
> ShellTestProcedure in 336msec
> {code}
> The completion of the ShellTestProcedure was after the assertion was raised.
> {code}
> def create_procedure_regexp(table_name)
> regexp_string = '[0-9]+ .*ShellTestProcedure SUCCESS.*' \
> {code}
> The regex used by the test isn't found in test output either.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21084) When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online
[
https://issues.apache.org/jira/browse/HBASE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594549#comment-16594549
]
Duo Zhang commented on HBASE-21084:
---
Let me try the patch.
> When cloning a snapshot including a split parent region, the split parent
> region of the cloned table will be online
> ---
>
> Key: HBASE-21084
> URL: https://issues.apache.org/jira/browse/HBASE-21084
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21084.branch-2.001.patch,
> HBASE-21084.branch-2.addendum.001.patch, HBASE-21084.master.001.patch,
> HBASE-21084.master.002.patch, HBASE-21084.master.003.patch,
> HBASE-21084.master.004.patch, HBASE-21084.master.addendum.001.patch
>
>
> Investigating HBASE-21015, I found another issue. It seems like after
> HBASE-20881, the split parent region of the cloned table will be online when
> cloning a snapshot including a split parent region.
> Steps to reproduce are as follows, which is the same as the steps in
> HBASE-21015:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some data into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After following the above steps, the split parent region of the cloned table
> will be online.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21017) Revisit the expected states for open/close
[ https://issues.apache.org/jira/browse/HBASE-21017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594547#comment-16594547 ] Duo Zhang commented on HBASE-21017: --- They can pass locally. I think the problem is that they are a bit slow so on a poor machine they will timeout... > Revisit the expected states for open/close > -- > > Key: HBASE-21017 > URL: https://issues.apache.org/jira/browse/HBASE-21017 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21017-debug.patch, HBASE-21017-v1.patch, > HBASE-21017-v2.patch, HBASE-21017-v3.patch, HBASE-21017.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21030) Correct javadoc for append operation
[
https://issues.apache.org/jira/browse/HBASE-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-21030:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Correct javadoc for append operation
>
>
> Key: HBASE-21030
> URL: https://issues.apache.org/jira/browse/HBASE-21030
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 1.5.0
>Reporter: Nihal Jain
>Assignee: Subrat Mishra
>Priority: Minor
> Labels: beginner, beginners
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 1.2.7, 2.1.1
>
> Attachments: HBASE-21030.master.001.patch
>
>
> The doc for {{append}} operation is incorrect. (see {{@param append}} in the
> code snippet below or
> [Table.java#L566|https://github.com/apache/hbase/blob/3f5033f88ee9da2a5a42d058b9aefe57b089b3e1/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java#L566])
> {code:java}
> /**
>* Appends values to one or more columns within a single row.
>*
>* This operation guaranteed atomicity to readers. Appends are done
>* under a single row lock, so write operations to a row are synchronized,
> and
>* readers are guaranteed to see this operation fully completed.
>*
>* @param append object that specifies the columns and amounts to be used
>* for the increment operations
>* @throws IOException e
>* @return values of columns after the append operation (maybe null)
>*/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21030) Correct javadoc for append operation
[
https://issues.apache.org/jira/browse/HBASE-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594545#comment-16594545
]
Toshihiro Suzuki commented on HBASE-21030:
--
Sure [~stack]. Thank you for letting me know. So closing this Jira. Thanks.
> Correct javadoc for append operation
>
>
> Key: HBASE-21030
> URL: https://issues.apache.org/jira/browse/HBASE-21030
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 1.5.0
>Reporter: Nihal Jain
>Assignee: Subrat Mishra
>Priority: Minor
> Labels: beginner, beginners
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 1.2.7, 2.1.1
>
> Attachments: HBASE-21030.master.001.patch
>
>
> The doc for {{append}} operation is incorrect. (see {{@param append}} in the
> code snippet below or
> [Table.java#L566|https://github.com/apache/hbase/blob/3f5033f88ee9da2a5a42d058b9aefe57b089b3e1/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java#L566])
> {code:java}
> /**
>* Appends values to one or more columns within a single row.
>*
>* This operation guaranteed atomicity to readers. Appends are done
>* under a single row lock, so write operations to a row are synchronized,
> and
>* readers are guaranteed to see this operation fully completed.
>*
>* @param append object that specifies the columns and amounts to be used
>* for the increment operations
>* @throws IOException e
>* @return values of columns after the append operation (maybe null)
>*/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21017) Revisit the expected states for open/close
[ https://issues.apache.org/jira/browse/HBASE-21017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594542#comment-16594542 ] Duo Zhang commented on HBASE-21017: --- Let me check. > Revisit the expected states for open/close > -- > > Key: HBASE-21017 > URL: https://issues.apache.org/jira/browse/HBASE-21017 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21017-debug.patch, HBASE-21017-v1.patch, > HBASE-21017-v2.patch, HBASE-21017-v3.patch, HBASE-21017.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21017) Revisit the expected states for open/close
[ https://issues.apache.org/jira/browse/HBASE-21017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594539#comment-16594539 ] stack commented on HBASE-21017: --- Are the failures related? Let me look at latest up on rb... > Revisit the expected states for open/close > -- > > Key: HBASE-21017 > URL: https://issues.apache.org/jira/browse/HBASE-21017 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21017-debug.patch, HBASE-21017-v1.patch, > HBASE-21017-v2.patch, HBASE-21017-v3.patch, HBASE-21017.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594537#comment-16594537 ] stack commented on HBASE-6721: -- Sounds good [~daisuke.kobayashi]. Open subtask or new issue? Have you tried it sir? > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt, > hbase-6721-v28.patch, hbase-6721-v28.patch, hbase-6721-v29.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594536#comment-16594536 ] zhangyanshan commented on HBASE-21122: -- [~yuzhih...@gmail.com] test updated in HBASE-21122.002.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21120) MoveRegionProcedure makes no progress; goes to STUCK
[
https://issues.apache.org/jira/browse/HBASE-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594535#comment-16594535
]
Hudson commented on HBASE-21120:
Results for branch branch-2.1
[build #252 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/252/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/252//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/252//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/252//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> MoveRegionProcedure makes no progress; goes to STUCK
>
>
> Key: HBASE-21120
> URL: https://issues.apache.org/jira/browse/HBASE-21120
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.2
>
> Attachments: HBASE-21120.apply-in-reverse.txt
>
>
> This is tip of branch-2.0. I'm running cluster tests. I've seen this twice
> now where a slowDeterministic schedules a move procedure just after startup
> and we go dead-in-the-water never going beyond schedule of the unassign.We
> even schedule subsequent moves but they go no where either.
> {code}
> ...
> 2018-08-26 10:19:30,352 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=286, ppid=260, state=SUCCESS; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0530.halxg.cloudera.com,16020,1535303391139 in 10.8100sec
> ...
> // MASTER KILLED
>
> 2018-08-26 10:21:10,585 INFO [Thread-19] assignment.RegionStateStore: Load
> hbase:meta entry region=fb79d389cf324aaf3ad4b686d3ad21d5, regionState=OPEN,
> lastHost=ve0530.halxg.cloudera.com,16020,1535303391139,
> regionLocation=ve0530.halxg.cloudera.com,16020,1535303391139, openSeqNum=42095
> ...
> 2018-08-26 10:26:15,531 INFO
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
> 2018-08-26 10:26:16,635 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=413,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:16,896 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=405, state=SUCCESS; ServerCrashProcedure
> server=ve0540.halxg.cloudera.com,16020,1535303391120, splitWal=true,
> meta=true in 25.8450sec
> 2018-08-26 10:26:16,896 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=414, ppid=413,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=fb79d389cf324aaf3ad4b686d3ad21d5,
> server=ve0530.halxg.cloudera.com,16020,1535303391139 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:29,308 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000]
> master.ServerManager: Registering
> regionserver=ve0540.halxg.cloudera.com,16020,1535304386779
> 2018-08-26
> 10:26:29,342 INFO [RegionServerTracker-0] master.RegionServerTracker:
> RegionServer ephemeral node created, adding
> [ve0540.halxg.cloudera.com,16020,1535304386779]
>2018-08-26
> 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819]
> zookeeper.ZooKeeper: Session: 0x16577368c840054 closed
> 2018-08-26 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819-EventThread]
> zookeeper.ClientCnxn: EventThread shut down for session: 0x16577368c840054
>
> 2018-08-26 10:27:16,411 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000] master.HMaster:
>
[jira] [Updated] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyanshan updated HBASE-21122: - Attachment: (was: HBASE-21122.002.patch) > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyanshan updated HBASE-21122: - Attachment: HBASE-21122.002.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21030) Correct javadoc for append operation
[
https://issues.apache.org/jira/browse/HBASE-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594525#comment-16594525
]
stack commented on HBASE-21030:
---
Its good to close [~brfrn169]
FYI, doc goes to master branch and then when we go to cut a release, we copy
back from master branch editing out inappropriate parts. Not the best but thats
how it is currently sir.
So, your commit to master means it'll get picked up by earlier branches.
Thanks.
> Correct javadoc for append operation
>
>
> Key: HBASE-21030
> URL: https://issues.apache.org/jira/browse/HBASE-21030
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 1.5.0
>Reporter: Nihal Jain
>Assignee: Subrat Mishra
>Priority: Minor
> Labels: beginner, beginners
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 1.2.7, 2.1.1
>
> Attachments: HBASE-21030.master.001.patch
>
>
> The doc for {{append}} operation is incorrect. (see {{@param append}} in the
> code snippet below or
> [Table.java#L566|https://github.com/apache/hbase/blob/3f5033f88ee9da2a5a42d058b9aefe57b089b3e1/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java#L566])
> {code:java}
> /**
>* Appends values to one or more columns within a single row.
>*
>* This operation guaranteed atomicity to readers. Appends are done
>* under a single row lock, so write operations to a row are synchronized,
> and
>* readers are guaranteed to see this operation fully completed.
>*
>* @param append object that specifies the columns and amounts to be used
>* for the increment operations
>* @throws IOException e
>* @return values of columns after the append operation (maybe null)
>*/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21124) SCP stuck unable to split WALs (NoServerDispatchException and STUCK regions in CLOSING in logs)
stack created HBASE-21124:
-
Summary: SCP stuck unable to split WALs (NoServerDispatchException
and STUCK regions in CLOSING in logs)
Key: HBASE-21124
URL: https://issues.apache.org/jira/browse/HBASE-21124
Project: HBase
Issue Type: Bug
Components: amv2
Reporter: stack
At first blush, we go to dispatch an unassign to a server that is not online, a
NoServerDispatchException is thrown. The server is not online, so a SCP is NOT
scheduled and the region close does not go to completion. Eventually regions
complain they are STUCK.
{code}
2018-08-27 15:43:41,578 WARN [PEWorker-13]
assignment.RegionTransitionProcedure: Remote call failed
ve0530.halxg.cloudera.com,16020,1535387669315; pid=20723, ppid=20578,
state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
table=IntegrationTestBigLinkedList, region=464487fa9329df991cae84519adf253c,
server=ve0530.halxg.cloudera.com,16020,1535387669315; rit=CLOSING,
location=ve0530.halxg.cloudera.com,16020,1535387669315;
exception=NoServerDispatchException
org.apache.hadoop.hbase.procedure2.NoServerDispatchException:
ve0530.halxg.cloudera.com,16020,1535387669315; pid=20723, ppid=20578,
state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
table=IntegrationTestBigLinkedList, region=464487fa9329df991cae84519adf253c,
server=ve0530.halxg.cloudera.com,16020,1535387669315
at
org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.addOperationToNode(RemoteProcedureDispatcher.java:177)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.addToRemoteDispatcher(RegionTransitionProcedure.java:259)
at
org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:207)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:345)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:97)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:876)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1556)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1344)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1855)
2018-08-27 15:43:41,578 WARN [PEWorker-16]
assignment.UnassignProcedure: Expiring
ve0530.halxg.cloudera.com,16020,1535387669315, pid=20720, ppid=20580,
state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
table=IntegrationTestBigLinkedList, region=aa45bd6f1aabd0ee778c2664cca76d5e,
server=ve0530.halxg.cloudera.com,16020,1535387669315 rit=CLOSING,
location=ve0530.halxg.cloudera.com,16020,1535387669315;
exception=NoServerDispatchException
2018-08-27
15:43:41,578 WARN [PEWorker-16] master.ServerManager: Expiration of
ve0530.halxg.cloudera.com,16020,1535387669315 but server not online
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594520#comment-16594520 ] Vishal Khandelwal commented on HBASE-20940: --- [~apurtell] I have completed the patch and executing alll test. Somehow getting OOM while executing alltest. So trying to run small, large and medium separately. should be able to update patch by EOD. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20942) Improve RpcServer TRACE logging
[
https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594519#comment-16594519
]
Hadoop QA commented on HBASE-20942:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 1s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}200m 45s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}239m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestSplitTransactionOnCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20942 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937359/HBASE-20942.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1ae0d79e874d 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5089256529 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14234/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14234/testReport/ |
| Max. process+thread count | 4692 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output
[jira] [Commented] (HBASE-20942) Improve RpcServer TRACE logging
[
https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594511#comment-16594511
]
Hadoop QA commented on HBASE-20942:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 28s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}175m
48s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}210m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20942 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937360/HBASE-20942.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c7f06ac7cb3d 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5089256529 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14235/testReport/ |
| Max. process+thread count | 4778 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14235/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Improve RpcServer TRACE logging
>
[jira] [Commented] (HBASE-21084) When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online
[
https://issues.apache.org/jira/browse/HBASE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594509#comment-16594509
]
Toshihiro Suzuki commented on HBASE-21084:
--
It looks like the QA is okay. Could you please review the addendum patches for
mater and branch-2? [~Apache9] [~yuzhih...@gmail.com]
If you are okay with them, I'll push them.
> When cloning a snapshot including a split parent region, the split parent
> region of the cloned table will be online
> ---
>
> Key: HBASE-21084
> URL: https://issues.apache.org/jira/browse/HBASE-21084
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21084.branch-2.001.patch,
> HBASE-21084.branch-2.addendum.001.patch, HBASE-21084.master.001.patch,
> HBASE-21084.master.002.patch, HBASE-21084.master.003.patch,
> HBASE-21084.master.004.patch, HBASE-21084.master.addendum.001.patch
>
>
> Investigating HBASE-21015, I found another issue. It seems like after
> HBASE-20881, the split parent region of the cloned table will be online when
> cloning a snapshot including a split parent region.
> Steps to reproduce are as follows, which is the same as the steps in
> HBASE-21015:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some data into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After following the above steps, the split parent region of the cloned table
> will be online.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21045) Make a 2.0.2 release
[ https://issues.apache.org/jira/browse/HBASE-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594507#comment-16594507 ] stack commented on HBASE-21045: --- * Tagged dc53d30876fefc9a88367ef30ea784add0a14efa as 2.0.2RC0 (Will push later). * ./dev-support/checkcompatibility.py --annotation org.apache.hadoop.hbase.classification.InterfaceAudience.Public rel/2.0.1 2.0.2RC0 * Tried out the product; made sure I could build from src tarball and checked out bin tarball layout, ran it, loaded it, checked data loaded. * Signed it * Checked out release up on repository.apache.org in staging.. then closed the repo. * Sent out VOTE email. > Make a 2.0.2 release > > > Key: HBASE-21045 > URL: https://issues.apache.org/jira/browse/HBASE-21045 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Fix For: 2.0.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21030) Correct javadoc for append operation
[
https://issues.apache.org/jira/browse/HBASE-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594500#comment-16594500
]
Toshihiro Suzuki commented on HBASE-21030:
--
Sure [~stack]. So should I keep this Jira open? Or we can close it? I've
already pushed to all branches except branch-2.0.
> Correct javadoc for append operation
>
>
> Key: HBASE-21030
> URL: https://issues.apache.org/jira/browse/HBASE-21030
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 1.5.0
>Reporter: Nihal Jain
>Assignee: Subrat Mishra
>Priority: Minor
> Labels: beginner, beginners
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 1.2.7, 2.1.1
>
> Attachments: HBASE-21030.master.001.patch
>
>
> The doc for {{append}} operation is incorrect. (see {{@param append}} in the
> code snippet below or
> [Table.java#L566|https://github.com/apache/hbase/blob/3f5033f88ee9da2a5a42d058b9aefe57b089b3e1/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java#L566])
> {code:java}
> /**
>* Appends values to one or more columns within a single row.
>*
>* This operation guaranteed atomicity to readers. Appends are done
>* under a single row lock, so write operations to a row are synchronized,
> and
>* readers are guaranteed to see this operation fully completed.
>*
>* @param append object that specifies the columns and amounts to be used
>* for the increment operations
>* @throws IOException e
>* @return values of columns after the append operation (maybe null)
>*/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21123) Commit 2.0.2 RELEASENOTES and CHANGES
[
https://issues.apache.org/jira/browse/HBASE-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594478#comment-16594478
]
Hudson commented on HBASE-21123:
Results for branch branch-2.0
[build #741 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Commit 2.0.2 RELEASENOTES and CHANGES
> -
>
> Key: HBASE-21123
> URL: https://issues.apache.org/jira/browse/HBASE-21123
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Ran {{./release-doc-maker/releasedocmaker.py -p HBASE --fileversions -v 2.0.2
> -l --sortorder=newer --skip-credits}} then carefully stitched the product
> into the current CHANGES.md and RELEASENOTES.md files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20941) Create and implement HbckService in master
[
https://issues.apache.org/jira/browse/HBASE-20941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594476#comment-16594476
]
Hudson commented on HBASE-20941:
Results for branch branch-2.0
[build #741 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Create and implement HbckService in master
> --
>
> Key: HBASE-20941
> URL: https://issues.apache.org/jira/browse/HBASE-20941
> Project: HBase
> Issue Type: Sub-task
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: hbase-20941.master.001.patch,
> hbase-20941.master.002.patch, hbase-20941.master.003.patch,
> hbase-20941.master.004.patch, hbase-20941.master.004.patch,
> hbase-20941.master.004.patch
>
>
> Create HbckService in master and implement following methods:
> # setTableState(): If table state are inconsistent with action/ procedures
> working on them, sometimes manipulating their states in meta fix things.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21120) MoveRegionProcedure makes no progress; goes to STUCK
[
https://issues.apache.org/jira/browse/HBASE-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594477#comment-16594477
]
Hudson commented on HBASE-21120:
Results for branch branch-2.0
[build #741 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/741//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> MoveRegionProcedure makes no progress; goes to STUCK
>
>
> Key: HBASE-21120
> URL: https://issues.apache.org/jira/browse/HBASE-21120
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.2
>
> Attachments: HBASE-21120.apply-in-reverse.txt
>
>
> This is tip of branch-2.0. I'm running cluster tests. I've seen this twice
> now where a slowDeterministic schedules a move procedure just after startup
> and we go dead-in-the-water never going beyond schedule of the unassign.We
> even schedule subsequent moves but they go no where either.
> {code}
> ...
> 2018-08-26 10:19:30,352 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=286, ppid=260, state=SUCCESS; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0530.halxg.cloudera.com,16020,1535303391139 in 10.8100sec
> ...
> // MASTER KILLED
>
> 2018-08-26 10:21:10,585 INFO [Thread-19] assignment.RegionStateStore: Load
> hbase:meta entry region=fb79d389cf324aaf3ad4b686d3ad21d5, regionState=OPEN,
> lastHost=ve0530.halxg.cloudera.com,16020,1535303391139,
> regionLocation=ve0530.halxg.cloudera.com,16020,1535303391139, openSeqNum=42095
> ...
> 2018-08-26 10:26:15,531 INFO
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
> 2018-08-26 10:26:16,635 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=413,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:16,896 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=405, state=SUCCESS; ServerCrashProcedure
> server=ve0540.halxg.cloudera.com,16020,1535303391120, splitWal=true,
> meta=true in 25.8450sec
> 2018-08-26 10:26:16,896 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=414, ppid=413,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=fb79d389cf324aaf3ad4b686d3ad21d5,
> server=ve0530.halxg.cloudera.com,16020,1535303391139 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:29,308 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000]
> master.ServerManager: Registering
> regionserver=ve0540.halxg.cloudera.com,16020,1535304386779
> 2018-08-26
> 10:26:29,342 INFO [RegionServerTracker-0] master.RegionServerTracker:
> RegionServer ephemeral node created, adding
> [ve0540.halxg.cloudera.com,16020,1535304386779]
>2018-08-26
> 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819]
> zookeeper.ZooKeeper: Session: 0x16577368c840054 closed
> 2018-08-26 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819-EventThread]
> zookeeper.ClientCnxn: EventThread shut down for session: 0x16577368c840054
>
> 2018-08-26 10:27:16,411 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[
https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594475#comment-16594475
]
Hadoop QA commented on HBASE-20734:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
6s{color} | {color:red} hbase-server: The patch generated 10 new + 387
unchanged - 5 fixed = 397 total (was 392) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 13s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}236m 59s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}278m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.tool.TestLoadIncrementalHFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20734 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937348/HBASE-20734.master.006.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8083c13155c8 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-21072) Block out HBCK1 in hbase2
[
https://issues.apache.org/jira/browse/HBASE-21072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594471#comment-16594471
]
Hudson commented on HBASE-21072:
Results for branch branch-2.1
[build #251 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/251/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/251//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/251//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/251//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/251//console].
> Block out HBCK1 in hbase2
> -
>
> Key: HBASE-21072
> URL: https://issues.apache.org/jira/browse/HBASE-21072
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 2.0.1
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: HBASE-21072-addendum.patch,
> HBASE-21072.branch-2.0.001.patch, HBASE-21072.branch-2.0.002.patch,
> HBASE-21072.branch-2.0.003.patch, HBASE-21072.branch-2.0.003.patch
>
>
> [~busbey] left a note in the parent issue that I only just read which has a
> prescription for how we might block hbck1 from running against an hbase-2.x
> (hbck1 could damage a hbase-2Its disabled in hbase-2 but an errant hbck1
> from an hbase-1.x install might run).
> Here is quote from parent issue:
> {code}
> I was idly thinking about how to stop HBase v1 HBCK. Thanks to HBASE-11405,
> we know that all HBase 1.y.z hbck instances should refuse to run if there's a
> lock file at '/hbase/hbase-hbck.lock' (given defaults). How about HBase v2
> places that file permanently in place and replace the contents (usually just
> an IP address) with a note about how you must not run HBase v1 HBCK against
> the cluster?
> {code}
> There is also the below:
> {code}
> We could pick another location for locking on HBase version 2 and start
> building in a version check of some kind?
> {code}
> ... to which I'd answer, lets see. hbck2 is a different beast. It asks the
> master to do stuff. It doesn't do it itself, as hbck1 did. So no need of a
> lock/version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-21088.
---
Resolution: Fixed
Fix Version/s: 2.0.3
2.1.1
Re-Resolving after pushing to branch-2.0 and branch-2.1
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21017) Revisit the expected states for open/close
[
https://issues.apache.org/jira/browse/HBASE-21017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594463#comment-16594463
]
Hadoop QA commented on HBASE-21017:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 8s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}241m 15s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}275m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestSpaceQuotas |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.tool.TestLoadIncrementalHFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21017 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937346/HBASE-21017-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3d20055b3957 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5089256529 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14231/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14231/testReport/ |
| Max. process+thread count |
[jira] [Commented] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594453#comment-16594453 ] Ted Yu commented on HBASE-21122: Can you take a look at other tests under the same directory, such as: hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseCatalogSuite.scala Your test * misses license header * should extend FunSuite > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594449#comment-16594449 ] zhangyanshan edited comment on HBASE-21122 at 8/28/18 3:07 AM: --- [~yuzhih...@gmail.com] test added in HBASE-21122.002.patch was (Author: zhangyanshan): test added in HBASE-21122.002.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594449#comment-16594449 ] zhangyanshan commented on HBASE-21122: -- test added in HBASE-21122.002.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyanshan updated HBASE-21122: - Attachment: HBASE-21122.002.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch, > HBASE-21122.002.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyanshan updated HBASE-21122: - Attachment: (was: HBASE-21122.000.patch) > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21122) bulkload multi-version Cell supported in hbase-spark module
[ https://issues.apache.org/jira/browse/HBASE-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyanshan updated HBASE-21122: - Attachment: HBASE-21122.000.patch > bulkload multi-version Cell supported in hbase-spark module > > > Key: HBASE-21122 > URL: https://issues.apache.org/jira/browse/HBASE-21122 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: 1.1.2, 2.1.1 >Reporter: zhangyanshan >Priority: Major > Labels: features > Attachments: HBASE-21122.000.patch, HBASE-21122.001.patch > > > bulkload multi-version Cell not supported in HBaseContext.scala, this issue > will add function of bulkload multi-version Cell -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20941) Create and implement HbckService in master
[
https://issues.apache.org/jira/browse/HBASE-20941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594406#comment-16594406
]
Hudson commented on HBASE-20941:
Results for branch branch-2
[build #1174 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1174/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1174//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1174//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1174//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Create and implement HbckService in master
> --
>
> Key: HBASE-20941
> URL: https://issues.apache.org/jira/browse/HBASE-20941
> Project: HBase
> Issue Type: Sub-task
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: hbase-20941.master.001.patch,
> hbase-20941.master.002.patch, hbase-20941.master.003.patch,
> hbase-20941.master.004.patch, hbase-20941.master.004.patch,
> hbase-20941.master.004.patch
>
>
> Create HbckService in master and implement following methods:
> # setTableState(): If table state are inconsistent with action/ procedures
> working on them, sometimes manipulating their states in meta fix things.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594403#comment-16594403 ] Mingliang Liu commented on HBASE-21070: --- Last, I think it might be helpful to add a UT. Thanks, > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[
https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594402#comment-16594402
]
Mingliang Liu commented on HBASE-21070:
---
The patch looks good to me overall to remove the dependency on last modified
time of top snapshort dir, which is not necessarily updated per [Hadoop
FileSystem
contract|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/introduction.html#Timestamps].
The {{readLock}} is not used and can be removed. If all access to the {{cache}}
and {{snapshots}} are accessed via {{synchronized}}, we don't need the
{{volatile}} keyword here. {{triggerCacheRefreshForTesting()}} can be
{{synchronized}} I think.
> SnapshotFileCache won't update for snapshots stored in S3
> -
>
> Key: HBASE-21070
> URL: https://issues.apache.org/jira/browse/HBASE-21070
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0, 2.1.1, 1.4.7
>Reporter: Zach York
>Assignee: Zach York
>Priority: Critical
> Labels: FSRedo
> Attachments: HBASE-21070.master.001.patch,
> HBASE-21070.master.002.patch
>
>
> The SnapshotFileCache depends on last modified time to determine whether to
> update the Snapshot HFile cache. However, in S3, real 'folders' don't exist.
> S3 filesystems create a dummy file in place of a folder, but the dummy file
> last modified time is not updated when files are changed 'under' it. This
> means that the SnapshotFileCache doesn't pick up new snapshot HFiles and
> these files aren't removed from the HFileCleaner and can be eligible for
> deletion.
>
> My patch removes the lastmodified assumption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[
https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594390#comment-16594390
]
Hadoop QA commented on HBASE-20734:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
33s{color} | {color:red} hbase-server: The patch generated 10 new + 387
unchanged - 5 fixed = 397 total (was 392) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 0s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestDisableTableProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20734 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937345/HBASE-20734.master.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fbbe5b0d8681 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision |
[jira] [Reopened] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reopened HBASE-21088:
---
Reopen for backport. I'll do it.
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594383#comment-16594383 ] stack commented on HBASE-21075: --- I can help w/ the refguide bit. Could try it myself first so I knew what I was talking about (smile). > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1, 2.0.3 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594380#comment-16594380 ] Duo Zhang commented on HBASE-21075: --- I will try to write a UT first. And also test it on a cluster. And then we need to open a issue to write down the instructions in our ref guide. > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1, 2.0.3 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594377#comment-16594377
]
Ted Yu commented on HBASE-21088:
I would think those two branches should have this fix.
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594376#comment-16594376 ] stack commented on HBASE-21075: --- Yeah, you are right. I should have marked it 2.0.3. Whats to be done here to prove this mechanism will work [~Apache9]? Thanks. > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1, 2.0.3 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594375#comment-16594375
]
stack commented on HBASE-21088:
---
[~yuzhih...@gmail.com] Should this be in 2.1.x and 2.0.x?
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[
https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594373#comment-16594373
]
Hadoop QA commented on HBASE-20734:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
38s{color} | {color:red} hbase-server: The patch generated 14 new + 661
unchanged - 10 fixed = 675 total (was 671) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 52s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 26m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.TestHeapSize |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20734 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937352/HBASE-20734.branch-1.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cdfed8c8640c
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krish Dey updated HBASE-20942: -- Status: Open (was: Patch Available) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch, HBASE-20942.005.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krish Dey updated HBASE-20942: -- Attachment: (was: HBASE-20942.005.patch) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch, HBASE-20942.005.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krish Dey updated HBASE-20942: -- Attachment: HBASE-20942.005.patch Status: Patch Available (was: Open) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch, HBASE-20942.005.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[ https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594362#comment-16594362 ] Mingliang Liu commented on HBASE-20642: --- Thanks [~an...@apache.org]. After discussion with [~apurtell] on related work HBASE-20408, I think this will fix branch-1 client unexpected exception as well. I can help review the patch for branch-1. > IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException > - > > Key: HBASE-20642 > URL: https://issues.apache.org/jira/browse/HBASE-20642 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.2 > > Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch, > HBASE-20642.patch > > > [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing > while adding column family during the time master is restarting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krish Dey updated HBASE-20942: -- Attachment: HBASE-20942.005.patch Status: Patch Available (was: Open) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch, HBASE-20942.005.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krish Dey updated HBASE-20942: -- Status: Open (was: Patch Available) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594354#comment-16594354
]
Daisuke Kobayashi commented on HBASE-6721:
--
Revisiting this as I am wondering if we could evaluate and commit the master
WebUI patch {{6721-master-webUI.patch}}, which was posted by [~zjushch] but not
committed yet. Thoughts?
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: Francis Liu
>Assignee: Francis Liu
>Priority: Major
> Labels: hbase-6721
> Fix For: 2.0.0
>
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch,
> hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt,
> hbase-6721-v28.patch, hbase-6721-v28.patch, hbase-6721-v29.patch,
> immediateAssignments Sequence Diagram.svg, randomAssignment Sequence
> Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment
> Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[ https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594353#comment-16594353 ] Ankit Singhal commented on HBASE-20642: --- bq. Does it make sense to branch-1 as well? Thanks. [~liuml07] , [~elserj] , Yes, actually nonce changes are applicable as the handling of retry with nonce is also not correct in branch-1. Do you want me to backport it for branch-1? > IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException > - > > Key: HBASE-20642 > URL: https://issues.apache.org/jira/browse/HBASE-20642 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.2 > > Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch, > HBASE-20642.patch > > > [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing > while adding column family during the time master is restarting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594351#comment-16594351
]
Reid Chan commented on HBASE-20993:
---
ping [~apurtell], would you want to take a look at branch-1.v9?
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.2.001.patch,
> HBASE-20993.branch-1.wip.002.patch, HBASE-20993.branch-1.wip.patch,
> yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
>
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594349#comment-16594349
]
Reid Chan commented on HBASE-20993:
---
Please starting working on master branch:D
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.2.001.patch,
> HBASE-20993.branch-1.wip.002.patch, HBASE-20993.branch-1.wip.patch,
> yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by:
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[
https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594345#comment-16594345
]
Hadoop QA commented on HBASE-21070:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} hbase-server: The patch generated 0 new + 1
unchanged - 1 fixed = 1 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 26s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 9s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestDisableTableProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21070 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937337/HBASE-21070.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 44fa2998392d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5089256529 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14229/artifact/patchprocess/patch-unit-hbase-server.txt
|
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594341#comment-16594341
]
Hadoop QA commented on HBASE-20306:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 1 new + 8 unchanged
- 0 fixed = 9 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 40s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.assignment.TestCloseRegionWhileRSCrash |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20306 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937332/HBASE-20306.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 67d830b41771 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5089256529 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14228/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14228/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20734: -- Attachment: HBASE-20734.branch-1.002.patch > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.branch-1.002.patch, HBASE-20734.master.001.patch, > HBASE-20734.master.002.patch, HBASE-20734.master.003.patch, > HBASE-20734.master.004.patch, HBASE-20734.master.005.patch, > HBASE-20734.master.006.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[
https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594333#comment-16594333
]
Ted Yu commented on HBASE-20734:
Took a quick look at patch v6.
{code}
+ public HRegionFileSystem getRegionWALFileSystem() throws IOException {
{code}
The above is only used by HRegion and doesn't need to be public.
{code}
+ public Path getWALRegionDir() throws IOException {
+if (regionDir == null) {
{code}
You use lazy initialization because the field is only used in tests ?
{code}
+ for (Path file: filesUnderRootDir) {
+if (!rootFS.delete(file, false)) {
+ LOG.error("Failed delete of " + file);
{code}
Probably mention in the log that the file was under root dir.
Once reviewboard is updated, I will review again.
thanks
> Colocate recovered edits directory with hbase.wal.dir
> -
>
> Key: HBASE-20734
> URL: https://issues.apache.org/jira/browse/HBASE-20734
> Project: HBase
> Issue Type: Improvement
> Components: MTTR, Recovery, wal
>Reporter: Ted Yu
>Assignee: Zach York
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20734.branch-1.001.patch,
> HBASE-20734.master.001.patch, HBASE-20734.master.002.patch,
> HBASE-20734.master.003.patch, HBASE-20734.master.004.patch,
> HBASE-20734.master.005.patch, HBASE-20734.master.006.patch
>
>
> During investigation of HBASE-20723, I realized that we wouldn't get the best
> performance when hbase.wal.dir is configured to be on different (fast) media
> than hbase rootdir w.r.t. recovered edits since recovered edits directory is
> currently under rootdir.
> Such setup may not result in fast recovery when there is region server
> failover.
> This issue is to find proper (hopefully backward compatible) way in
> colocating recovered edits directory with hbase.wal.dir .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594325#comment-16594325
]
Hadoop QA commented on HBASE-21098:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 1 fixed = 283 total (was 284) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}199m 17s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
10s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}264m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21098 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937324/HBASE-21098.master.006.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2bedc8ea190a 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594323#comment-16594323 ] Zach York commented on HBASE-20734: --- [~yuzhih...@gmail.com] did you have any comments on the master patch? > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch, HBASE-20734.master.006.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594319#comment-16594319 ] Zach York commented on HBASE-20734: --- Attached a rebased patch since the checkstyle didn't seem related to some of my edits. I'll fix when it comes back. I'll work on updating the branch-1 patch now. > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch, HBASE-20734.master.006.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594316#comment-16594316
]
Hudson commented on HBASE-21088:
Results for branch branch-2
[build #1173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21071) HBaseTestingUtility::startMiniCluster() to use builder pattern
[
https://issues.apache.org/jira/browse/HBASE-21071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594317#comment-16594317
]
Hudson commented on HBASE-21071:
Results for branch branch-2
[build #1173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBaseTestingUtility::startMiniCluster() to use builder pattern
> --
>
> Key: HBASE-21071
> URL: https://issues.apache.org/jira/browse/HBASE-21071
> Project: HBase
> Issue Type: Improvement
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>Priority: Major
> Labels: reviewed
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21071.000.patch, HBASE-21071.001.patch,
> HBASE-21071.002.patch, HBASE-21071.003.patch, HBASE-21071.004.patch,
> HBASE-21071.005.patch, HBASE-21071.006.patch, HBASE-21071.006.patch,
> HBASE-21071.006.patch, HBASE-21071.006.patch, HBASE-21071.006.patch,
> HBASE-21071.006.patch, HBASE-21071.branch-2.006.patch
>
>
> Currently there are 13 {{startMiniCluster()}} methods to set up a mini
> cluster. I'm not surprised if we have a few more in future. It's good to
> support different combination of optional parameters. We have to pick up one
> of them carefully while still wondering the default values of other
> parameters; if we add a new option, we may bring more new methods.
> One solution is to use builder pattern: create a class {{MiniClusterOptions}}
> along with a static class {{MiniClusterOptionsBuilder}}, create a new method
> {{startMiniCluster(MiniClusterOptions)}}. In {{master}} we delete the old 13
> methods while in branch-2, we deprecate the old 13 methods.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21072) Block out HBCK1 in hbase2
[
https://issues.apache.org/jira/browse/HBASE-21072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594315#comment-16594315
]
Hudson commented on HBASE-21072:
Results for branch branch-2
[build #1173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1173//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Block out HBCK1 in hbase2
> -
>
> Key: HBASE-21072
> URL: https://issues.apache.org/jira/browse/HBASE-21072
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 2.0.1
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: HBASE-21072-addendum.patch,
> HBASE-21072.branch-2.0.001.patch, HBASE-21072.branch-2.0.002.patch,
> HBASE-21072.branch-2.0.003.patch, HBASE-21072.branch-2.0.003.patch
>
>
> [~busbey] left a note in the parent issue that I only just read which has a
> prescription for how we might block hbck1 from running against an hbase-2.x
> (hbck1 could damage a hbase-2Its disabled in hbase-2 but an errant hbck1
> from an hbase-1.x install might run).
> Here is quote from parent issue:
> {code}
> I was idly thinking about how to stop HBase v1 HBCK. Thanks to HBASE-11405,
> we know that all HBase 1.y.z hbck instances should refuse to run if there's a
> lock file at '/hbase/hbase-hbck.lock' (given defaults). How about HBase v2
> places that file permanently in place and replace the contents (usually just
> an IP address) with a note about how you must not run HBase v1 HBCK against
> the cluster?
> {code}
> There is also the below:
> {code}
> We could pick another location for locking on HBase version 2 and start
> building in a version check of some kind?
> {code}
> ... to which I'd answer, lets see. hbck2 is a different beast. It asks the
> master to do stuff. It doesn't do it itself, as hbck1 did. So no need of a
> lock/version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20734: -- Attachment: HBASE-20734.master.006.patch > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch, HBASE-20734.master.006.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21017) Revisit the expected states for open/close
[ https://issues.apache.org/jira/browse/HBASE-21017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21017: -- Attachment: HBASE-21017-v3.patch > Revisit the expected states for open/close > -- > > Key: HBASE-21017 > URL: https://issues.apache.org/jira/browse/HBASE-21017 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21017-debug.patch, HBASE-21017-v1.patch, > HBASE-21017-v2.patch, HBASE-21017-v3.patch, HBASE-21017.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20734: -- Attachment: HBASE-20734.master.005.patch > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21075: -- Fix Version/s: (was: 2.2.0) 2.0.3 2.1.1 > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1, 2.0.3 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594306#comment-16594306 ] Duo Zhang commented on HBASE-21075: --- Oh I thought you mean the name of the flag... For this issue, it should be a block of 2.1.x and 2.0.x, as we will only commit the patch to these two branches... > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.2.0 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594302#comment-16594302 ] stack commented on HBASE-21075: --- Agree [~Apache9] I changed it. > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.2.0 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21075: -- Fix Version/s: (was: 2.1.1) 2.2.0 > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.2.0 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21123) Commit 2.0.2 RELEASENOTES and CHANGES
stack created HBASE-21123:
-
Summary: Commit 2.0.2 RELEASENOTES and CHANGES
Key: HBASE-21123
URL: https://issues.apache.org/jira/browse/HBASE-21123
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Ran {{./release-doc-maker/releasedocmaker.py -p HBASE --fileversions -v 2.0.2
-l --sortorder=newer --skip-credits}} then carefully stitched the product into
the current CHANGES.md and RELEASENOTES.md files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20704: -- Fix Version/s: (was: 2.0.2) 2.0.3 > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21083: -- Fix Version/s: (was: 2.0.2) 2.0.3 > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 > system, we need something to interfere with stuck procedures before HBCK2 can > work. This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594282#comment-16594282 ] Duo Zhang commented on HBASE-21075: --- HBASE-20881 has been pushed to branch-2, so I think it should be 2.2? > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20940: -- Fix Version/s: (was: 2.0.2) 2.0.3 > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21075) Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after HBASE-20881
[ https://issues.apache.org/jira/browse/HBASE-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21075: -- Fix Version/s: (was: 2.0.2) > Confirm that we can (rolling) upgrade from 2.0.x and 2.1.x to 2.2.x after > HBASE-20881 > - > > Key: HBASE-21075 > URL: https://issues.apache.org/jira/browse/HBASE-21075 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.1.1 > > Attachments: HBASE-21075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20942) Improve RpcServer TRACE logging
[
https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594276#comment-16594276
]
Josh Elser commented on HBASE-20942:
{quote}
Well I could have done it. But I want to pass the configuration for example.
conf.setInt("hbase.ipc.trace.log.max.length", 250);
This will test the scenario that the configuration is used correctly for the
variable traceLogMaxLength.
{quote}
Just pass your configuration object into your method :)
> Improve RpcServer TRACE logging
> ---
>
> Key: HBASE-20942
> URL: https://issues.apache.org/jira/browse/HBASE-20942
> Project: HBase
> Issue Type: Task
>Reporter: Esteban Gutierrez
>Assignee: Krish Dey
>Priority: Major
> Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch,
> HBASE-20942.004.patch
>
>
> Two things:
> * We truncate RpcServer output to 1000 characters for trace logging. Would
> be better if that value was configurable.
> * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE
> log message.
> Esteban mentioned this to me earlier, so I'm crediting him as the reporter.
> cc: [~elserj]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594275#comment-16594275 ] Zach York commented on HBASE-21070: --- patch 2 adds additionally logging suggested by [~stack]. Responded to questions up in RB. > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-21070: -- Attachment: HBASE-21070.master.002.patch > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20942) Improve RpcServer TRACE logging
[
https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594274#comment-16594274
]
Krish Dey commented on HBASE-20942:
---
Hi [~elserj],
Thanks for the advice here.
Will follow it.
On your comment.
{code:java}
RpcServer mockServer = Mockito.mock(RpcServer.class);
Mockito.when(mockServer.truncateTraceLogLength(Mockito.any(String.class))).thenCallRealMethod();
{code}
Well I could have done it. But I want to pass the configuration for example.
conf.setInt("hbase.ipc.trace.log.max.length", 250);
This will test the scenario that the configuration is used correctly for the
variable traceLogMaxLength.
Thanks again.
Krish
> Improve RpcServer TRACE logging
> ---
>
> Key: HBASE-20942
> URL: https://issues.apache.org/jira/browse/HBASE-20942
> Project: HBase
> Issue Type: Task
>Reporter: Esteban Gutierrez
>Assignee: Krish Dey
>Priority: Major
> Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch,
> HBASE-20942.004.patch
>
>
> Two things:
> * We truncate RpcServer output to 1000 characters for trace logging. Would
> be better if that value was configurable.
> * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE
> log message.
> Esteban mentioned this to me earlier, so I'm crediting him as the reporter.
> cc: [~elserj]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21120) MoveRegionProcedure makes no progress; goes to STUCK
[
https://issues.apache.org/jira/browse/HBASE-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-21120.
---
Resolution: Fixed
Assignee: stack
Fix Version/s: 2.0.2
2.1.1
Pushed to branch-2.0 and branch-2.1 which is where HBASE-21078 went.
With this patch, cluster tests get their robustness back.
> MoveRegionProcedure makes no progress; goes to STUCK
>
>
> Key: HBASE-21120
> URL: https://issues.apache.org/jira/browse/HBASE-21120
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.2
>
> Attachments: HBASE-21120.apply-in-reverse.txt
>
>
> This is tip of branch-2.0. I'm running cluster tests. I've seen this twice
> now where a slowDeterministic schedules a move procedure just after startup
> and we go dead-in-the-water never going beyond schedule of the unassign.We
> even schedule subsequent moves but they go no where either.
> {code}
> ...
> 2018-08-26 10:19:30,352 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=286, ppid=260, state=SUCCESS; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0530.halxg.cloudera.com,16020,1535303391139 in 10.8100sec
> ...
> // MASTER KILLED
>
> 2018-08-26 10:21:10,585 INFO [Thread-19] assignment.RegionStateStore: Load
> hbase:meta entry region=fb79d389cf324aaf3ad4b686d3ad21d5, regionState=OPEN,
> lastHost=ve0530.halxg.cloudera.com,16020,1535303391139,
> regionLocation=ve0530.halxg.cloudera.com,16020,1535303391139, openSeqNum=42095
> ...
> 2018-08-26 10:26:15,531 INFO
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
> 2018-08-26 10:26:16,635 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=413,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:16,896 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=405, state=SUCCESS; ServerCrashProcedure
> server=ve0540.halxg.cloudera.com,16020,1535303391120, splitWal=true,
> meta=true in 25.8450sec
> 2018-08-26 10:26:16,896 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=414, ppid=413,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=fb79d389cf324aaf3ad4b686d3ad21d5,
> server=ve0530.halxg.cloudera.com,16020,1535303391139 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:29,308 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000]
> master.ServerManager: Registering
> regionserver=ve0540.halxg.cloudera.com,16020,1535304386779
> 2018-08-26
> 10:26:29,342 INFO [RegionServerTracker-0] master.RegionServerTracker:
> RegionServer ephemeral node created, adding
> [ve0540.halxg.cloudera.com,16020,1535304386779]
>2018-08-26
> 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819]
> zookeeper.ZooKeeper: Session: 0x16577368c840054 closed
> 2018-08-26 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819-EventThread]
> zookeeper.ClientCnxn: EventThread shut down for session: 0x16577368c840054
>
> 2018-08-26 10:27:16,411 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
>
>
>2018-08-26
> 10:27:17,001 INFO [PEWorker-13] procedure.MasterProcedureScheduler: pid=415,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5, source=ve0530.halxg
> .cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
[jira] [Commented] (HBASE-20430) Improve store file management for non-HDFS filesystems
[ https://issues.apache.org/jira/browse/HBASE-20430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594270#comment-16594270 ] Andrew Purtell commented on HBASE-20430: [~liuml07] Sorry I missed this earlier. Yes I think the best first approach is to open references to store files as needed and close them as soon as possible. > Improve store file management for non-HDFS filesystems > -- > > Key: HBASE-20430 > URL: https://issues.apache.org/jira/browse/HBASE-20430 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > > HBase keeps a file open for every active store file so no additional round > trips to the NameNode are needed after the initial open. HDFS internally > multiplexes open files, but the Hadoop S3 filesystem implementations do not, > or, at least, not as well. As the bulk of data under management increases we > observe the required number of concurrently open connections will rise, and > expect it will eventually exhaust a limit somewhere (the client, the OS file > descriptor table or open file limits, or the S3 service). > Initially we can simply introduce an option to close every store file after > the reader has finished, and determine the performance impact. Use cases > backed by non-HDFS filesystems will already have to cope with a different > read performance profile. Based on experiments with the S3 backed Hadoop > filesystems, notably S3A, even with aggressively tuned options simple reads > can be very slow when there are blockcache misses, 15-20 seconds observed for > Get of a single small row, for example. We expect extensive use of the > BucketCache to mitigate in this application already. Could be backed by > offheap storage, but more likely a large number of cache files managed by the > file engine on local SSD storage. If misses are already going to be super > expensive, then the motivation to do more than simply open store files on > demand is largely absent. > Still, we could employ a predictive cache. Where frequent access to a given > store file (or, at least, its store) is predicted, keep a reference to the > store file open. Can keep statistics about read frequency, write it out to > HFiles during compaction, and note these stats when opening the region, > perhaps by reading all meta blocks of region HFiles when opening. Otherwise, > close the file after reading and open again on demand. Need to be careful not > to use ARC or equivalent as cache replacement strategy as it is encumbered. > The size of the cache can be determined at startup after detecting the > underlying filesystem. Eg. setCacheSize(VERY_LARGE_CONSTANT) if (fs > instanceof DistributedFileSystem), so we don't lose much when on HDFS still. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[
https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594269#comment-16594269
]
Josh Elser commented on HBASE-20642:
{quote}Does it make sense to branch-1 as well? Thanks.
{quote}
My thought was that this fix was predicated on some hbase-2.x nonce changes,
but I honestly don't remember anymore.
> IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
> -
>
> Key: HBASE-20642
> URL: https://issues.apache.org/jira/browse/HBASE-20642
> Project: HBase
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.2
>
> Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch,
> HBASE-20642.patch
>
>
> [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing
> while adding column family during the time master is restarting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21120) MoveRegionProcedure makes no progress; goes to STUCK
[
https://issues.apache.org/jira/browse/HBASE-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21120:
--
Attachment: HBASE-21120.apply-in-reverse.txt
> MoveRegionProcedure makes no progress; goes to STUCK
>
>
> Key: HBASE-21120
> URL: https://issues.apache.org/jira/browse/HBASE-21120
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Priority: Major
> Attachments: HBASE-21120.apply-in-reverse.txt
>
>
> This is tip of branch-2.0. I'm running cluster tests. I've seen this twice
> now where a slowDeterministic schedules a move procedure just after startup
> and we go dead-in-the-water never going beyond schedule of the unassign.We
> even schedule subsequent moves but they go no where either.
> {code}
> ...
> 2018-08-26 10:19:30,352 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=286, ppid=260, state=SUCCESS; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0530.halxg.cloudera.com,16020,1535303391139 in 10.8100sec
> ...
> // MASTER KILLED
>
> 2018-08-26 10:21:10,585 INFO [Thread-19] assignment.RegionStateStore: Load
> hbase:meta entry region=fb79d389cf324aaf3ad4b686d3ad21d5, regionState=OPEN,
> lastHost=ve0530.halxg.cloudera.com,16020,1535303391139,
> regionLocation=ve0530.halxg.cloudera.com,16020,1535303391139, openSeqNum=42095
> ...
> 2018-08-26 10:26:15,531 INFO
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
> 2018-08-26 10:26:16,635 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=413,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:16,896 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Finished pid=405, state=SUCCESS; ServerCrashProcedure
> server=ve0540.halxg.cloudera.com,16020,1535303391120, splitWal=true,
> meta=true in 25.8450sec
> 2018-08-26 10:26:16,896 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler: pid=414, ppid=413,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=fb79d389cf324aaf3ad4b686d3ad21d5,
> server=ve0530.halxg.cloudera.com,16020,1535303391139 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> 2018-08-26 10:26:29,308 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000]
> master.ServerManager: Registering
> regionserver=ve0540.halxg.cloudera.com,16020,1535304386779
> 2018-08-26
> 10:26:29,342 INFO [RegionServerTracker-0] master.RegionServerTracker:
> RegionServer ephemeral node created, adding
> [ve0540.halxg.cloudera.com,16020,1535304386779]
>2018-08-26
> 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819]
> zookeeper.ZooKeeper: Session: 0x16577368c840054 closed
> 2018-08-26 10:27:03,820 INFO
> [ReadOnlyZKClient-ve0524.halxg.cloudera.com:@0x1da09819-EventThread]
> zookeeper.ClientCnxn: EventThread shut down for session: 0x16577368c840054
>
> 2018-08-26 10:27:16,411 INFO
> [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16000] master.HMaster:
> Client=stack//10.17.240.20 move hri=fb79d389cf324aaf3ad4b686d3ad21d5,
> source=ve0530.halxg.cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180, running balancer
>
>
>2018-08-26
> 10:27:17,001 INFO [PEWorker-13] procedure.MasterProcedureScheduler: pid=415,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=fb79d389cf324aaf3ad4b686d3ad21d5, source=ve0530.halxg
> .cloudera.com,16020,1535303391139,
> destination=ve0534.halxg.cloudera.com,16020,1535303812180 checking lock on
> fb79d389cf324aaf3ad4b686d3ad21d5
> ...
> {code}
> Looking in UI, it says the below which has NO mention of pid=414 ... like it
> disappeared!!!
> {code}
> 413 WAITING stack MoveRegionProcedure
>
[jira] [Commented] (HBASE-21078) [amv2] CODE-BUG NPE in RTP doing Unassign
[
https://issues.apache.org/jira/browse/HBASE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594268#comment-16594268
]
stack commented on HBASE-21078:
---
I committed .003 instead of .004. .003 does too much. HBASE-21120 takes care of
backing out the changes to UnassignProcedure. It is an amendment to this issue.
> [amv2] CODE-BUG NPE in RTP doing Unassign
> -
>
> Key: HBASE-21078
> URL: https://issues.apache.org/jira/browse/HBASE-21078
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.1
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.2
>
> Attachments: HBASE-21078.branch-2.0.001.patch,
> HBASE-21078.branch-2.0.002.patch, HBASE-21078.branch-2.0.003.patch,
> HBASE-21078.branch-2.0.004.patch, HBASE-21078.branch-2.0.004.patch,
> HBASE-21078.branch-2.0.004.patch
>
>
> Saw this is a run against tip of branch-2.0. The region had just finished
> being split when the move goes to run.
> {code}
> 2018-08-18 16:55:14,908 INFO [PEWorker-2] procedure2.ProcedureExecutor:
> Finished pid=2028, state=SUCCESS, hasLock=false; SplitTableRegionProcedure
> table=IntegrationTestBigLinkedList, parent=c3f199b5af62ae2ff8f8b6426b21d95d,
> daughterA=31ccbf098ae615ce30f28ec84c956b8f,
> daughterB=1890b4c96736f223f31efef11c817c90 in 9.0090sec
> 2018-08-18 16:55:14,908 INFO [PEWorker-16]
> procedure.MasterProcedureScheduler: pid=2038, ppid=2030,
> state=RUNNABLE:MOVE_REGION_UNASSIGN, hasLock=false; MoveRegionProcedure
> hri=c3f199b5af62ae2ff8f8b6426b21d95d,
> source=ve0540.halxg.cloudera.com,16020,1534632630737,
> destination=ve0540.halxg.cloudera.com,16020,1534632630737 checking lock on
> c3f199b5af62ae2ff8f8b6426b21d95d
> 2018-08-18 16:55:14,958 INFO [PEWorker-16] procedure2.ProcedureExecutor:
> Initialized subprocedures=[{pid=2095, ppid=2038,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=false; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d,
> server=ve0540.halxg.cloudera.com,16020,1534632630737}]
> 2018-08-18 16:55:15,008 INFO [PEWorker-3]
> procedure.MasterProcedureScheduler: pid=2095, ppid=2038,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=false; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d,
> server=ve0540.halxg.cloudera.com,16020,1534632630737 checking lock on
> c3f199b5af62ae2ff8f8b6426b21d95d
> 2018-08-18 16:55:15,085 ERROR [PEWorker-3] procedure2.ProcedureExecutor:
> CODE-BUG: Uncaught runtime exception: pid=2095, ppid=2038,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=true; UnassignProcedure
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d,
> server=ve0540.halxg.cloudera.com,16020,1534632630737
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.getOrCreateServer(RegionStates.java:1097)
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.addRegionToServer(RegionStates.java:1125)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1477)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:204)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:345)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:873)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1556)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1344)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1854)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21110) Issues with Unsafe and JDK 11
[
https://issues.apache.org/jira/browse/HBASE-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594263#comment-16594263
]
Andrew Purtell commented on HBASE-21110:
bq. We already hide most of our Unsafe access behind a facade, maybe there is
something we can do to set it up inline.
Yeah I should amend my comment because I was mixing concerns.
Wherever we use Unsafe, we'd pull out the surrounding code into a separate
Maven module, and then implement in another a VarHandle based alternative. This
is partly done now, we just don't have the relevant classes moved out to their
own module. Of the alternative implementations only one would be used over the
lifetime of a JVM process, so the JVM will see the calls through the interface
as monomorphic and they will be inlined.
> Issues with Unsafe and JDK 11
> -
>
> Key: HBASE-21110
> URL: https://issues.apache.org/jira/browse/HBASE-21110
> Project: HBase
> Issue Type: Task
>Reporter: Mike Drob
>Priority: Major
> Fix For: 3.0.0
>
>
> Using Java 11 RC 1, I get the following warning, probably need to add the
> suggested flag to our scripts?
> {noformat}
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ java -version
> java version "11" 2018-09-25
> Java(TM) SE Runtime Environment 18.9 (build 11+28)
> Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ bin/start-hbase.sh
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ cat
> /Users/mdrob/IdeaProjects/hbase/bin/../logs/hbase-mdrob-master-mdrob-MBP.local.out
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/mdrob/IdeaProjects/hbase/hbase-common/target/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594259#comment-16594259 ] Zach York commented on HBASE-21098: --- +1 LGTM. > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.1.1 >Reporter: Tyler Mi >Priority: Major > Attachments: HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594254#comment-16594254 ] Mike Drob commented on HBASE-20306: --- looks good overall from a quick skim, I'll take a closer look later this week! > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594244#comment-16594244
]
Colin Garcia commented on HBASE-20306:
--
{code:java}
2018-08-27 14:29:33,296 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
[W:20] Keys=87549, cols=985.4 K, time=00:00:50 Overall: [keys/s= 1749,
latency=11.34 ms] Current: [keys/s=1732, latency=11.47 ms], wroteUpTo=-1
2018-08-27 14:29:38,299 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
[W:20] Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754,
latency=11.31 ms] Current: [keys/s=1807, latency=10.98 ms], wroteUpTo=-1
2018-08-27 14:29:43,303 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
RUN SUMMARY: W Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754,
latency=11.31 ms] 2018-08-27 14:29:43,324 INFO
[ReadOnlyZKClient-localhost:2181@0x0b6b1987-EventThread] zookeeper.ClientCnxn:
EventThread shut down for session: 0x1657d2d21d30023
{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Colin Garcia
>Priority: Major
> Labels: beginner
> Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch
>
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what happened so users have to infer it from the last status line printed.
> Would be nice to print a real summary with statistics about what was run.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20942) Improve RpcServer TRACE logging
[
https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594242#comment-16594242
]
Josh Elser commented on HBASE-20942:
[~krish.dey], your attempts to mention me directly aren't quite right. Type the
"\@" symbol and then someone's name. A link to my Jira profile doesn't actually
trigger a message to me (but I'm already watching this Jira issue, so I see
everything you do). That said, please be a little more gentle in the comment
deletion. We generally don't like folks doing this – we want a clear history of
what was done and removing comments make that harder.
{code:java}
private class RpcServerImplForTesting extends RpcServer{code}
Why the need for a custom implementation when you're already using Mockito? You
should be able to do something like this:
{code:java}
RpcServer mockServer = Mockito.mock(RpcServer.class);
Mockito.when(mockServer.truncateTraceLogLength(Mockito.any(String.class))).thenCallRealMethod();{code}
{code:java}
+ Logger rpcServerLog =
Logger.getLogger("org.apache.hadoop.hbase.ipc.RpcServer");{code}
Again, use the RpcServer.class, not a String that is the full class name.
nit: how about your method be named {{truncateTraceLog}} instead of
{{truncateTraceLogLength}}?
> Improve RpcServer TRACE logging
> ---
>
> Key: HBASE-20942
> URL: https://issues.apache.org/jira/browse/HBASE-20942
> Project: HBase
> Issue Type: Task
>Reporter: Esteban Gutierrez
>Assignee: Krish Dey
>Priority: Major
> Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch,
> HBASE-20942.004.patch
>
>
> Two things:
> * We truncate RpcServer output to 1000 characters for trace logging. Would
> be better if that value was configurable.
> * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE
> log message.
> Esteban mentioned this to me earlier, so I'm crediting him as the reporter.
> cc: [~elserj]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.001.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594241#comment-16594241 ] Colin Garcia commented on HBASE-20306: -- Added your suggestions! > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594235#comment-16594235
]
Mike Drob commented on HBASE-21061:
---
Soem thoughts:
{noformat}
public VersionInfo getVersionInfo() {
if (connectionHeader.hasVersionInfo()) {
return connectionHeader.getVersionInfo();
}
return null;
}
{noformat}
Shouldn't this just be {{return connectionHeader.getVersionInfo();}} since if
it doesn't have the version info we're going to be returning null anyway?
Looking at the other usages and unsynchronized instances... I don't think we
care? I think they happen to be synchronized some of the time because they're
in a big block, but otherwise they don't really matter.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594230#comment-16594230
]
Hadoop QA commented on HBASE-21098:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 1 fixed = 283 total (was 284) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 26s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
19s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
11s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21098 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937321/HBASE-21098.master.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux dd277fde 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-21110) Issues with Unsafe and JDK 11
[
https://issues.apache.org/jira/browse/HBASE-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594227#comment-16594227
]
Mike Drob commented on HBASE-21110:
---
bq. When is Unsafe supposed to go away? Or is it?
It is supposed to go away, but I don't know what the timeline for it is.
Starting with Java 9, users need to pass an explicit flag to access it, and I
don't know how hard it will be for large enterprises to get away with that when
there are security posture concerns.
bq. I believe VarHandles are the blessed replacement for Unsafe access, but
VarHandles cannot be naively added to a code base that has to still support
Java 7 and up, like Hadoop 2.x, HBase 1.x, others. We might be able to move
Unsafe access behind an interface with Java version specific implementations in
separate Maven modules
We already hide most of our Unsafe access behind a facade, maybe there is
something we can do to set it up inline.
> Issues with Unsafe and JDK 11
> -
>
> Key: HBASE-21110
> URL: https://issues.apache.org/jira/browse/HBASE-21110
> Project: HBase
> Issue Type: Task
>Reporter: Mike Drob
>Priority: Major
> Fix For: 3.0.0
>
>
> Using Java 11 RC 1, I get the following warning, probably need to add the
> suggested flag to our scripts?
> {noformat}
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ java -version
> java version "11" 2018-09-25
> Java(TM) SE Runtime Environment 18.9 (build 11+28)
> Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ bin/start-hbase.sh
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ cat
> /Users/mdrob/IdeaProjects/hbase/bin/../logs/hbase-mdrob-master-mdrob-MBP.local.out
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/mdrob/IdeaProjects/hbase/hbase-common/target/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21110) Issues with Unsafe and JDK 11
[
https://issues.apache.org/jira/browse/HBASE-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594219#comment-16594219
]
Andrew Purtell commented on HBASE-21110:
When is Unsafe supposed to go away? Or is it? I think Java wants to drop it,
but push back given extensive use of it is holding back the change via public
pressure. Not sure what is the latest status.
I believe VarHandles are the blessed replacement for Unsafe access, but
VarHandles cannot be naively added to a code base that has to still support
Java 7 and up, like Hadoop 2.x, HBase 1.x, others. We might be able to move
Unsafe access behind an interface with Java version specific implementations in
separate Maven modules, but somehow this abstraction cannot itself cause a loss
of performance. It might be ok, as long as whatever we do still results in
inlining of the Unsafe intrinsics on the older JVMs.
> Issues with Unsafe and JDK 11
> -
>
> Key: HBASE-21110
> URL: https://issues.apache.org/jira/browse/HBASE-21110
> Project: HBase
> Issue Type: Task
>Reporter: Mike Drob
>Priority: Major
> Fix For: 3.0.0
>
>
> Using Java 11 RC 1, I get the following warning, probably need to add the
> suggested flag to our scripts?
> {noformat}
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ java -version
> java version "11" 2018-09-25
> Java(TM) SE Runtime Environment 18.9 (build 11+28)
> Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ bin/start-hbase.sh
> mdrob@mdrob-MBP:~/IdeaProjects/hbase$ cat
> /Users/mdrob/IdeaProjects/hbase/bin/../logs/hbase-mdrob-master-mdrob-MBP.local.out
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/mdrob/IdeaProjects/hbase/hbase-common/target/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20719) HTable.batch() doesn't handle TableNotFound correctly.
[
https://issues.apache.org/jira/browse/HBASE-20719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594199#comment-16594199
]
Abhishek Goyal commented on HBASE-20719:
I'd like to try and fix this issue.
I've looked at the code for batch() and delete(), and I'm unable to find any
calls that could result in a TableNotFoundException. Could someone maybe point
me in the right direction? I'm just new to the codebase, and I apologize if
this is a stupid question.
> HTable.batch() doesn't handle TableNotFound correctly.
> --
>
> Key: HBASE-20719
> URL: https://issues.apache.org/jira/browse/HBASE-20719
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.1.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Minor
> Fix For: 2.2.0
>
>
> batch() as well as delete() are processing using AsyncRequest. To report
> about problems we are using RetriesExhaustedWithDetailsException and there is
> no special handling for TableNotFound exception. So, the final result for
> running batch or delete operations against not existing table looks really
> weird and missleading:
> {noformat}
> hbase(main):003:0> delete 't1', 'r1', 'c1'
> 2018-06-12 15:02:50,742 ERROR [main] client.AsyncRequestFutureImpl: Cannot
> get replica 0 location for
> {"totalColumns":1,"row":"r1","families":{"c1":[{"qualifier":"","vlen":0,"tag":[],"timestamp":9223372036854775807}]},"ts":9223372036854775807}
> ERROR: Failed 1 action: t1: 1 time, servers with issues: null
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594198#comment-16594198 ] Andrew Purtell commented on HBASE-20940: Thanks [~vishk]. Please provide the fixes as an addendum commit. By that what I mean is we need a patch for branch-1 with the new test fixes. Happy to wait another day rather than revert this from 1.4.7. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 2.0.2, 1.4.7 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20679) Add the ability to compile JSP dynamically in Jetty
[
https://issues.apache.org/jira/browse/HBASE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594185#comment-16594185
]
Hadoop QA commented on HBASE-20679:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 3m
44s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 5m
7s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 5m 7s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
22s{color} | {color:red} root: The patch generated 3 new + 43 unchanged - 0
fixed = 46 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
47s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
40s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
38s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}158m 23s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}204m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestDrainReplicationQueuesForStandBy |
| | hadoop.hbase.regionserver.TestEncryptionKeyRotation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20679 |
|
