[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065950#comment-17065950
]
Hudson commented on HBASE-23741:
Results for branch branch-2.3
[build #6 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/6//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Assignee: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0, 2.4.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record
> 3. Flush the table
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065248#comment-17065248
]
Hudson commented on HBASE-23741:
Results for branch branch-2
[build #2562 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2562//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Assignee: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0, 2.4.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record
> 3. Flush the table
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17064981#comment-17064981
]
Hudson commented on HBASE-23741:
Results for branch master
[build #1676 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1676/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Assignee: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0, 2.4.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record
> 3. Flush the table
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055592#comment-17055592 ] Guanghao Zhang commented on HBASE-23741: Ping [~pankajkumar][~stack][~zhangduo] for reviewing. > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Bug > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Assignee: Guanghao Zhang >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055482#comment-17055482 ] Michael Stack commented on HBASE-23741: --- Good find [~pankajkumar] (and [~sreenivasulureddy] ) > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Bug > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054977#comment-17054977 ] Sean Busbey commented on HBASE-23741: - it looks like the jira for making split-to-hfile the default wasn't done yet. I targeted it to 2.4.0. > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Bug > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054971#comment-17054971 ] Sean Busbey commented on HBASE-23741: - I converted this to an issue to raise visibility. fyi [~ndimiduk]. If split-to-hfile is still the default I agree that it should not be. > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Bug > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024247#comment-17024247 ] Anoop Sam John commented on HBASE-23741: Might be some thing to do with the HFiles flush seqNo or so? Did not see the code > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Sub-task > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Critical > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024246#comment-17024246 ] Anoop Sam John commented on HBASE-23741: We have a jira to make it default for 2.3.0. IMHO we should not do that in the 1st release itself. May be target it for 2.4.0 only? I mean the one which tries to make this way as the default recovery way. Good tests [~pankajkumar], [~sreenivasulureddy] > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Sub-task > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Critical > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024229#comment-17024229 ] Pankaj Kumar commented on HBASE-23741: -- That's okay [~zghao], we need to identify and fix the bugs as early as possible. > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Sub-task > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Critical > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled
[ https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024202#comment-17024202 ] Guanghao Zhang commented on HBASE-23741: Sorry, I didn't run ITBLL so there may some case not handle. Will make this feature stable before we release 2.3.0. Thanks for the test. > Data loss when WAL split to HFile enabled > - > > Key: HBASE-23741 > URL: https://issues.apache.org/jira/browse/HBASE-23741 > Project: HBase > Issue Type: Sub-task > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Pankaj Kumar >Priority: Critical > > Very simple steps as below, > 1. Create table with 1 region > 2. Insert 1 record > 3. Flush the table > 4. Scan table and observe timestamp of the inserted row > 5. Insert same row key with same timestamp as previously inserted but with > different value > 6. Kill -9 RS where table region is online > 7. Start RS > Scan the table and check the result, latest cell must be returned. > Thanks [~sreenivasulureddy] for finding this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
