[jira] [Commented] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[ https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125294#comment-16125294 ] Chia-Ping Tsai commented on HBASE-18471: Ping for reviews. This bug makes our hbase lose data. We should resolve it before next release. > Deleted qualifier re-appearing after multiple puts. > --- > > Key: HBASE-18471 > URL: https://issues.apache.org/jira/browse/HBASE-18471 > Project: HBase > Issue Type: Bug > Components: Deletes, hbase, scan >Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1 >Reporter: Thomas Martens >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18471.v0.patch, HBaseDmlTest.java > > > The qualifier of a deleted row (with keep deleted cells true) re-appears > after re-inserting the same row multiple times (with different timestamp) > with an empty qualifier. > Scenario: > # Put row with family and qualifier (timestamp 1). > # Delete entire row (timestamp 2). > # Put same row again with family without qualifier (timestamp 3). > A scan (latest version) returns the row with family without qualifier, > version 3 (which is correct). > # Put the same row again with family without qualifier (timestamp 4). > A scan (latest version) returns multiple rows: > * the row with family without qualifier, version 4 (which is correct). > * the row with family with qualifier, version 1 (which is wrong). > There is a test scenario attached. > output: > 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml > 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml > 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml > 13:42:58,592 [main] client.HBaseAdmin - Created test_dml > Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with > timestamp: '1' > Scan printout => > Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier', > Value: 'myValue' > Delete row: 'myRow' > Scan printout => > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '3' > Scan printout => > Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '4' > Scan printout => > Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: > 'myQualifier', Value: 'myValue'{color} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[ https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124705#comment-16124705 ] Hadoop QA commented on HBASE-18471: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 21s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 27s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 30m 16s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 14s{color} | {color:green} hbase-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green}114m 32s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 28s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}165m 42s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 | | JIRA Issue | HBASE-18471 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881625/HBASE-18471.v0.patch | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux eeac63df9221 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 173dce7 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC3 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/8058/testReport/ | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/8058/console | | Powered by | Apache Yetus 0.4.0 http://yetus.apache.org | This message was automatically generated. > Deleted qualifier re-appearing after multiple
[jira] [Commented] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[ https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110871#comment-16110871 ] Chia-Ping Tsai commented on HBASE-18471: We add a cell without qualifier for deleting all columns of the specified family. {code} public Delete addFamily(final byte [] family, final long timestamp) { if (timestamp < 0) { throw new IllegalArgumentException("Timestamp cannot be negative. ts=" + timestamp); } List list = familyMap.get(family); if(list == null) { list = new ArrayList<>(1); } else if(!list.isEmpty()) { list.clear(); } KeyValue kv = new KeyValue(row, family, null, timestamp, KeyValue.Type.DeleteFamily); list.add(kv); familyMap.put(family, list); return this; } {code} The *hint cell* created by CellUtil.createLastOnRowCol force the KVScanner to skip the remaining cells which have the same family/qualifier with *hint cell* {code} public Cell getKeyForNextColumn(Cell cell) { ColumnCount nextColumn = columns.getColumnHint(); if (nextColumn == null) { return CellUtil.createLastOnRowCol(cell); } else { return CellUtil.createFirstOnRowCol(cell, nextColumn.getBuffer(), nextColumn.getOffset(), nextColumn.getLength()); } } {code} > Deleted qualifier re-appearing after multiple puts. > --- > > Key: HBASE-18471 > URL: https://issues.apache.org/jira/browse/HBASE-18471 > Project: HBase > Issue Type: Bug > Components: Deletes, hbase, scan >Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1 >Reporter: Thomas Martens >Priority: Critical > Attachments: HBaseDmlTest.java > > > The qualifier of a deleted row (with keep deleted cells true) re-appears > after re-inserting the same row multiple times (with different timestamp) > with an empty qualifier. > Scenario: > # Put row with family and qualifier (timestamp 1). > # Delete entire row (timestamp 2). > # Put same row again with family without qualifier (timestamp 3). > A scan (latest version) returns the row with family without qualifier, > version 3 (which is correct). > # Put the same row again with family without qualifier (timestamp 4). > A scan (latest version) returns multiple rows: > * the row with family without qualifier, version 4 (which is correct). > * the row with family with qualifier, version 1 (which is wrong). > There is a test scenario attached. > output: > 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml > 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml > 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml > 13:42:58,592 [main] client.HBaseAdmin - Created test_dml > Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with > timestamp: '1' > Scan printout => > Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier', > Value: 'myValue' > Delete row: 'myRow' > Scan printout => > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '3' > Scan printout => > Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '4' > Scan printout => > Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: > 'myQualifier', Value: 'myValue'{color} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[ https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108238#comment-16108238 ] Biju Nair commented on HBASE-18471: --- Another difference in {{scanner}} behavior, after the test code is run ``` hbase(main):023:0> scan 'test_dml' ROWCOLUMN+CELL myRow column=myFamily:, timestamp=4, value=myValue myRow column=myFamily:myQualifier, timestamp=1, value=myValue 1 row(s) in 0.0330 seconds hbase(main):024:0> scan 'test_dml', {VERSIONS => 10} ROWCOLUMN+CELL myRow column=myFamily:, timestamp=4, value=myValue myRow column=myFamily:, timestamp=3, value=myValue 1 row(s) in 0.0290 seconds ``` > Deleted qualifier re-appearing after multiple puts. > --- > > Key: HBASE-18471 > URL: https://issues.apache.org/jira/browse/HBASE-18471 > Project: HBase > Issue Type: Bug > Components: Deletes, hbase, scan >Affects Versions: 1.3.0, 1.3.1 >Reporter: Thomas Martens > Attachments: HBaseDmlTest.java > > > The qualifier of a deleted row (with keep deleted cells true) re-appears > after re-inserting the same row multiple times (with different timestamp) > with an empty qualifier. > Scenario: > # Put row with family and qualifier (timestamp 1). > # Delete entire row (timestamp 2). > # Put same row again with family without qualifier (timestamp 3). > A scan (latest version) returns the row with family without qualifier, > version 3 (which is correct). > # Put the same row again with family without qualifier (timestamp 4). > A scan (latest version) returns multiple rows: > * the row with family without qualifier, version 4 (which is correct). > * the row with family with qualifier, version 1 (which is wrong). > There is a test scenario attached. > output: > 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml > 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml > 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml > 13:42:58,592 [main] client.HBaseAdmin - Created test_dml > Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with > timestamp: '1' > Scan printout => > Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier', > Value: 'myValue' > Delete row: 'myRow' > Scan printout => > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '3' > Scan printout => > Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with > timestamp: '4' > Scan printout => > Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value: > 'myValue' > {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: > 'myQualifier', Value: 'myValue'{color} -- This message was sent by Atlassian JIRA (v6.4.14#64029)