[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526826#comment-15526826 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-0.98-on-Hadoop-1.1 #1277 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1277/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 1bdf05b1fa1c3d09b9e3c4fb43957c5ff32ac701) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526031#comment-15526031 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-0.98-matrix #404 (See [https://builds.apache.org/job/HBase-0.98-matrix/404/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 1bdf05b1fa1c3d09b9e3c4fb43957c5ff32ac701) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524882#comment-15524882 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.4 #432 (See [https://builds.apache.org/job/HBase-1.4/432/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 4566e4df58bdd176228aab2cd3cfd80dd983072f) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524804#comment-15524804 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #24 (See [https://builds.apache.org/job/HBase-1.3-JDK8/24/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 1441b7c795292ce5b056ff9e3b5b2443ecd8e8cb) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524799#comment-15524799 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.2-JDK8 #30 (See [https://builds.apache.org/job/HBase-1.2-JDK8/30/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 2733e24d3f2f110ac98d8876ee1de1fd9740b51e) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524798#comment-15524798 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.2-JDK7 #33 (See [https://builds.apache.org/job/HBase-1.2-JDK7/33/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 2733e24d3f2f110ac98d8876ee1de1fd9740b51e) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524685#comment-15524685 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #23 (See [https://builds.apache.org/job/HBase-1.3-JDK7/23/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 1441b7c795292ce5b056ff9e3b5b2443ecd8e8cb) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524616#comment-15524616 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.1-JDK8 #1871 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1871/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 88512be52b8707fb87ab2c5979fd71664a417a90) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524575#comment-15524575 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-1.1-JDK7 #1787 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1787/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev 88512be52b8707fb87ab2c5979fd71664a417a90) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524530#comment-15524530 ] Hudson commented on HBASE-16649: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1679 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1679/]) HBASE-16649 Truncate table with splits preserved can cause both data (matteo.bertozzi: rev f06c0060aa13a2b5b18edeb66b7479bdd3c6fdc8) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523740#comment-15523740 ] Stephen Yuan Jiang commented on HBASE-16649: +1. V2 looks good to me. (The only thing I am unsure is that whether we catch all cases here. For truncate table, the patch would fix the corruption.) I think this change should go to all branches (include non-proc-V2 based branch such as 0.98 and 1.0). > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch, > HBASE-16649-v2.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[
https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517768#comment-15517768
]
Hadoop QA commented on HBASE-16649:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 45s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 115m 54s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 177m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionServerMetrics |
| Timed out junit tests |
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.client.TestMetaWithReplicas |
| | org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient |
| | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.0 Server=1.12.0 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12830086/HBASE-16649-v2.patch |
| JIRA Issue | HBASE-16649 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 19f0c4fcffcc 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4082424 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3695/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3695/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3695/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505639#comment-15505639 ] Allan Yang commented on HBASE-16649: I mean add the line 'flushedSequenceIdByRegion.put(encodedRegionName, l);' In trunk, when smaller seq id is reported, it will be ignored. My point is that is no harm to update the seqid even if it is smaller. Otherwise, if this situation do happen, data loss it is a sure thing. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Attachments: HBASE-16649-v0.patch, HBASE-16649-v1.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[
https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505548#comment-15505548
]
Hadoop QA commented on HBASE-16649:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 42s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 123m 9s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.TestZooKeeper |
| |
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster |
| | org.apache.hadoop.hbase.master.TestRollingRestart |
| | org.apache.hadoop.hbase.master.TestDistributedLogSplitting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12829316/HBASE-16649-v1.patch |
| JIRA Issue | HBASE-16649 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 24fe8c64ed9b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b2eac0d |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3614/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3614/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3614/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3614/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.or
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505298#comment-15505298 ] Duo Zhang commented on HBASE-16649: --- Theoretically this should not happen? I mean a smaller flush sequence id. And remove this can not solve all the problems as you may do log replay before the new sequence id reported. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[
https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505293#comment-15505293
]
Duo Zhang commented on HBASE-16649:
---
{code}
private static List recreteRegionInfo(final List
regions) {
{code}
Typo? recreate?
> Truncate table with splits preserved can cause both data loss and truncated
> data appeared again
> ---
>
> Key: HBASE-16649
> URL: https://issues.apache.org/jira/browse/HBASE-16649
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.3
>Reporter: Allan Yang
>Assignee: Matteo Bertozzi
> Attachments: HBASE-16649-v0.patch
>
>
> Since truncate table with splits preserved will delete hfiles and use the
> previous regioninfo. It can cause odd behaviors
> - Case 1: *Data appeared after truncate*
> reproduce procedure：
> 1. create a table, let's say 'test'
> 2. write data to 'test', make sure memstore of 'test' is not empty
> 3. truncate 'test' with splits preserved
> 4. kill the regionserver hosting the region(s) of 'test'
> 5. start the regionserver, now it is the time to witness the miracle! the
> truncated data appeared in table 'test'
> - Case 2: *Data loss*
> reproduce procedure:
> 1. create a table, let's say 'test'
> 2. write some data to 'test', no matter how many
> 3. truncate 'test' with splits preserved
> 4. restart the regionserver to reset the seqid
> 5. write some data, but less than 2 since we don't want the seqid to run over
> the one in 2
> 6. kill the regionserver hosting the region(s) of 'test'
> 7. restart the regionserver. Congratulations! the data writen in 4 is now all
> lost
> *Why?*
> for case 1
> Since preserve splits in truncate table procedure will not change the
> regioninfo, when log replay happens, the 'unflushed' data will restore back
> to the region
> for case 2
> since the flushedSequenceIdByRegion are stored in Master in a map with the
> region's encodedName. Although the table is truncated, the region's name is
> not changed since we chose to preserve the splits. So after truncate the
> table, the region's sequenceid is reset in the regionserver, but not reset in
> master. When flush comes and report to master, master will reject the update
> of sequenceid since the new one is smaller than the old one. The same happens
> in log replay, all the edits writen in 4 will be skipped since they have a
> smaller seqid
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505231#comment-15505231 ] Enis Soztutar commented on HBASE-16649: --- Agreed that it is the safest approach to re-create the regions with different encoded names. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang >Assignee: Matteo Bertozzi > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[
https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505189#comment-15505189
]
Hadoop QA commented on HBASE-16649:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 5s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 7s {color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestCatalogJanitor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12829117/HBASE-16649-v0.patch |
| JIRA Issue | HBASE-16649 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 51e5ae5b0817 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b2eac0d |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3610/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3610/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3610/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3610/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Truncate table with splits preserved can cause both data loss and truncated
> data appeared again
>
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503964#comment-15503964 ] Matteo Bertozzi commented on HBASE-16649: - let's open another jira to discuss this. since it seems a bit more involved and it is not related directly to DDLs. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503717#comment-15503717 ] Matteo Bertozzi commented on HBASE-16649: - table delete, table truncate, catalog janitor when removing merged region and split parent > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[
https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502490#comment-15502490
]
Allan Yang commented on HBASE-16649:
Another thing is that we should change the seqid in master even if the new one
is smaller than the old one, better have some redundant data when replay than
lose them, right?
{code:java}
if (existingValue == null || (l != HConstants.NO_SEQNUM && l > existingValue)) {
flushedSequenceIdByRegion.put(encodedRegionName, l);
} else if (l != HConstants.NO_SEQNUM && l < existingValue) {
flushedSequenceIdByRegion.put(encodedRegionName, l); //update the
seqid but log a warning
LOG.warn("RegionServer " + sn + " indicates a last flushed sequence id
("
+ l + ") that is less than the previous last flushed sequence id ("
+ existingValue + ") for region " + Bytes.toString(entry.getKey())
+ " Ignoring.");
}
{code}
> Truncate table with splits preserved can cause both data loss and truncated
> data appeared again
> ---
>
> Key: HBASE-16649
> URL: https://issues.apache.org/jira/browse/HBASE-16649
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.3
>Reporter: Allan Yang
> Attachments: HBASE-16649-v0.patch
>
>
> Since truncate table with splits preserved will delete hfiles and use the
> previous regioninfo. It can cause odd behaviors
> - Case 1: *Data appeared after truncate*
> reproduce procedure：
> 1. create a table, let's say 'test'
> 2. write data to 'test', make sure memstore of 'test' is not empty
> 3. truncate 'test' with splits preserved
> 4. kill the regionserver hosting the region(s) of 'test'
> 5. start the regionserver, now it is the time to witness the miracle! the
> truncated data appeared in table 'test'
> - Case 2: *Data loss*
> reproduce procedure:
> 1. create a table, let's say 'test'
> 2. write some data to 'test', no matter how many
> 3. truncate 'test' with splits preserved
> 4. restart the regionserver to reset the seqid
> 5. write some data, but less than 2 since we don't want the seqid to run over
> the one in 2
> 6. kill the regionserver hosting the region(s) of 'test'
> 7. restart the regionserver. Congratulations! the data writen in 4 is now all
> lost
> *Why?*
> for case 1
> Since preserve splits in truncate table procedure will not change the
> regioninfo, when log replay happens, the 'unflushed' data will restore back
> to the region
> for case 2
> since the flushedSequenceIdByRegion are stored in Master in a map with the
> region's encodedName. Although the table is truncated, the region's name is
> not changed since we chose to preserve the splits. So after truncate the
> table, the region's sequenceid is reset in the regionserver, but not reset in
> master. When flush comes and report to master, master will reject the update
> of sequenceid since the new one is smaller than the old one. The same happens
> in log replay, all the edits writen in 4 will be skipped since they have a
> smaller seqid
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502238#comment-15502238 ] Ted Yu commented on HBASE-16649: When is the obsolete region taken off storeFlushedSequenceIdsByRegion ? > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502168#comment-15502168 ] Matteo Bertozzi commented on HBASE-16649: - what do you mean? I'm not sure to understand the above. In the patch above, we are removing the deleted regions from ServerManager. in HMaster we have at least two places I know of where we have region states. The AM and the ServerManager. the attached patch, adds the removal of the ServerManager stuff when delete/truncate table is called and when the CatalogJanitor is removing the region from split/merge (those should be the only places where we remove regions). and these calls happen on the HMaster. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502157#comment-15502157 ] Ted Yu commented on HBASE-16649: Shouldn't data structures involving the obsolete region info be cleaned in master ? > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501944#comment-15501944 ] Allan Yang commented on HBASE-16649: Your patch is exactly what I want to do! > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > Attachments: HBASE-16649-v0.patch > > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501315#comment-15501315 ] Matteo Bertozzi commented on HBASE-16649: - current implementation we drop the fs dirs, and recreate them. we can just regenerate the regionInfo, so they will get a new encodedName > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501245#comment-15501245 ] Ted Yu commented on HBASE-16649: That would be slower compared to the current implementation (with bugs fixed) since layout on hdfs would be touched. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501192#comment-15501192 ] Allan Yang commented on HBASE-16649: I have another thought, can we change the region's encoded name even if we want to preserve the splits? Creating whole new regions after truncating may solve the two cases, I think. > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. restart the regionserver to reset the seqid > 5. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 6. kill the regionserver hosting the region(s) of 'test' > 7. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16649) Truncate table with splits preserved can cause both data loss and truncated data appeared again
[ https://issues.apache.org/jira/browse/HBASE-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501182#comment-15501182 ] Ted Yu commented on HBASE-16649: For #2, the region's sequenceid's should be reset in master. Do you want to attach patch ? Thanks > Truncate table with splits preserved can cause both data loss and truncated > data appeared again > --- > > Key: HBASE-16649 > URL: https://issues.apache.org/jira/browse/HBASE-16649 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Allan Yang > > Since truncate table with splits preserved will delete hfiles and use the > previous regioninfo. It can cause odd behaviors > - Case 1: *Data appeared after truncate* > reproduce procedure： > 1. create a table, let's say 'test' > 2. write data to 'test', make sure memstore of 'test' is not empty > 3. truncate 'test' with splits preserved > 4. kill the regionserver hosting the region(s) of 'test' > 5. start the regionserver, now it is the time to witness the miracle! the > truncated data appeared in table 'test' > - Case 2: *Data loss* > reproduce procedure: > 1. create a table, let's say 'test' > 2. write some data to 'test', no matter how many > 3. truncate 'test' with splits preserved > 4. write some data, but less than 2 since we don't want the seqid to run over > the one in 2 > 5. kill the regionserver hosting the region(s) of 'test' > 6. restart the regionserver. Congratulations! the data writen in 4 is now all > lost > *Why?* > for case 1 > Since preserve splits in truncate table procedure will not change the > regioninfo, when log replay happens, the 'unflushed' data will restore back > to the region > for case 2 > since the flushedSequenceIdByRegion are stored in Master in a map with the > region's encodedName. Although the table is truncated, the region's name is > not changed since we chose to preserve the splits. So after truncate the > table, the region's sequenceid is reset in the regionserver, but not reset in > master. When flush comes and report to master, master will reject the update > of sequenceid since the new one is smaller than the old one. The same happens > in log replay, all the edits writen in 4 will be skipped since they have a > smaller seqid -- This message was sent by Atlassian JIRA (v6.3.4#6332)
