[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3443:
-
Fix Version/s: 0.95.0
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Assignee: Lars Hofhansl
>Priority: Critical
> Labels: corruption
> Fix For: 0.94.0, 0.95.0
>
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3443:
-
Fix Version/s: (was: 0.95.0)
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Assignee: Lars Hofhansl
>Priority: Critical
> Labels: corruption
> Fix For: 0.94.0
>
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-3443:
-
Fix Version/s: 0.94.0
a'right, committed to 0.94 aswell, and added release notes.
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Assignee: Lars Hofhansl
>Priority: Critical
> Labels: corruption
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-3443:
-
Release Note:
This is a correctness fix and will incur a 10-20% performance penalty for ICV
and Increment operations. Other operations are not affected.
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Assignee: Lars Hofhansl
>Priority: Critical
> Labels: corruption
> Fix For: 0.96.0
>
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-3443:
-
Fix Version/s: 0.96.0
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Assignee: Lars Hofhansl
>Priority: Critical
> Labels: corruption
> Fix For: 0.96.0
>
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-3443:
-
Attachment: 3443.txt
Here's a patch. Quite trivial, just removing some code and using the existing
get without invoking coprocessors
BUT... To test the performance I jacked up the number of increments per thread
in TestAtomicOperation.testIncrementMultiThreads to 1 and ran it with and
without this change.
On my machine I find that the test runs around 23-24s with the existing code
and around 27-28s with this patch.
Not sure that is a hit we want take.
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Priority: Critical
> Labels: corruption
> Attachments: 3443.txt
>
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3443:
-
Priority: Critical (was: Major)
Marking critical
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
>Priority: Critical
> Labels: corruption
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
[
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benoit Sigoure updated HBASE-3443:
--
Component/s: regionserver
Affects Version/s: 0.90.0
0.90.1
0.90.2
0.90.3
0.90.4
0.90.5
0.90.6
0.92.0
0.92.1
Labels: corruption (was: )
> ICV optimization to look in memstore first and then store files (HBASE-3082)
> does not work when deletes are in the mix
> --
>
> Key: HBASE-3443
> URL: https://issues.apache.org/jira/browse/HBASE-3443
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6,
> 0.92.0, 0.92.1
>Reporter: Kannan Muthukkaruppan
> Labels: corruption
>
> For incrementColumnValue() HBASE-3082 adds an optimization to check memstores
> first, and only if not present in the memstore then check the store files. In
> the presence of deletes, the above optimization is not reliable.
> If the column is marked as deleted in the memstore, one should not look
> further into the store files. But currently, the code does so.
> Sample test code outline:
> {code}
> admin.createTable(desc)
> table = HTable.new(conf, tableName)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> admin.flush(tableName)
> sleep(2)
> del = Delete.new(Bytes.toBytes("row"))
> table.delete(del)
> table.incrementColumnValue(Bytes.toBytes("row"), cf1name,
> Bytes.toBytes("column"), 5);
> get = Get.new(Bytes.toBytes("row"))
> keyValues = table.get(get).raw()
> keyValues.each do |keyValue|
> puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
> end
> {code}
> The above prints:
> {code}
> Expect 5; Got Value=10
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
