[jira] [Comment Edited] (OMID-107) Replace HTableInterface with Table
[
https://issues.apache.org/jira/browse/OMID-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585016#comment-16585016
]
James Taylor edited comment on OMID-107 at 8/19/18 5:19 AM:
[~ohads] - would you mind reviewing this patch? I still get one failure, and
I'm not sure why.
{code:java}
Running TestSuite
Tests run: 91, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 606.813 sec
<<< FAILURE! - in TestSuite
testSIPreventsReadSkewUsingWritePredicate(org.apache.omid.transaction.TestBaillisAnomaliesWithTXs)
Time elapsed: 0.101 sec <<< FAILURE!
java.lang.AssertionError: Should be aborted
at
org.apache.omid.transaction.TestBaillisAnomaliesWithTXs.testSIPreventsReadSkewUsingWritePredicate(TestBaillisAnomaliesWithTXs.java:416)
{code}
I don't really understand the test. Why should the transaction fail?
was (Author: jamestaylor):
[~ohads] - would you mind reviewing this patch? I still get one failure, and
I'm not sure why.
{code:java}
Running TestSuite
Tests run: 91, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 606.813 sec
<<< FAILURE! - in TestSuite
testSIPreventsReadSkewUsingWritePredicate(org.apache.omid.transaction.TestBaillisAnomaliesWithTXs)
Time elapsed: 0.101 sec <<< FAILURE!
java.lang.AssertionError: Should be aborted
at
org.apache.omid.transaction.TestBaillisAnomaliesWithTXs.testSIPreventsReadSkewUsingWritePredicate(TestBaillisAnomaliesWithTXs.java:416)
{code}
> Replace HTableInterface with Table
> --
>
> Key: OMID-107
> URL: https://issues.apache.org/jira/browse/OMID-107
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: James Taylor
>Priority: Major
> Attachments: OMID-107.patch
>
>
> In HBase 2.0, the HTableInterface has been replaced with the Table interface.
> Thus, in Omid, this same replacement should take place. This will mostly be
> isolated to having TTable implement Table instead of HTableInterface. This
> will be required to check-in Omid support in Phoenix, since Phoenix 5.x is
> based on HBase 2.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OMID-107) Replace HTableInterface with Table
[
https://issues.apache.org/jira/browse/OMID-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585016#comment-16585016
]
James Taylor commented on OMID-107:
---
[~ohads] - would you mind reviewing this patch? I still get one failure, and
I'm not sure why.
{code:java}
Running TestSuite
Tests run: 91, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 606.813 sec
<<< FAILURE! - in TestSuite
testSIPreventsReadSkewUsingWritePredicate(org.apache.omid.transaction.TestBaillisAnomaliesWithTXs)
Time elapsed: 0.101 sec <<< FAILURE!
java.lang.AssertionError: Should be aborted
at
org.apache.omid.transaction.TestBaillisAnomaliesWithTXs.testSIPreventsReadSkewUsingWritePredicate(TestBaillisAnomaliesWithTXs.java:416)
{code}
> Replace HTableInterface with Table
> --
>
> Key: OMID-107
> URL: https://issues.apache.org/jira/browse/OMID-107
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: James Taylor
>Priority: Major
> Attachments: OMID-107.patch
>
>
> In HBase 2.0, the HTableInterface has been replaced with the Table interface.
> Thus, in Omid, this same replacement should take place. This will mostly be
> isolated to having TTable implement Table instead of HTableInterface. This
> will be required to check-in Omid support in Phoenix, since Phoenix 5.x is
> based on HBase 2.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OMID-107) Replace HTableInterface with Table
[ https://issues.apache.org/jira/browse/OMID-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated OMID-107: -- Attachment: OMID-107.patch > Replace HTableInterface with Table > -- > > Key: OMID-107 > URL: https://issues.apache.org/jira/browse/OMID-107 > Project: Apache Omid > Issue Type: Sub-task >Reporter: James Taylor >Priority: Major > Attachments: OMID-107.patch > > > In HBase 2.0, the HTableInterface has been replaced with the Table interface. > Thus, in Omid, this same replacement should take place. This will mostly be > isolated to having TTable implement Table instead of HTableInterface. This > will be required to check-in Omid support in Phoenix, since Phoenix 5.x is > based on HBase 2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OMID-102) Implement visibility filter as pure HBase Filter
[ https://issues.apache.org/jira/browse/OMID-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor resolved OMID-102. --- Resolution: Fixed > Implement visibility filter as pure HBase Filter > > > Key: OMID-102 > URL: https://issues.apache.org/jira/browse/OMID-102 > Project: Apache Omid > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Yonatan Gottesman >Priority: Major > Attachments: bug_fix.patch > > > The way Omid currently filters through it's own RegionScanner won't work the > way it's implemented (i.e. the way the filtering is done *after* the next > call). The reason is that the state of HBase filters get messed up since > these filters will start to see cells that it shouldn't (i.e. cells that > would be filtered based on snapshot isolation). It cannot be worked around by > manually running filters afterwards because filters may issue seek calls > which are handled during the running of scans by HBase. > > Instead, the filtering needs to be implemented as a pure HBase filter and > that filter needs to delegate to the other, delegate filter once it's > determined that the cell is visible. See Tephra's TransactionVisibilityFilter > and they way it calls the delegate filter (cellFilters) only after it's > determined that the cell is visible. You may run into TEPHRA-169 without > including the CellSkipFilter too. > Because it'll be easier if you see shadow cells *before* their corresponding > real cells you can prefix instead of suffix the column qualifiers to > guarantee that you'd see the shadow cells prior to the actual cells. Or you > could buffer cells in your filter prior to omitting them. Another issue would > be if the shadow cells aren't found and you need to consult the commit table > - I suppose if the shadow cells are first, this logic would be easier to know > when it needs to be called. > > To reproduce, see the Phoenix unit tests > FlappingTransactionIT.testInflightUpdateNotSeen() and > testInflightDeleteNotSeen(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-102) Implement visibility filter as pure HBase Filter
[ https://issues.apache.org/jira/browse/OMID-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584734#comment-16584734 ] Yonatan Gottesman commented on OMID-102: Yes, I uploaded the patch to here, I am not a commiter, so we will wait for [~ohads] to return, apply the patch to phoenix-integration branch, and see everything is ok. Meanwhile ill take a look at why omid fails to build under jenkins - OMID-109 > Implement visibility filter as pure HBase Filter > > > Key: OMID-102 > URL: https://issues.apache.org/jira/browse/OMID-102 > Project: Apache Omid > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Yonatan Gottesman >Priority: Major > Attachments: bug_fix.patch > > > The way Omid currently filters through it's own RegionScanner won't work the > way it's implemented (i.e. the way the filtering is done *after* the next > call). The reason is that the state of HBase filters get messed up since > these filters will start to see cells that it shouldn't (i.e. cells that > would be filtered based on snapshot isolation). It cannot be worked around by > manually running filters afterwards because filters may issue seek calls > which are handled during the running of scans by HBase. > > Instead, the filtering needs to be implemented as a pure HBase filter and > that filter needs to delegate to the other, delegate filter once it's > determined that the cell is visible. See Tephra's TransactionVisibilityFilter > and they way it calls the delegate filter (cellFilters) only after it's > determined that the cell is visible. You may run into TEPHRA-169 without > including the CellSkipFilter too. > Because it'll be easier if you see shadow cells *before* their corresponding > real cells you can prefix instead of suffix the column qualifiers to > guarantee that you'd see the shadow cells prior to the actual cells. Or you > could buffer cells in your filter prior to omitting them. Another issue would > be if the shadow cells aren't found and you need to consult the commit table > - I suppose if the shadow cells are first, this logic would be easier to know > when it needs to be called. > > To reproduce, see the Phoenix unit tests > FlappingTransactionIT.testInflightUpdateNotSeen() and > testInflightDeleteNotSeen(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OMID-102) Implement visibility filter as pure HBase Filter
[ https://issues.apache.org/jira/browse/OMID-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonatan Gottesman updated OMID-102: --- Attachment: bug_fix.patch > Implement visibility filter as pure HBase Filter > > > Key: OMID-102 > URL: https://issues.apache.org/jira/browse/OMID-102 > Project: Apache Omid > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Yonatan Gottesman >Priority: Major > Attachments: bug_fix.patch > > > The way Omid currently filters through it's own RegionScanner won't work the > way it's implemented (i.e. the way the filtering is done *after* the next > call). The reason is that the state of HBase filters get messed up since > these filters will start to see cells that it shouldn't (i.e. cells that > would be filtered based on snapshot isolation). It cannot be worked around by > manually running filters afterwards because filters may issue seek calls > which are handled during the running of scans by HBase. > > Instead, the filtering needs to be implemented as a pure HBase filter and > that filter needs to delegate to the other, delegate filter once it's > determined that the cell is visible. See Tephra's TransactionVisibilityFilter > and they way it calls the delegate filter (cellFilters) only after it's > determined that the cell is visible. You may run into TEPHRA-169 without > including the CellSkipFilter too. > Because it'll be easier if you see shadow cells *before* their corresponding > real cells you can prefix instead of suffix the column qualifiers to > guarantee that you'd see the shadow cells prior to the actual cells. Or you > could buffer cells in your filter prior to omitting them. Another issue would > be if the shadow cells aren't found and you need to consult the commit table > - I suppose if the shadow cells are first, this logic would be easier to know > when it needs to be called. > > To reproduce, see the Phoenix unit tests > FlappingTransactionIT.testInflightUpdateNotSeen() and > testInflightDeleteNotSeen(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
