[ https://issues.apache.org/jira/browse/OMID-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565061#comment-16565061 ]
ASF GitHub Bot commented on OMID-102: ------------------------------------- GitHub user yonigottesman reopened a pull request: https://github.com/apache/incubator-omid/pull/41 [OMID-102] Support for user Filter when using coprocessor for snapshot filtering You can merge this pull request into a Git repository by running: $ git pull https://github.com/yonigottesman/incubator-omid scan_filters Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-omid/pull/41.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #41 ---- commit 41554ec4c3cdfdbfd271e9adbef866b3653c3382 Author: Yonatan Gottesman <yonigo10@...> Date: 2018-07-25T06:40:14Z Support for user Filter when using coprocessor for snapshot filtering commit ba575384dcdca427bf80e55dcee58a1ed93bf744 Author: Yonatan Gottesman <yonigo10@...> Date: 2018-07-25T07:41:42Z In coprocessor filtering, get shadow cell of delete family before going to commit table commit 26469a0893b4c9bc36e68d66583951fb6e82cf9d Author: Yonatan Gottesman <yonigo10@...> Date: 2018-07-26T08:15:27Z add inMemoryCommitTable client option in omid coprocessor for testing commit 8d632670ab580ca04e0c5cec555211d1fa71a11a Author: Yonatan Gottesman <yonigo10@...> Date: 2018-07-31T11:29:13Z Fix visibilityFilter to check if delete family is in current TX commit b2980c7cea2ef9c83a772d4ce3bd9a82f5ae4d81 Author: Yonatan Gottesman <yonigo10@...> Date: 2018-08-01T09:34:00Z Merge OMID-105 changes ---- > Implement visibility filter as pure HBase Filter > ------------------------------------------------ > > Key: OMID-102 > URL: https://issues.apache.org/jira/browse/OMID-102 > Project: Apache Omid > Issue Type: Sub-task > Reporter: James Taylor > Assignee: Yonatan Gottesman > Priority: Major > > The way Omid currently filters through it's own RegionScanner won't work the > way it's implemented (i.e. the way the filtering is done *after* the next > call). The reason is that the state of HBase filters get messed up since > these filters will start to see cells that it shouldn't (i.e. cells that > would be filtered based on snapshot isolation). It cannot be worked around by > manually running filters afterwards because filters may issue seek calls > which are handled during the running of scans by HBase. > > Instead, the filtering needs to be implemented as a pure HBase filter and > that filter needs to delegate to the other, delegate filter once it's > determined that the cell is visible. See Tephra's TransactionVisibilityFilter > and they way it calls the delegate filter (cellFilters) only after it's > determined that the cell is visible. You may run into TEPHRA-169 without > including the CellSkipFilter too. > Because it'll be easier if you see shadow cells *before* their corresponding > real cells you can prefix instead of suffix the column qualifiers to > guarantee that you'd see the shadow cells prior to the actual cells. Or you > could buffer cells in your filter prior to omitting them. Another issue would > be if the shadow cells aren't found and you need to consult the commit table > - I suppose if the shadow cells are first, this logic would be easier to know > when it needs to be called. > > To reproduce, see the Phoenix unit tests > FlappingTransactionIT.testInflightUpdateNotSeen() and > testInflightDeleteNotSeen(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)