[
https://issues.apache.org/jira/browse/PHOENIX-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576505#comment-16576505
]
James Taylor commented on PHOENIX-4841:
---------------------------------------
Agreed - we should be able to push this to the start/stop of the scan. The
basic flow is:
* WhereCompiler produces an Expression tree that will correctly evaluate the
expressions (taking into account sort order). This is before any attempt is
made to push expressions into the scan and would be basically "if a full table
scan was performed, the expressions should evaluate correctly"
* WhereOptimizer works off of these Expressions and attempts to set as much of
the start/stop row of the scan as possible. It needs to "undo" any changes to
the comparison operator or inversion of the ranges first. Then the logic
throughout KeyExpressionVisitor and andKeySlots works based on non inverted
ranges, and at the very end in pushKeyExpressionsToScan, we take into account
the sort order by inverting the ranges if necessary
I'm not sure exactly which part of the code you're referring to, but if it's in
WhereOptimizer, then we should likely either ignore sort order differences, or
we shouldn't have any because we've transformed the sort order to ASC (until
the end).
A good test is that the evaluation of the expression generated by the
WhereCompiler for a set of rows should match the evaluation when the start/stop
row is pushed into the scan (i.e. you should get the same results both if the
columns are PK columns as well as if they are KeyValue columns). That wasn't
the case before the fix for PHOENIX-3383.
> Filters that uses RVC with pk columns where with DESC sort order don't work
> correctly
> -------------------------------------------------------------------------------------
>
> Key: PHOENIX-4841
> URL: https://issues.apache.org/jira/browse/PHOENIX-4841
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.15.0, 5.1.0
> Reporter: Thomas D'Silva
> Priority: Major
>
> If we filter on pk columns where one of the columns is DESC, we don't get the
> expected results. If the PK columns are of sorted by ASC we get the correct
> results. For eg. the following test fails:
> {code}
> @Test
> public void testRVCWithDescAndAscPK() throws Exception {
> String fullTableName = generateUniqueName();
> // create base table and global view using global connection
> try (Connection conn = DriverManager.getConnection(getUrl())) {
> Statement stmt = conn.createStatement();
> stmt.execute("CREATE TABLE " + fullTableName + "(\n" +
> " A VARCHAR NOT NULL,\n" +
> " B VARCHAR NOT NULL,\n" +
> " C VARCHAR NOT NULL,\n" +
> " CONSTRAINT PK PRIMARY KEY (A, B DESC, C))");
>
> conn.createStatement().execute("UPSERT INTO " + fullTableName + "
> VALUES ('x', 'd', '1')");
> conn.createStatement().execute("UPSERT INTO " + fullTableName + "
> VALUES ('x', 'c', '2')");
> conn.createStatement().execute("UPSERT INTO " + fullTableName + "
> VALUES ('x', 'b', '3')");
> conn.createStatement().execute("UPSERT INTO " + fullTableName + "
> VALUES ('x', 'b', '4')");
> conn.createStatement().execute("UPSERT INTO " + fullTableName + "
> VALUES ('x', 'a', '4')");
> conn.commit();
> }
> // validate that running query using global view gives same results
> try (Connection conn = DriverManager.getConnection(getUrl())) {
> ResultSet rs =
> conn.createStatement().executeQuery(
> "SELECT B, C FROM " + fullTableName + " WHERE (B, C)
> > ('b', '3')");
> assertTrue(rs.next());
> assertEquals("d", rs.getString(1));
> assertEquals("1", rs.getString(2));
> assertTrue(rs.next());
> assertEquals("c", rs.getString(1));
> assertEquals("2", rs.getString(2));
> assertTrue(rs.next());
> assertEquals("b", rs.getString(1));
> assertEquals("4", rs.getString(2));
> assertFalse(rs.next());
> }
> }
> {code}
> The comparison expression for the above query is
> {code}
> (PK[-1], PK[-1]) > (TO_VARCHAR('b'), '3')
> {code}
> When the first row is evaluated the lhs bytes is:
> {code}
> [-101, -1, 49]
> {code}
> and rhs bytes:
> {code}
> [-99, -1, 51]
> {code}
> We invert the bytes of the B column but since the greater than comparison
> operator usedthe row is filtered out (even though it should be returned).
> [~jamestaylor]
> When a column is DESC order do we need to rewrite the comparison expression?
> Instead of
> {code}
> WHERE (B, C) > ('b', '3')
> {code}
> we need something like
> {code}
> WHERE B<~'b' OR (B=~'b' AND C>'3')
> {code}
> Is there a better way to handle this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
