[jira] [Commented] (CALCITE-3600) Rule to solve the filter partially by end application and remaining by calcite
[
https://issues.apache.org/jira/browse/CALCITE-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995481#comment-16995481
]
Stamatis Zampetakis commented on CALCITE-3600:
--
Another API that seems relevant for this use-case is
[FilterableTable|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/schema/FilterableTable.java].
> Rule to solve the filter partially by end application and remaining by calcite
> --
>
> Key: CALCITE-3600
> URL: https://issues.apache.org/jira/browse/CALCITE-3600
> Project: Calcite
> Issue Type: Wish
>Reporter: anjali shrishrimal
>Priority: Minor
>
> Add Rule to check if filter condition is solvable by end application. If part
> of the filter condition can be solved by end application, then it should get
> pushed to end application, and remaining part which can not be solved by end
> application, should get solved by calcite secondarily (i.e. upon fetch remove
> unwanted data as per filter condition)
>
> Consider an application which can solve only limited operators while
> filtering, say "=,<,>" and can not solve operator 'LIKE'.
>
> Example, filter condition is "id > 1000 AND name LIKE '%an%'"
>
> we would like to restrict the condition passed to application to "id > 1000"
> and remaining part "name LIKE '%an%'" should get solved by calcite. (The way
> it does for csv-adapter)
>
> To replicate the situation, consider test-case testFilter in MongoAdapterTest
> (org.apache.calcite.adapter.mongodb.MongoAdapterTest) of mongo-adapter.
> And modify it like below:
>
> @Test public void testFilter()
> { assertModel(MODEL) .query("select state, city from zips where state = 'CA'
> AND city LIKE '%E%'") .returnsUnordered("STATE=CA; CITY=LOS ANGELES",
> "STATE=CA; CITY=BELL GARDENS"); }
>
>
> Expected output of above query :
> STATE=CA; CITY=LOS ANGELES,
> STATE=CA; CITY=BELL GARDENS
>
> Expected plan :
> EnumerableFilter(condition=[LIKE(CAST(ITEM($0, 'city')):VARCHAR(20), '%E%')])
> {{MongoToEnumerableConverter}}
> {{MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2)],
> CITY=[CAST(ITEM($0, 'city')):VARCHAR(20)])}}
> {{MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2), 'CA')])}}
> {{MongoTableScan(table=[[mongo_raw, zips]])}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3600) Rule to solve the filter partially by end application and remaining by calcite
[
https://issues.apache.org/jira/browse/CALCITE-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995478#comment-16995478
]
Stamatis Zampetakis commented on CALCITE-3600:
--
Calcite already offers a high level API to achieve this by sub-classing the
[CalcRelSplitter|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/CalcRelSplitter.java].
Have you checked this class?
Another issue that I think is relevant is CALCITE-3541.
> Rule to solve the filter partially by end application and remaining by calcite
> --
>
> Key: CALCITE-3600
> URL: https://issues.apache.org/jira/browse/CALCITE-3600
> Project: Calcite
> Issue Type: Wish
>Reporter: anjali shrishrimal
>Priority: Minor
>
> Add Rule to check if filter condition is solvable by end application. If part
> of the filter condition can be solved by end application, then it should get
> pushed to end application, and remaining part which can not be solved by end
> application, should get solved by calcite secondarily (i.e. upon fetch remove
> unwanted data as per filter condition)
>
> Consider an application which can solve only limited operators while
> filtering, say "=,<,>" and can not solve operator 'LIKE'.
>
> Example, filter condition is "id > 1000 AND name LIKE '%an%'"
>
> we would like to restrict the condition passed to application to "id > 1000"
> and remaining part "name LIKE '%an%'" should get solved by calcite. (The way
> it does for csv-adapter)
>
> To replicate the situation, consider test-case testFilter in MongoAdapterTest
> (org.apache.calcite.adapter.mongodb.MongoAdapterTest) of mongo-adapter.
> And modify it like below:
>
> @Test public void testFilter()
> { assertModel(MODEL) .query("select state, city from zips where state = 'CA'
> AND city LIKE '%E%'") .returnsUnordered("STATE=CA; CITY=LOS ANGELES",
> "STATE=CA; CITY=BELL GARDENS"); }
>
>
> Expected output of above query :
> STATE=CA; CITY=LOS ANGELES,
> STATE=CA; CITY=BELL GARDENS
>
> Expected plan :
> EnumerableFilter(condition=[LIKE(CAST(ITEM($0, 'city')):VARCHAR(20), '%E%')])
> {{MongoToEnumerableConverter}}
> {{MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2)],
> CITY=[CAST(ITEM($0, 'city')):VARCHAR(20)])}}
> {{MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2), 'CA')])}}
> {{MongoTableScan(table=[[mongo_raw, zips]])}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3600) Rule to solve the filter partially by end application and remaining by calcite
[
https://issues.apache.org/jira/browse/CALCITE-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995451#comment-16995451
]
Rui Wang commented on CALCITE-3600:
---
I think in BeamSQL we had same problem, due to the reason that underlying
source read API may not support all filter conditions.
As a beginning to solve it, we thought at least need to implement a RexCall
splitter for specific sources to split filter conditions to two conditions,
which was a hard problem. Think about A OR B that A is a valid condition but B
is not but we cannot split it (cannot lose the OR to move only A to the
source). But it can be split for A AND B. It will also become complicated for
nested condition.
> Rule to solve the filter partially by end application and remaining by calcite
> --
>
> Key: CALCITE-3600
> URL: https://issues.apache.org/jira/browse/CALCITE-3600
> Project: Calcite
> Issue Type: Wish
>Reporter: anjali shrishrimal
>Priority: Minor
>
> Add Rule to check if filter condition is solvable by end application. If part
> of the filter condition can be solved by end application, then it should get
> pushed to end application, and remaining part which can not be solved by end
> application, should get solved by calcite secondarily (i.e. upon fetch remove
> unwanted data as per filter condition)
>
> Consider an application which can solve only limited operators while
> filtering, say "=,<,>" and can not solve operator 'LIKE'.
>
> Example, filter condition is "id > 1000 AND name LIKE '%an%'"
>
> we would like to restrict the condition passed to application to "id > 1000"
> and remaining part "name LIKE '%an%'" should get solved by calcite. (The way
> it does for csv-adapter)
>
> To replicate the situation, consider test-case testFilter in MongoAdapterTest
> (org.apache.calcite.adapter.mongodb.MongoAdapterTest) of mongo-adapter.
> And modify it like below:
>
> @Test public void testFilter()
> { assertModel(MODEL) .query("select state, city from zips where state = 'CA'
> AND city LIKE '%E%'") .returnsUnordered("STATE=CA; CITY=LOS ANGELES",
> "STATE=CA; CITY=BELL GARDENS"); }
>
>
> Expected output of above query :
> STATE=CA; CITY=LOS ANGELES,
> STATE=CA; CITY=BELL GARDENS
>
> Expected plan :
> EnumerableFilter(condition=[LIKE(CAST(ITEM($0, 'city')):VARCHAR(20), '%E%')])
> {{MongoToEnumerableConverter}}
> {{MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2)],
> CITY=[CAST(ITEM($0, 'city')):VARCHAR(20)])}}
> {{MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2), 'CA')])}}
> {{MongoTableScan(table=[[mongo_raw, zips]])}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
