[
https://issues.apache.org/jira/browse/CALCITE-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995451#comment-16995451
]
Rui Wang edited comment on CALCITE-3600 at 12/13/19 8:15 AM:
-------------------------------------------------------------
I think in BeamSQL we had same problem, due to the reason that underlying
source read API may not support all filter conditions.
As a beginning to solve it, we thought at least need to implement a RexCall
splitter for specific sources to split filter conditions to two conditions(one
can be executed by source and another cannot), which was a hard problem. Think
about A OR B that A is a valid condition but B is not but we cannot split it
(cannot lose the OR to move only A to the source). But it can be split for A
AND B. It will also become complicated for nested condition.
was (Author: amaliujia):
I think in BeamSQL we had same problem, due to the reason that underlying
source read API may not support all filter conditions.
As a beginning to solve it, we thought at least need to implement a RexCall
splitter for specific sources to split filter conditions to two conditions,
which was a hard problem. Think about A OR B that A is a valid condition but B
is not but we cannot split it (cannot lose the OR to move only A to the
source). But it can be split for A AND B. It will also become complicated for
nested condition.
> Rule to solve the filter partially by end application and remaining by calcite
> ------------------------------------------------------------------------------
>
> Key: CALCITE-3600
> URL: https://issues.apache.org/jira/browse/CALCITE-3600
> Project: Calcite
> Issue Type: Wish
> Reporter: anjali shrishrimal
> Priority: Minor
>
> Add Rule to check if filter condition is solvable by end application. If part
> of the filter condition can be solved by end application, then it should get
> pushed to end application, and remaining part which can not be solved by end
> application, should get solved by calcite secondarily (i.e. upon fetch remove
> unwanted data as per filter condition)
>
> Consider an application which can solve only limited operators while
> filtering, say "=,<,>" and can not solve operator 'LIKE'.
>
> Example, filter condition is "id > 1000 AND name LIKE '%an%'"
>
> we would like to restrict the condition passed to application to "id > 1000"
> and remaining part "name LIKE '%an%'" should get solved by calcite. (The way
> it does for csv-adapter)
>
> To replicate the situation, consider test-case testFilter in MongoAdapterTest
> (org.apache.calcite.adapter.mongodb.MongoAdapterTest) of mongo-adapter.
> And modify it like below:
>
> @Test public void testFilter()
> { assertModel(MODEL) .query("select state, city from zips where state = 'CA'
> AND city LIKE '%E%'") .returnsUnordered("STATE=CA; CITY=LOS ANGELES",
> "STATE=CA; CITY=BELL GARDENS"); }
>
>
> Expected output of above query :
> STATE=CA; CITY=LOS ANGELES,
> STATE=CA; CITY=BELL GARDENS
>
> Expected plan :
> EnumerableFilter(condition=[LIKE(CAST(ITEM($0, 'city')):VARCHAR(20), '%E%')])
> {{MongoToEnumerableConverter}}
> {{MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2)],
> CITY=[CAST(ITEM($0, 'city')):VARCHAR(20)])}}
> {{MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2), 'CA')])}}
> {{MongoTableScan(table=[[mongo_raw, zips]])}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
