[jira] [Comment Edited] (FLINK-17826) Add missing custom query support on new jdbc connector
[
https://issues.apache.org/jira/browse/FLINK-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340636#comment-17340636
]
Flavio Pompermaier edited comment on FLINK-17826 at 5/7/21, 7:26 AM:
-
This feature was added in the old connector[1] and I think it becomes useless
if Flink allows to read only the necessary data: if I register the Flink
catalog a table for db1.A and 1 for db1.B is Flink able to optimize query
like "select * from table_A join table_B" without requiring to fetch all data
from A and B and then joining..? Wouldn't this be a waste of time?
If the problem is that it becomes a table while it should be a view (read-only)
maybe Flink could introduce the concept of VIEW. Maybe this could be a
reasonable trade-off for people coming from SQL world..
[1] https://issues.apache.org/jira/browse/FLINK-17361
was (Author: f.pompermaier):
This feature was added in the old connecto[1] and I think it becomes useless
if Flink allows to read only the necessary data (i.e. If I register the Flink
catalog a table for db1.A and 1 for db1.B is Flink able to optimize query
like "select * from table_A join table_B" without requiring to fetch all data
from A and B and then joining..? Wouldn't this be a waste of time?
If the problem is that it becomes a table while it should be a view (read-only)
maybe Flink could introduce the concept of VIEW. Maybe this could be a
reasonable trade-off for people coming from SQL world..
[1] https://issues.apache.org/jira/browse/FLINK-17361
> Add missing custom query support on new jdbc connector
> --
>
> Key: FLINK-17826
> URL: https://issues.apache.org/jira/browse/FLINK-17826
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
>Reporter: Jark Wu
>Assignee: Leonard Xu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> In FLINK-17361, we added custom query on JDBC tables, but missing to add the
> same ability on new jdbc connector (i.e.
> {{JdbcDynamicTableSourceSinkFactory}}).
> In the new jdbc connector, maybe we should call it {{scan.query}} to keep
> consistent with other scan options, besides we need to make {{"table-name"}}
> optional, but add validation that "table-name" and "scan.query" shouldn't
> both be empty, and "table-name" must not be empty when used as sink.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-17826) Add missing custom query support on new jdbc connector
[
https://issues.apache.org/jira/browse/FLINK-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335131#comment-17335131
]
Leonard Xu edited comment on FLINK-17826 at 4/29/21, 4:02 AM:
--
Hi, [~f.pompermaier]
The concern about the custom query is that
(1) After add a custom query, the JDBC table should be a view of JDBC (e.g
select * from db1.A join db1.B) instead of a table from semantics, the
'table-name' option is useless in this case.
(2) The table may can only be used as source table and can not as sink table
for example above case.
base on these, I tend to do not add the unclear option.
But If you're interesting this issue, you can take over and I can help review
and merge. We can add some specification for (1) and add more checks for (2)
was (Author: leonard xu):
Hi, [~f.pompermaier]
The concern about the custom query is that
(1) After add a custom query, the JDBC table should be a view of JDBC (e.g
select * from db1.A join db1.B) instead of a table from semantics.
(2) The table may can only be used as source table and can not as sink table
for example above case.
base on these, I tend to do not add the unclear option.
But If you're interesting this issue, you can take over and I can help review
and merge. We can add some specification for (1) and add more checks for (2)
> Add missing custom query support on new jdbc connector
> --
>
> Key: FLINK-17826
> URL: https://issues.apache.org/jira/browse/FLINK-17826
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
>Reporter: Jark Wu
>Assignee: Leonard Xu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> In FLINK-17361, we added custom query on JDBC tables, but missing to add the
> same ability on new jdbc connector (i.e.
> {{JdbcDynamicTableSourceSinkFactory}}).
> In the new jdbc connector, maybe we should call it {{scan.query}} to keep
> consistent with other scan options, besides we need to make {{"table-name"}}
> optional, but add validation that "table-name" and "scan.query" shouldn't
> both be empty, and "table-name" must not be empty when used as sink.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-17826) Add missing custom query support on new jdbc connector
[
https://issues.apache.org/jira/browse/FLINK-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180220#comment-17180220
]
Yao WANG edited comment on FLINK-17826 at 8/19/20, 2:35 AM:
[~Leonard Xu], ok, you can just refer FLINK-17361
was (Author: vaiome):
[~Leonard Xu], ok, you can just refer to FLINK-17361
> Add missing custom query support on new jdbc connector
> --
>
> Key: FLINK-17826
> URL: https://issues.apache.org/jira/browse/FLINK-17826
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
>Reporter: Jark Wu
>Assignee: Flavio Pompermaier
>Priority: Major
> Fix For: 1.12.0
>
>
> In FLINK-17361, we added custom query on JDBC tables, but missing to add the
> same ability on new jdbc connector (i.e.
> {{JdbcDynamicTableSourceSinkFactory}}).
> In the new jdbc connector, maybe we should call it {{scan.query}} to keep
> consistent with other scan options, besides we need to make {{"table-name"}}
> optional, but add validation that "table-name" and "scan.query" shouldn't
> both be empty, and "table-name" must not be empty when used as sink.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-17826) Add missing custom query support on new jdbc connector
[
https://issues.apache.org/jira/browse/FLINK-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179401#comment-17179401
]
Flavio Pompermaier edited comment on FLINK-17826 at 8/18/20, 6:41 AM:
--
Unfortunately I can't find the time to contribute to this right now...feel free
to assign it to you if you want
was (Author: f.pompermaier):
Unfortunately I can't fond the time to contribute to this right now...feel free
to assign it to you if you want
> Add missing custom query support on new jdbc connector
> --
>
> Key: FLINK-17826
> URL: https://issues.apache.org/jira/browse/FLINK-17826
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
>Reporter: Jark Wu
>Assignee: Flavio Pompermaier
>Priority: Major
> Fix For: 1.12.0
>
>
> In FLINK-17361, we added custom query on JDBC tables, but missing to add the
> same ability on new jdbc connector (i.e.
> {{JdbcDynamicTableSourceSinkFactory}}).
> In the new jdbc connector, maybe we should call it {{scan.query}} to keep
> consistent with other scan options, besides we need to make {{"table-name"}}
> optional, but add validation that "table-name" and "scan.query" shouldn't
> both be empty, and "table-name" must not be empty when used as sink.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
