Hi Jing
I think your thoughts are interesting. We can consider a flink cluster with
Jdbc-Driver as a warehouse and flink jobs can interact with it by
Flink-Jdbc-Connector. This may have higher requirements for the throughput
and latency of the warehouse (flink cluster), but we can try it after
Hi,
@Benchao
you are right, no, the question did not intend to block this FLIP.
@Shammon
It looks quite interesting, if we imagine how the Flink jdbc connector
could work with Flink jdbc driver. It makes "on-the-fly"
cross-flink-applications data consumption possible, i.e. bounded source
scan,
Hi Jing & Shammon,
IMO, Jdbc is a Java standard and Flink-JDBC-Driver is an implementation to
interact with Flink, so in theory, it would be doable to use
Flink-JDBC-Driver in Flink-JDBC-Connector if we add it as a dialect.
However, I don't see a scenario which requires this yet. All in all, it
Hi Jing
Flink-Jdbc-Driver is mainly used to facilitate users to submit flink sql
jobs. I think we will not use Flink-Jdbc-Driver in Flink-Jdbc-Connector to
read/write data in general. If users can read/write specific tables by
Flink-Jdbc-Driver, they can use these tables in their flink jobs
Hi, Shammon,
This FLIP is very interesting. Thanks for driving it! Just out of
curiosity, Could it be used in the Flink JDBC connector[1] after
doing the appropriate implementation there?
Best regards,
Jing
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/jdbc/
On
Hi,
Thanks jinsong. I think implementing JDBC for batch mode first sounds good.
This will simplify the implementation and we can also remove the row kind
first. We can claim this in the FLIP and docs, I will update the FLIP.
Best,
Shammon
On Fri, Mar 3, 2023 at 2:36 PM Jingsong Li wrote:
>
Hi, Shammon,
I took a look at JDBC `ResultSet` and `Statement`. They are
complicated and have many interfaces. Some of the interfaces may not
be very suitable for streaming.
I think maybe we can just implement JDBC for batch/olap only. It is
hard to have an integration for JDBC and streaming...
Hi
Thanks for the feedback from Jingsong and Benchao.
For @Jingsong
> If the user does not cast into a FlinkResultSet, will there be
serious consequences here (RowKind is ignored)?
I agree with you that it's indeed a big deal if users ignore the row kind
when they must know it. One idea that
+1 for the FLIP, thanks Shammon for driving this.
JDBC is quite useful in OLAP scenarios, supporting JDBC would enable Flink
to be used with existing tools, such as Tableau.
Regarding the JDBC interfaces listed in the FLIP, I think they looks good
already. Besides `java.sql.Driver`, have you
Thanks Shammon for driving.
Big +1 for this.
I heard that many users want to use FlinkGateway + JDBC to do some
queries, but at present, only Hive JDBC can be used. It is Hive
dialect by default, and the experience is also different from
FlinkSQL. We need to have our own JDBC.
I took a look at
Hi devs,
I'd like to start a discussion about FLIP-293: Introduce Flink Jdbc Driver
For Sql Gateway[1].
FLIP-275[2] supports remote sql client based on gateway, users can interact
with gateway by flink console. However, for users who create session
clusters with Flink, they'd like to use Jdbc
11 matches
Mail list logo
