Hi Jing Flink-Jdbc-Driver is mainly used to facilitate users to submit flink sql jobs. I think we will not use Flink-Jdbc-Driver in Flink-Jdbc-Connector to read/write data in general. If users can read/write specific tables by Flink-Jdbc-Driver, they can use these tables in their flink jobs directly. Thanks
Best, Shammon On Mon, Mar 13, 2023 at 6:21 AM Jing Ge <j...@ververica.com.invalid> wrote: > Hi, Shammon, > > This FLIP is very interesting. Thanks for driving it! Just out of > curiosity, Could it be used in the Flink JDBC connector[1] after > doing the appropriate implementation there? > > Best regards, > Jing > > [1] > > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/jdbc/ > > On Fri, Mar 3, 2023 at 8:56 AM Shammon FY <zjur...@gmail.com> wrote: > > > Hi, > > > > Thanks jinsong. I think implementing JDBC for batch mode first sounds > good. > > This will simplify the implementation and we can also remove the row kind > > first. We can claim this in the FLIP and docs, I will update the FLIP. > > > > Best, > > Shammon > > > > > > On Fri, Mar 3, 2023 at 2:36 PM Jingsong Li <jingsongl...@gmail.com> > wrote: > > > > > Hi, Shammon, > > > > > > I took a look at JDBC `ResultSet` and `Statement`. They are > > > complicated and have many interfaces. Some of the interfaces may not > > > be very suitable for streaming. > > > > > > I think maybe we can just implement JDBC for batch/olap only. It is > > > hard to have an integration for JDBC and streaming... > > > > > > Do you need to use JDBC in streaming mode? Or do we just implement > > > JDBC for batch only first? > > > > > > Best, > > > Jingsong > > > > > > > > > On Thu, Mar 2, 2023 at 6:22 PM Shammon FY <zjur...@gmail.com> wrote: > > > > > > > > Hi > > > > > > > > Thanks for the feedback from Jingsong and Benchao. > > > > > > > > For @Jingsong > > > > > If the user does not cast into a FlinkResultSet, will there be > > > > serious consequences here (RowKind is ignored)? > > > > > > > > I agree with you that it's indeed a big deal if users ignore the row > > kind > > > > when they must know it. One idea that comes to my mind is we can add > an > > > > option such as `table.result.changelog-mode`, users can set it > through > > > > connection properties or set dynamic parameters. The option value can > > be > > > > `insert-only`, `upset` or `all` and the default value is > `insert-only`. > > > > > > > > If the result does not conform to the changelog mode, the jdbc driver > > > > throws an exception. What do you think? > > > > > > > > > > > > For @Benchao > > > > > Besides `java.sql.Driver`, have you considered also adding support > > for > > > > `javax.sql.DataSource` interface? > > > > > > > > I missed the `javax.sql.DataSource` and I have added it to the FLIP, > > > thanks > > > > Benchao > > > > > > > > > > > > Best, > > > > Shammon > > > > > > > > On Wed, Mar 1, 2023 at 7:57 PM Benchao Li <libenc...@apache.org> > > wrote: > > > > > > > > > +1 for the FLIP, thanks Shammon for driving this. > > > > > > > > > > JDBC is quite useful in OLAP scenarios, supporting JDBC would > enable > > > Flink > > > > > to be used with existing tools, such as Tableau. > > > > > > > > > > Regarding the JDBC interfaces listed in the FLIP, I think they > looks > > > good > > > > > already. Besides `java.sql.Driver`, have you considered also adding > > > support > > > > > for `javax.sql.DataSource` interface? > > > > > > > > > > Jingsong Li <jingsongl...@gmail.com> 于2023年3月1日周三 17:53写道： > > > > > > > > > > > Thanks Shammon for driving. > > > > > > > > > > > > Big +1 for this. > > > > > > > > > > > > I heard that many users want to use FlinkGateway + JDBC to do > some > > > > > > queries, but at present, only Hive JDBC can be used. It is Hive > > > > > > dialect by default, and the experience is also different from > > > > > > FlinkSQL. We need to have our own JDBC. > > > > > > > > > > > > I took a look at your `Public Interface` part, only > > > > > > `FlinkResultSet.getRowKind` is a true new interface, others are > > just > > > > > > implementations. > > > > > > > > > > > > If the user does not cast into a FlinkResultSet, will there be > > > serious > > > > > > consequences here (RowKind is ignored)? > > > > > > > > > > > > Best, > > > > > > Jingsong > > > > > > > > > > > > On Wed, Mar 1, 2023 at 4:59 PM Shammon FY <zjur...@gmail.com> > > wrote: > > > > > > > > > > > > > > Hi devs, > > > > > > > > > > > > > > I'd like to start a discussion about FLIP-293: Introduce Flink > > Jdbc > > > > > > Driver > > > > > > > For Sql Gateway[1]. > > > > > > > > > > > > > > FLIP-275[2] supports remote sql client based on gateway, users > > can > > > > > > interact > > > > > > > with gateway by flink console. However, for users who create > > > session > > > > > > > clusters with Flink, they'd like to use Jdbc Driver to interact > > > with > > > > > the > > > > > > > gateway in their applications, such as olap queries.. > > > > > > > > > > > > > > I have discussed this proposal with @shengkaifang and > > @jinsonglee. > > > In > > > > > > this > > > > > > > FLIP, we'd like to introduce Jdbc Driver for gateway. Users can > > use > > > > > Jdbc > > > > > > > Driver to submit their queries and get results like a database > in > > > their > > > > > > > applications. > > > > > > > > > > > > > > Looking forward to your feedback, thanks. > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-293%3A+Introduce+Flink+Jdbc+Driver+For+Sql+Gateway > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-275%3A+Support+Remote+SQL+Client+Based+on+SQL+Gateway > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > Shammon > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best, > > > > > Benchao Li > > > > > > > > > > >
