Re: [DISCUSS] Update Scala 2.12.7 to 2.12.15

2022-05-20 Thread Ran Tao
2.13 and Scala 3 support > in Flink; I think that deserves its own discussion thread. > > Best regards, > > Martijn Visser > https://twitter.com/MartijnVisser82 > https://github.com/MartijnVisser > > [1] https://issues.apache.org/jira/browse/FLINK-12461 > [2] https://issues.apache.org/jira/browse/FLINK-25000 > -- Best, Ran Tao

Re: [DISCUSS] Update Scala 2.12.7 to 2.12.15

2022-05-20 Thread Ran Tao
hes the > version of the java byte code. > > By and large such inconsistencies are inevitable w.r.t. external libraries. > > On 20/05/2022 12:23, Ran Tao wrote: > > Hi, Martijn. Even if we upgrade scala to 2.12.15 to support Java17, it > just > > fix the compilation of FLINK-250

Re: [VOTE] Update Flink's Scala 2.12 support from 2.12.7 to 2.12.16

2022-06-10 Thread Ran Tao
; https://github.com/MartijnVisser > > [1] https://lists.apache.org/thread/9ft0tbhj45fplt5j3gtb7jzbt4tdr6rh > -- Best, Ran Tao

Re: [DISCUSS] Update Scala 2.12.7 to 2.12.15

2022-06-07 Thread Ran Tao
oting thread to make the > upgrade. > > Best regards, > > Martijn > > Op vr 20 mei 2022 om 14:10 schreef Ran Tao : > > > Got it. But I think the runtime java environment e.g. jdk11 env may > cannot > > optimize these scala lower bytecode very well. However

Re: [DISCUSS] Update Scala 2.12.7 to 2.12.15

2022-06-07 Thread Ran Tao
t haven't seen any progress since. Do you know more about the > timeline? > > I would also like to make a final call towards Scala users to provide their > input in the next 72 hours. Else, I'll open up a voting thread to make the > upgrade. > > Best regards, > > Martijn &

[DISCUSS ] HybridSouce Table & Sql api timeline

2022-05-06 Thread Ran Tao
HybridSource is a good feature, but now release version did not support table & sql api, i wonder which time it will be ready to end-users. And i have implemented a inner version of my company and it works well now. the implementation of table & sql api may involve some core questions, e.g.

Re: [DISCUSS ] HybridSouce Table & Sql api timeline

2022-05-11 Thread Ran Tao
g/jira/browse/FLINK-22793 > > On Mon, May 9, 2022 at 11:34 PM Becket Qin wrote: > > > > Cool, I've granted you the permission. > > > > Cheers, > > > > Jiangjie (Becket) Qin > > > > On Tue, May 10, 2022 at 1:14 PM Ran Tao wrote: > > > >

[DISCUSS] Add scala-2.13 support for implementing completed jdk11 or higher flink version

2022-05-09 Thread Ran Tao
ews/jdk-compatibility/overview.html -- Best, Ran Tao

Re: [DISCUSS] Add scala-2.13 support for implementing completed jdk11 or higher flink version

2022-05-09 Thread Ran Tao
om Java 8 to Java 11 run into the > problem that his savepoint isn't compatible if we would make such a change? > > Best regards, > > Martijn > > On Mon, 9 May 2022 at 13:31, Ran Tao wrote: > > > Hi, Martijn. Thanks for responding, i have read these issues, a

Re: [DISCUSS] Add scala-2.13 support for implementing completed jdk11 or higher flink version

2022-05-09 Thread Ran Tao
hink that was done before. > > Best regards, > > Martijn Visser > https://twitter.com/MartijnVisser82 > https://github.com/MartijnVisser > > [1] https://issues.apache.org/jira/browse/FLINK-13414 > [2 https://issues.apache.org/jira/browse/FLINK-20969 > [3] https://issues.ap

Re: [DISCUSS ] HybridSouce Table & Sql api timeline

2022-05-09 Thread Ran Tao
Hi, Becket. Thanks for your suggestions. My id is: Ran Tao And i will draft this flip in a few days. thanks~ Becket Qin 于2022年5月10日周二 12:40写道: > Hi Ran, > > The FLIP process can be found here[1]. > > You don't need to pass the vote, in fact the vote is based on the FLIP >

Re: [DISCUSS ] HybridSouce Table & Sql api timeline

2022-05-09 Thread Ran Tao
Hi, Martijn, Jacky. Thanks for your responding. It indeed need a designed doc or FLIP to illustrate some details and concrete implementation. And i'm glad to work on this issue. I wonder whether i can create a FLIP under discussion firstly to write the draft design of the implementation about

[DISCUSS] Support 'DYNAMIC MetadataColumn' in flink table

2022-08-25 Thread Ran Tao
. If a certain connector without this metadata, the column value will return null otherwise return it's concrete value. It has great benefits in some scenarios. Looking forward to your opinions. -- Best Regards, Ran Tao https://github.com/chucheng92

Re: [DISCUSS] Support 'DYNAMIC MetadataColumn' in flink table

2022-08-29 Thread Ran Tao
gt; > If the metadata is not from the message properties, how does the connector > know which field to convert from? > Shouldn’t the connector be modified to support this new metadata column? > > Best, > Jark > > > > > 2022年8月26日 15:30,Ran Tao 写道: > > > >

Re: [DISCUSS] Support 'DYNAMIC MetadataColumn' in flink table

2022-08-29 Thread Ran Tao
w does the > connector know which field to convert from? > > Shouldn’t the connector be modified to support this new metadata column? > > > > Best, > > Jark > > > > > > > >> 2022年8月26日 15:30,Ran Tao 写道: > >> > >>

Re: [DISCUSS] Support 'DYNAMIC MetadataColumn' in flink table

2022-09-01 Thread Ran Tao
olumn DEFAULT is a reasonable feature and can also help in other > cases. > I’m fine with adding this feature. > Do you want to prepare a FLIP for it? > > Best, > Jark > > > 2022年8月29日 15:02,Ran Tao 写道: > > > > Hi Jack. Timo summed it up very well. In fact, my problem i

Re: [DISCUSS] Support 'DYNAMIC MetadataColumn' in flink table

2022-08-26 Thread Ran Tao
rough this property and can be > accessed via `SELECT col['my-new-property'] FROM x` including being NULL > be default if not present. > > Regards, > Timo > > > On 25.08.22 14:04, Ran Tao wrote: > > ``` > > create table test_source( > > __test_metadata__ varchar ME

[DISCUSS] Hybrid Source Connector

2022-12-08 Thread Ran Tao
rote a discussed FLIP, look forward to your comments. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235836225 -- Best Regards, Ran Tao https://github.com/chucheng92

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2022-12-16 Thread Ran Tao
OffsetsInitializer.timestamp(switchTimestamp + 1) but how would you > specify that in the SQL landscape? > > Last but not least: your examples are all SQL only. How do you propose that > this works in the Table API? > > Best regards, > > Martijn > > On Thu, Dec 15, 202

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2022-12-19 Thread Ran Tao
and finally > switch over to live data from Kafka or something). > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/kafka/#start-reading-position > [2] > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/hybridsou

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2022-12-19 Thread Ran Tao
a mistake, childSources.get(sourceIndex).setStartTimetamp(switchedTimestamp); Ran Tao 于2022年12月19日周一 16:10写道: > Hi, John. thanks for your comments. > About question-2 the "handoff" is using for switching next source > seamlessly. but it's an option. Not every hybrid sourc

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2022-12-15 Thread Ran Tao
Fyi. This flip using index as child source option prefix because we may use the same connector as hybrid child sources. e.g. create table hybrid_source( f0 varchar, f1 varchar, f2 bigint ) with( 'connector'='hybrid', 'sources'='filesystem,filesystem', '0.path' = '/tmp/a.csv', '0.format' =

[DISCUSS] FLIP-278: Hybrid Source Connector

2022-12-15 Thread Ran Tao
ar future. Thanks, Timo ``` -- Best Regards, Ran Tao https://github.com/chucheng92

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-06 Thread Ran Tao
thanks Lau. The vote will last for at least 72 hours (03/09, 19:30 UTC+8). It needs consensus approval, requiring 3 binding +1 votes and no binding vetoes. Best Regards, Ran Tao Jacky Lau 于2023年3月7日周二 15:11写道: > Thanks Ran. > +1 (non-binding) > > Regards, > Jacky Lau > >

Re: [VOTE] Release 1.17.0, release candidate #2

2023-03-16 Thread Ran Tao
Hi, Martijn, QingSheng. I have a issue[1], QingSheng has reviewed and approved it. We need to backport it to release-1.17[2]. please help to do final checking. thanks. [1] https://issues.apache.org/jira/browse/FLINK-31319 [2] https://github.com/apache/flink/pull/22193 Best Regards, Ran Tao

Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement

2023-03-23 Thread Ran Tao
to support it. [1] https://issues.apache.org/jira/browse/FLINK-31464 [2] https://issues.apache.org/jira/browse/FLINK-31368 Best Regards, Ran Tao https://github.com/chucheng92 yuxia 于2023年3月22日周三 21:13写道: > Hi, devs. > > I'd like to start a discussion about FLIP-302: Support TRUNC

Re: [ANNOUNCE] Kafka Connector Code Removal from apache/flink:main branch and code freezing

2023-03-24 Thread Ran Tao
Thank you Gordon and all the people who have worked on the externalized kafka implementation. I have another pr related to Kafka[1]. I will be very appreciative if you can help me review it in your free time. [1] https://github.com/apache/flink-connector-kafka/pull/10 Best Regards, Ran Tao Tzu

Re: [VOTE] FLIP-293: Introduce Flink Jdbc Driver For Sql Gateway

2023-03-14 Thread Ran Tao
It's a very nice improvement. +1 (non-binding) Best Regards, Ran Tao Shammon FY 于2023年3月13日周一 13:47写道: > Hi Devs, > > I'd like to start the vote on FLIP-293: Introduce Flink Jdbc Driver For Sql > Gateway [1]. > > The FLIP was discussed in thread [2], and it aims to in

Re: [VOTE] Release 1.17.0, release candidate #2

2023-03-16 Thread Ran Tao
Hi Martijn, Leonard. Thanks for your explanation. Best Regards, Ran Tao Leonard Xu 于2023年3月16日周四 22:32写道: > Thanks everyone for validating the RC2! > > As the new blocker issue FLINK-31485[1] we found , the RC2 of 1.17.0 is > officially canceled. > We'll build a new RC o

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-09 Thread Ran Tao
Thanks, everyone, I'm closing this vote now. I'll follow up with the result in another email. Best Regards, Ran Tao Ran Tao 于2023年3月7日周二 15:59写道: > thanks Lau. > > The vote will last for at least 72 hours (03/09, 19:30 UTC+8). > It needs consensus approval, requiring 3 bind

[RESULT][VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-09 Thread Ran Tao
) There are no disapproving votes. Thanks everyone for participating! [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-297%3A+Improve+Auxiliary+Sql+Statements Best Regards, Ran Tao

[DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Ran Tao
] https://docs.google.com/document/d/1hAiOfPx14VTBTOlpyxG7FA2mB1k5M31VnKYad2XpJ1I/ [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-297%3A+Improve+Auxiliary+Sql+Statements Best Regards, Ran Tao

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-25 Thread Ran Tao
Hi, Jark, thanks for explantations. if there are no other questions, i will try to start a VOTE to address current things in this FLIP. We may open another dedicated FLIP to discuss INFORMATION_SCHEMA. hi, guys, WDYT? Best Regards, Ran Tao Jark Wu 于2023年2月25日周六 22:57写道: > Hi Sergey, &g

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-26 Thread Ran Tao
. yes, two parameters for constructor can work. thanks for your advice. Best Regards, Ran Tao Jing Ge 于2023年2月27日周一 06:29写道: > Hi, > > @Aitozi > > This is exactly why LoD has been introduced: to avoid exposing internal > structure(2nd and lower level API).

[DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Ran Tao
No describe query No current flink only supports describing tables. Also, please let me know if there is a mistake. Looking forward to your reply. Best Regards, Ran Tao https://github.com/chucheng92

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Ran Tao
way to solve this problem to avoid subsequent frequent changes to the parser module and keywords. Best Regards, Ran Tao https://github.com/chucheng92 Timo Walther 于2023年2月21日周二 22:58写道: > Hi Ran, > > adding additional filter and selection capabilities makes sense. > However, I wou

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-27 Thread Ran Tao
? Best Regards, Ran Tao Sergey Nuyanzin 于2023年2月27日周一 21:12写道: > Hi Jark, > > thanks for your comment. > > >Considering they > > are orthogonal and information schema requires more complex design and > > discussion, it deserves a separate FLIP > I'm ok with a sep

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-27 Thread Ran Tao
Hi Jing, I think we can support it, it's not a very difficult job. thank you. Best Regards, Ran Tao Jing Ge 于2023年2月28日周二 06:04写道: > Hi Ran, > > It would be great if we could support ILIKE [1]. If you figure out that the > effort is bigger than you expected, you might want to cre

[VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-06 Thread Ran Tao
(03/09, 19:30 UTC+8) unless there is an objection or insufficient votes. Thank you all. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-297%3A+Improve+Auxiliary+Sql+Statements [2] https://lists.apache.org/thread/54fyd27m8on1cf3hn6dz564zqmkobjyd Best Regards, Ran Tao https://github.com

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-03-04 Thread Ran Tao
Hi all, If there is no question related to this FLIP, we will start the voting thread on 3/6 (Monday). thank you all. Best Regards, Ran Tao https://github.com/chucheng92 Ran Tao 于2023年3月1日周三 17:40写道: > Hi, Timo. thanks. > > I have copied the content of docs and updated the FLIP. &

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-24 Thread Ran Tao
plugin? Just Like: Kafka Hudi Files Best Regards, Ran Tao Sergey Nuyanzin 于2023年2月24日周五 19:11写道: > Thanks for driving the FLIP > > I have a couple of questions > Am I right that INFORMATION_SCHEMA mentioned by Timo[1] is out of scope of > this FLIP? > I noticed there are some

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Ran Tao
. what do you think? Best Regards, Ran Tao https://github.com/chucheng92 Jing Ge 于2023年2月23日周四 00:50写道: > Hi Ran, > > Thanks for driving the FLIP. It looks overall good. Would you like to add > a description of useLike and notLike? I guess useLike true is for "LIKE&qu

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-24 Thread Ran Tao
and opinions later. Thank you all. Best Regards, Ran Tao John Roesler 于2023年2月24日周五 21:53写道: > Hello Ran, > > Thanks for the FLIP! > > Do you mind if we revisit the topic of doing this by adding an Information > schema? The SHOW approach requires modifying the parser/language

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Ran Tao
[] listDatabases() { return catalogManager .getCatalog(catalogManager.getCurrentCatalog()) .get() .listDatabases() .toArray(new String[0]); } Best Regards, Ran Tao Jingsong Li 于2023年2月23日周四 11:47写道: > Thanks for the propo

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Ran Tao
Hi Jingsong. thanks. i got it. In this way, there is no need to introduce new API changes. Best Regards, Ran Tao Jingsong Li 于2023年2月23日周四 12:26写道: > Hi Ran, > > I mean we can just use > TableEnvironment.getCatalog(getCurrentCatalog).get().listDatabases(). > > We don't n

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Ran Tao
thanks Sergey, sounds good. You can add in FLIP ticket[1]. [1] https://issues.apache.org/jira/browse/FLINK-31256 Best Regards, Ran Tao https://github.com/chucheng92 Sergey Nuyanzin 于2023年2月28日周二 19:44写道: > >>Currently I think we can load from the jar and check the services file to

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Ran Tao
ary we may continue to discuss. Hi, Sergey, WDYT?or can we open a single ticket under this FLIP? Best Regards, Ran Tao Jark Wu 于2023年2月28日周二 17:45写道: > Besides, if we introduce the ILIKE, we should also add this feature for > the previous SHOW with LIKE statements. They should be include

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Ran Tao
Thanks. I will create a FLIP to illustrate some details. Best Regards, Ran Tao https://github.com/chucheng92 Jark Wu 于2023年2月21日周二 20:03写道: > Thank you, > > I think this is worth a FLIP design doc to discuss the detailed syntax. > Could you prepare a FLIP for that? >

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Ran Tao
de, your syntax table is broken. Could you share it with a > Google doc or create a JIRA issue? > > Best, > Jark > > > > > 2023年2月21日 17:51,Ran Tao 写道: > > > > Hi guys. When I recently used flink sql to manage internal metadata > > (catalog/databas

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-23 Thread Ran Tao
, Ran Tao Jing Ge 于2023年2月23日周四 18:58写道: > Hi Jingson, > > Thanks for sharing your thoughts. Please see my reply below. > > On Thu, Feb 23, 2023 at 10:16 AM Jingsong Li > wrote: > > > Hi Jing Ge, > > > > First, flink-table-common contains all common classes

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-03-01 Thread Ran Tao
it in table & sql api and not need to change sql keywords or Parser.jj. However, in flink docs, we may let users know that this is not standard, just for better use. Best Regards, Ran Tao Timo Walther 于2023年2月28日周二 22:23写道: > Hi Ran Tao, > > Thanks for working on this F

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Ran Tao
Yes, I totally agree with your point. The current flink sql syntax is tiled together without classification. Let me try to summarize these four categories, not just Auxiliary Statements. And compare it with other popular engines. Thank you, Jing ! Best Regards, Ran Tao https://github.com

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

2023-04-26 Thread Ran Tao
Congratulations, Qingsheng! Best Regards, Ran Tao Qingsheng Ren 于2023年4月24日周一 19:53写道: > Thank you all! I'm so honored and happy to work with all contributors in > the Apache Flink community. Let's make Flink better together! > > Cheers, > Qingsheng > > On Mon, Apr 24,

Re: [jira] [Created] (FLINK-31660) flink-connectors-kafka is not runnable in the IDE

2023-03-29 Thread Ran Tao
mvn test. 3.clone external kafka-connector[1] repo which it works to me. [1] https://github.com/apache/flink-connector-kafka Best Regards, Ran Tao https://github.com/chucheng92 Natea Eshetu Beshada (Jira) 于2023年3月30日周四 01:29写道: > Natea Eshetu Beshada created FLIN

Re: [DISCUSS] Add support for Apache Arrow format

2023-03-30 Thread Ran Tao
for integrating Arrow. I think you can try to make a simple poc for verification and some specific plans. Best Regards, Ran Tao Aitozi 于2023年3月29日周三 19:12写道: > Hi guys > I'm opening this thread to discuss supporting the Apache Arrow format > in Flink. > Arrow is a language

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-04-10 Thread Ran Tao
Best Regards, Ran Tao Ran Tao 于2022年12月15日周四 16:02写道: > Hi guys. HybridSource is a good feature, but now released version did not > support table & sql api for a long time. > > I have wrote a discussed FLIP. > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235

Re: [VOTE] Release 1.17.0, release candidate #3

2023-03-20 Thread Ran Tao
this release candidate. Best Regards, Ran Tao Qingsheng Ren 于2023年3月17日周五 22:02写道: > Hi everyone, > > Please review and vote on the release candidate #3 for the version 1.17.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-02-03 Thread Ran Tao
ld you > specify that in the SQL landscape? > > Last but not least: your examples are all SQL only. How do you propose that > this works in the Table API? > > Best regards, > > Martijn > > On Thu, Dec 15, 2022 at 9:16 AM Ran Tao wrote: > > > Fyi. > > &g

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-02-03 Thread Ran Tao
at draft pr[2], it works well. 1.https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235836225 2.https://github.com/apache/flink/pull/21841 Ran Tao 于2022年12月19日周一 16:16写道: > a mistake, > childSources.get(sourceIndex).setStartTimetamp(switchedTimestamp); > > Ran Tao 于2022

Re: Confusion about some overlapping functionality of SupportsProjectionPushDown and SupportsReadingMetadata

2023-02-09 Thread Ran Tao
:09写道: > Hi, Ran Tao. > Thanks for bring it up. > TBH, to me, it's not as so confusing. > Is that the fact that the applyReadableMetadata and applyProjection all > will pass producedDataType and the source conneector developer will > need to choose which one as the finnal output typ

Confusion about some overlapping functionality of SupportsProjectionPushDown and SupportsReadingMetadata

2023-02-08 Thread Ran Tao
Regards, Ran Tao https://github.com/chucheng92

Re: [ANNOUNCE] New Apache Flink Committer - Jing Ge

2023-02-15 Thread Ran Tao
Congratulations JingGe! Best Regards, Ran Tao Shiwei Wang 于2023年2月15日周三 12:48写道: > Congratulations Jing! > > > > > 在 2023年2月14日,16:00,weijie guo 写道: > > > > Congratulations Jing! >

Re: Confusion about some overlapping functionality of SupportsProjectionPushDown and SupportsReadingMetadata

2023-02-12 Thread Ran Tao
t; Yuxia > > - 原始邮件 - > 发件人: "yuxia" > 收件人: "dev" > 发送时间: 星期一, 2023年 2 月 13日 上午 10:11:53 > 主题: Re: Confusion about some overlapping functionality of > SupportsProjectionPushDown and SupportsReadingMetadata > > Hi, Ran Tao. > I a

[DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

2023-07-03 Thread Ran Tao
Regards, Ran Tao https://github.com/chucheng92

Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

2023-07-03 Thread Ran Tao
. And bundle it in sql-connector uber jar. Then there is only one thing left that we need to make flink-python test not depend on the released flink-connector. Maybe we should check it out and decouple it like you suggested. Best Regards, Ran Tao https://github.com/chucheng92 Martijn Visser 于

Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

2023-07-04 Thread Ran Tao
. Best Regards, Ran Tao https://github.com/chucheng92 Dian Fu 于2023年7月4日周二 19:37写道: > Thanks Ran Tao for proposing this discussion and Martijn for sharing > the thought. > > > While flink-python now fails the CI, it shouldn't actually depend on the > externalized connectors.

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-06-30 Thread Ran Tao
define 3 fields, but only select 2 fields, the length hybrid generated row is 2 (both batch and streaming are 2). The processing of ProjectPushDown is consistent for the first batch source and the subsequent streaming source. Best Regards, Ran Tao https://github.com/chucheng92 Ilya Soin 于2023年5

Re: Whether to handle removed partitions in KafkaSourceEnumerator ?

2023-05-09 Thread Ran Tao
of removedPartitions will be printed every time of getPartitionChange. You can try to discuss and fix this problem if you are interested. Best Regards, Ran Tao Hongshun Wang 于2023年5月10日周三 02:34写道: > Hi Devs, > > There are some to-do comments and variables related to the removed > parti

Re: Whether to handle removed partitions in KafkaSourceEnumerator ?

2023-05-10 Thread Ran Tao
in SourceEnumerator). Best Regards, Ran Tao Hongshun Wang 于2023年5月10日周三 11:10写道: > Hi Ran Tao, > > Thank you for your response. It's very helpful. > > I still have some questions: > > Currently, if a partition is removed, the reader will not be notified about > it. Will

Re: Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-05-10 Thread Ran Tao
ere the batch and streaming schemas are consistent. I will update the POC pr then you can re-run your case. WDYT? Best Regards, Ran Tao Илья Соин 于2023年5月11日周四 03:12写道: > Hi devs, > > I think for this approach to work, the internal record schema generated by > Flink must be exact

Re: [DISCUSS] FLIP-278: Hybrid Source Connector

2023-05-09 Thread Ran Tao
Hi, devs. I don't know if you have any other considerations for this FLIP. All discussions are welcome. If there are no other opinions in the near days, I will try to initiate a vote. thank you all. Best Regards, Ran Tao Ran Tao 于2023年4月10日周一 15:33写道: > Hi, devs. I want to reo

[jira] [Created] (FLINK-8638) Job restart when Checkpoint On Barrier failed

2018-02-11 Thread Ran Tao (JIRA)
Ran Tao created FLINK-8638: -- Summary: Job restart when Checkpoint On Barrier failed Key: FLINK-8638 URL: https://issues.apache.org/jira/browse/FLINK-8638 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-27381) HybridSource split should use Arrays.hashcode

2022-04-25 Thread Ran Tao (Jira)
Ran Tao created FLINK-27381: --- Summary: HybridSource split should use Arrays.hashcode Key: FLINK-27381 URL: https://issues.apache.org/jira/browse/FLINK-27381 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27806) Datagen add binary & varbinary type support

2022-05-26 Thread Ran Tao (Jira)
Ran Tao created FLINK-27806: --- Summary: Datagen add binary & varbinary type support Key: FLINK-27806 URL: https://issues.apache.org/jira/browse/FLINK-27806 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-27886) Error way for skipping shade plugin in sub-modules

2022-06-02 Thread Ran Tao (Jira)
Ran Tao created FLINK-27886: --- Summary: Error way for skipping shade plugin in sub-modules Key: FLINK-27886 URL: https://issues.apache.org/jira/browse/FLINK-27886 Project: Flink Issue Type

[jira] [Created] (FLINK-27529) HybridSourceSplitEnumerator sourceIndex use error Integer check

2022-05-06 Thread Ran Tao (Jira)
Ran Tao created FLINK-27529: --- Summary: HybridSourceSplitEnumerator sourceIndex use error Integer check Key: FLINK-27529 URL: https://issues.apache.org/jira/browse/FLINK-27529 Project: Flink Issue

[jira] [Created] (FLINK-27547) Hardcode of pom aboout java11 & java17 target java version may cause hidden error

2022-05-08 Thread Ran Tao (Jira)
Ran Tao created FLINK-27547: --- Summary: Hardcode of pom aboout java11 & java17 target java version may cause hidden error Key: FLINK-27547 URL: https://issues.apache.org/jira/browse/FLINK-27547 Pro

[jira] [Created] (FLINK-27549) Inconsistent bytecode version when enable jdk11 or higher version

2022-05-08 Thread Ran Tao (Jira)
Ran Tao created FLINK-27549: --- Summary: Inconsistent bytecode version when enable jdk11 or higher version Key: FLINK-27549 URL: https://issues.apache.org/jira/browse/FLINK-27549 Project: Flink

[jira] [Created] (FLINK-28786) Cannot run PyFlink 1.16 on MacOS with M1 chip

2022-08-03 Thread Ran Tao (Jira)
Ran Tao created FLINK-28786: --- Summary: Cannot run PyFlink 1.16 on MacOS with M1 chip Key: FLINK-28786 URL: https://issues.apache.org/jira/browse/FLINK-28786 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-30334) SourceCoordinator error split check cause HybridSource hang

2022-12-07 Thread Ran Tao (Jira)
Ran Tao created FLINK-30334: --- Summary: SourceCoordinator error split check cause HybridSource hang Key: FLINK-30334 URL: https://issues.apache.org/jira/browse/FLINK-30334 Project: Flink Issue

[jira] [Created] (FLINK-30424) Add source operator restore readerState log to distinguish split is from newPartitions or split state

2022-12-14 Thread Ran Tao (Jira)
Ran Tao created FLINK-30424: --- Summary: Add source operator restore readerState log to distinguish split is from newPartitions or split state Key: FLINK-30424 URL: https://issues.apache.org/jira/browse/FLINK-30424

[jira] [Created] (FLINK-30259) Using flink Preconditions Util instead of uncertain Assert keyword to do checking

2022-12-01 Thread Ran Tao (Jira)
Ran Tao created FLINK-30259: --- Summary: Using flink Preconditions Util instead of uncertain Assert keyword to do checking Key: FLINK-30259 URL: https://issues.apache.org/jira/browse/FLINK-30259 Project

[jira] [Created] (FLINK-30274) Add commons-collections4 to replace commons-collections 3.x

2022-12-02 Thread Ran Tao (Jira)
Ran Tao created FLINK-30274: --- Summary: Add commons-collections4 to replace commons-collections 3.x Key: FLINK-30274 URL: https://issues.apache.org/jira/browse/FLINK-30274 Project: Flink Issue

[jira] [Created] (FLINK-31452) Show operations should better return sorted results

2023-03-14 Thread Ran Tao (Jira)
Ran Tao created FLINK-31452: --- Summary: Show operations should better return sorted results Key: FLINK-31452 URL: https://issues.apache.org/jira/browse/FLINK-31452 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-31481) Add filter support for ShowDatabases

2023-03-15 Thread Ran Tao (Jira)
Ran Tao created FLINK-31481: --- Summary: Add filter support for ShowDatabases Key: FLINK-31481 URL: https://issues.apache.org/jira/browse/FLINK-31481 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-31618) Broken links in docs for Pulsar connector

2023-03-26 Thread Ran Tao (Jira)
Ran Tao created FLINK-31618: --- Summary: Broken links in docs for Pulsar connector Key: FLINK-31618 URL: https://issues.apache.org/jira/browse/FLINK-31618 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-31574) Cleanup unused private methods in OperationConverterUtils

2023-03-22 Thread Ran Tao (Jira)
Ran Tao created FLINK-31574: --- Summary: Cleanup unused private methods in OperationConverterUtils Key: FLINK-31574 URL: https://issues.apache.org/jira/browse/FLINK-31574 Project: Flink Issue Type

[jira] [Created] (FLINK-31491) table planner NestedLoopJoinTest.testLeftOuterJoinWithFilter failed in branch release1.16

2023-03-16 Thread Ran Tao (Jira)
Ran Tao created FLINK-31491: --- Summary: table planner NestedLoopJoinTest.testLeftOuterJoinWithFilter failed in branch release1.16 Key: FLINK-31491 URL: https://issues.apache.org/jira/browse/FLINK-31491

[jira] [Created] (FLINK-31380) Add filter support for ShowOperations

2023-03-09 Thread Ran Tao (Jira)
Ran Tao created FLINK-31380: --- Summary: Add filter support for ShowOperations Key: FLINK-31380 URL: https://issues.apache.org/jira/browse/FLINK-31380 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-31382) Add select query ILIKE predicate support

2023-03-09 Thread Ran Tao (Jira)
Ran Tao created FLINK-31382: --- Summary: Add select query ILIKE predicate support Key: FLINK-31382 URL: https://issues.apache.org/jira/browse/FLINK-31382 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-31472) AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread

2023-03-15 Thread Ran Tao (Jira)
Ran Tao created FLINK-31472: --- Summary: AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread Key: FLINK-31472 URL: https://issues.apache.org/jira/browse/FLINK-31472 Project: Flink

[jira] [Created] (FLINK-31473) Add new show operations docs

2023-03-15 Thread Ran Tao (Jira)
Ran Tao created FLINK-31473: --- Summary: Add new show operations docs Key: FLINK-31473 URL: https://issues.apache.org/jira/browse/FLINK-31473 Project: Flink Issue Type: Sub-task Components

[jira] [Created] (FLINK-31256) FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Ran Tao (Jira)
Ran Tao created FLINK-31256: --- Summary: FLIP-297: Improve Auxiliary Sql Statements Key: FLINK-31256 URL: https://issues.apache.org/jira/browse/FLINK-31256 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-31319) Inconsistent condition judgement about kafka partitionDiscoveryIntervalMs cause potential bug

2023-03-03 Thread Ran Tao (Jira)
Ran Tao created FLINK-31319: --- Summary: Inconsistent condition judgement about kafka partitionDiscoveryIntervalMs cause potential bug Key: FLINK-31319 URL: https://issues.apache.org/jira/browse/FLINK-31319

[jira] [Created] (FLINK-31281) PythonFunctionRunner doesn't extend AutoCloseable but implements close

2023-03-01 Thread Ran Tao (Jira)
Ran Tao created FLINK-31281: --- Summary: PythonFunctionRunner doesn't extend AutoCloseable but implements close Key: FLINK-31281 URL: https://issues.apache.org/jira/browse/FLINK-31281 Project: Flink

[jira] [Created] (FLINK-31742) Remove deprecated TableSchema in flink-table-planner test

2023-04-05 Thread Ran Tao (Jira)
Ran Tao created FLINK-31742: --- Summary: Remove deprecated TableSchema in flink-table-planner test Key: FLINK-31742 URL: https://issues.apache.org/jira/browse/FLINK-31742 Project: Flink Issue Type

[jira] [Created] (FLINK-30935) Add KafkaSerializer deserialize check when using SimpleVersionedSerializer

2023-02-07 Thread Ran Tao (Jira)
Ran Tao created FLINK-30935: --- Summary: Add KafkaSerializer deserialize check when using SimpleVersionedSerializer Key: FLINK-30935 URL: https://issues.apache.org/jira/browse/FLINK-30935 Project: Flink

[jira] [Created] (FLINK-32310) Support enhanced show functions syntax

2023-06-12 Thread Ran Tao (Jira)
Ran Tao created FLINK-32310: --- Summary: Support enhanced show functions syntax Key: FLINK-32310 URL: https://issues.apache.org/jira/browse/FLINK-32310 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-32522) Flink sql connector kafka should include commons-collections in shade phase

2023-07-03 Thread Ran Tao (Jira)
Ran Tao created FLINK-32522: --- Summary: Flink sql connector kafka should include commons-collections in shade phase Key: FLINK-32522 URL: https://issues.apache.org/jira/browse/FLINK-32522 Project: Flink

[jira] [Created] (FLINK-32485) Flink State Backend Changelog should support build test-jar

2023-06-29 Thread Ran Tao (Jira)
Ran Tao created FLINK-32485: --- Summary: Flink State Backend Changelog should support build test-jar Key: FLINK-32485 URL: https://issues.apache.org/jira/browse/FLINK-32485 Project: Flink Issue

  1   2   >