Re: [VOTE] Release 1.12.5, release candidate #3

2021-07-29 Thread Jingsong Li
oint, everything is good. > >> > > >> > On Thu, Jul 29, 2021 at 2:34 PM Yun Tang wrote: > >> > > >> > > +1 (non-binding) > >> > > > >> > > Checked the signature. > >> > > > >> > > Revie

Re: [DISCUSS] Change Default Jira Priority from "Major" to "Minor"

2021-07-27 Thread Jingsong Li
I agree with Martijn. My problem is just minor, which will make me a little disappointed. Best, Jingsong On Tue, Jul 27, 2021 at 5:32 PM Martijn Visser wrote: > Hi, > > I personally would prefer to use "Normal" as a default priority because I > think a lot of people's first reaction is that t

[VOTE] Release 1.12.5, release candidate #3

2021-07-26 Thread Jingsong Li
Hi everyone, Please review and vote on the release candidate #3 for the version 1.12.5, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: [VOTE] FLIP-184: Refine ShuffleMaster lifecycle management for pluggable shuffle service framework

2021-07-21 Thread Jingsong Li
+1 (binding) Look forward to the production ready~ Best, Jingsong On Mon, Jul 19, 2021 at 3:25 PM Zhu Zhu wrote: > +1 (binding) > > Thanks, > Zhu > > XING JIN 于2021年7月19日周一 上午10:29写道: > > > +1 (non-binding) > > > > Best, > > Jin > > > > Guowei Ma 于2021年7月19日周一 上午9:41写道: > > > > > +1(binding)

Re: [NOTICE] flink-runtime now scala-free

2021-07-15 Thread Jingsong Li
Thanks Chesnay! Scala-free is really nice! Best, Jingsong On Fri, Jul 16, 2021 at 2:01 PM Zhu Zhu wrote: > Thanks Chesnay for this great work! > > Thanks, > Zhu > > Steven Wu 于2021年7月16日周五 上午12:49写道: > > > This is awesome. Thank you, Chesney! > > > > On Wed, Jul 14, 2021 at 1:50 AM Yun Tang

[VOTE] Release 1.12.5, release candidate #2

2021-07-15 Thread Jingsong Li
Hi everyone, Please review and vote on the release candidate #2 for the version 1.12.5, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-15 Thread Jingsong Li
BIG +1 for this. Thanks Stephan for starting this discussion. Too many warnings in IDE and mvn build are frustrating. We can have the opportunity to avoid the degradation of code quality. Best, Jingsong On Thu, Jul 15, 2021 at 3:26 PM David Morávek wrote: > I know that sonar can report back by

Re: [VOTE] Release 1.12.5, release candidate #1

2021-07-14 Thread Jingsong Li
> On Wed, Jul 7, 2021 at 1:29 PM Jingsong Li wrote: > > > Hi all, > > > > Thanks Xiongtong, Leonard, Yang, JING for the voting. Thanks Till for the > > information. > > > > +1 for canceling this RC. That should also give us the chance to fix > > FLI

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

2021-07-08 Thread Jingsong Li
Congratulations Yuan! Best, Jingsong On Thu, Jul 8, 2021 at 5:43 PM Arvid Heise wrote: > Yay! > > On Thu, Jul 8, 2021 at 10:02 AM Jiayi Liao > wrote: > > > Congratulations Yuan! > > > > Best, > > Jiayi Liao > > > > On Thu, Jul 8, 2021 at 3:55 PM Roman Khachatryan > wrote: > > > > > Congratula

Re: [VOTE] Release 1.12.5, release candidate #1

2021-07-07 Thread Jingsong Li
the result is expected > > > > - started SQL Client, ran a simple query, the result is expected > > > > - reviewed the web PR, left one minor name comment > > > > > > > > Best, > > > > Leonard > > > > > > > > > 在 20

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

2021-07-07 Thread Jingsong Li
Congratulations, Yang! Best, Jingsong On Wed, Jul 7, 2021 at 6:43 PM Arvid Heise wrote: > Congratulations! > > On Wed, Jul 7, 2021 at 12:17 PM godfrey he wrote: > > > Congratulations, Yang! > > > > Best, > > Godfrey > > > > Lijie Wang 于2021年7月7日周三 下午5:59写道: > > > > > Congratulations Yang! > >

Re: [ANNOUNCE] New PMC member: Guowei Ma

2021-07-07 Thread Jingsong Li
Congratulations, Guowei! Best, Jingsong On Wed, Jul 7, 2021 at 6:36 PM Arvid Heise wrote: > Congratulations! > > On Wed, Jul 7, 2021 at 11:30 AM Till Rohrmann > wrote: > > > Congratulations, Guowei! > > > > Cheers, > > Till > > > > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan > wrote: > >

Re: [VOTE] Release 1.13.2, release candidate #1

2021-07-04 Thread Jingsong Li
+1 (non-binding) - Verified checksums and signatures - Built from sources - run table example jobs - web-ui looks good - sql-client looks good I think we should update the unresolved JIRAs in [1] to 1.13.3. And we should check resolved JIRAs in [2], commits of some are not in the 1.13.2. We shou

[VOTE] Release 1.12.5, release candidate #1

2021-07-04 Thread Jingsong Li
Hi everyone, Please review and vote on the release candidate #1 for the version 1.12.5, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-28 Thread Jingsong Li
gt; > > > bumping/decreasing priorities of test instabilities (and bugs), > > but > > > > as > > > > > > > individual committer I don't have the full picture. As I wrote > > > > above, I > > >

Re: [DISCUSS] Releasing Flink 1.12.5

2021-06-27 Thread Jingsong Li
; > Best, > > Martijn > > > > On Thu, 24 Jun 2021 at 11:53, Jingsong Li wrote: > > > Dear devs, > > > > As discussed in [1], I would volunteer as the release manager for 1.12.5 > > and kick off the release process on next Wednesday (June 30th). &

[DISCUSS] Releasing Flink 1.12.5

2021-06-24 Thread Jingsong Li
Dear devs, As discussed in [1], I would volunteer as the release manager for 1.12.5 and kick off the release process on next Wednesday (June 30th). I went through 1.12.5 issues [2], all blockers have been fixed. If you think there is any blocker in 1.12.5, please reply to this thread at any time

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Jingsong Li
+1 to Xintong's proposal I also have some concerns about unstable cases. I think unstable cases can be divided into these types: - Force majeure: For example, network timeout, sudden environmental collapse, they are accidental and can always be solved by triggering azure again. Committers should

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Jingsong Li
+1 (binding) Thanks for driving. Best, Jingsong On Tue, Jun 22, 2021 at 12:07 AM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Mon, 21 Jun 2021 at 22:51, Timo Walther wrote: > > > +1 (binding) > > > > Thanks for driving this. > > > > Regards, > > Timo > > > > On 21.06.21 13:24, Ingo B

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Jingsong Li
+1 to the release. Thanks Dawid for driving this discussion. I am willing to volunteer as the release manager too. Best, Jingsong On Tue, Jun 22, 2021 at 9:58 AM godfrey he wrote: > Thanks for driving this, Dawid. +1 to the release. > I would also like to volunteer as the release manager for

Re: 退订

2021-06-20 Thread Jingsong Li
Hi, unsubscribe, please send to dev-unsubscr...@flink.apache.org instead of dev@flink.apache.org Best, Jingsong On Sat, Jun 19, 2021 at 9:51 PM 809590403 <809590...@qq.com.invalid> wrote: > 退订 -- Best, Jingsong Lee

Re: [ANNOUNCE] New PMC member: Arvid Heise

2021-06-16 Thread Jingsong Li
Congratulations, Arvid! Best, Jingsong On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax wrote: > Congrats! > > On 6/16/21 6:06 AM, Leonard Xu wrote: > > Congratulations, Arvid! > > > > > >> 在 2021年6月16日,20:08,Till Rohrmann 写道: > >> > >> Congratulations, Arvid! > >> > >> Cheers, > >> Till > >> >

Re: [ANNOUNCE] New PMC member: Xintong Song

2021-06-16 Thread Jingsong Li
Congratulations, Xintong! Best, Jingsong On Thu, Jun 17, 2021 at 10:26 AM Yun Tang wrote: > Congratulations, Xintong! > > Best > Yun Tang > > From: Leonard Xu > Sent: Wednesday, June 16, 2021 21:05 > To: dev (dev@flink.apache.org) > Subject: Re: [ANNOUNCE] New

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2021-06-06 Thread Jingsong Li
Thanks Yingjie for the great effort! This is really helpful to Flink Batch users! Best, Jingsong On Mon, Jun 7, 2021 at 10:11 AM Yingjie Cao wrote: > Hi devs & users, > > The FLIP-148[1] has been released with Flink 1.13 and the final > implementation has some differences compared with the ini

Re: Integrating new connector with Flink SQL.

2021-06-03 Thread Jingsong Li
urce abstraction? Is that > what you're recommending? > > 2. Also, if we integrate the Kafkadynamicsource in Flink table with the > KafkaSourceReader and KafkaSplitEnumerator abstractions, then would it be > possible for us to contribute it back to the community? > > Tha

Re: Integrating new connector with Flink SQL.

2021-06-02 Thread Jingsong Li
Hi santhosh, 1.I recommend you use the new source with ScanTablesource. 2.You can use `org.apache.flink.table.connector.source.SourceProvider` to integrate to ScanTablesource. (Introduced in 1.12) 3.You can just implement a new source, this one can be used by both Flink DataStream and Flink SQL.

Re: [DISCUSS] SQL CTAS Syntax

2021-06-01 Thread Jingsong Li
Thanks Danny for starting the discussion. +1 for this feature. I like schema evolution. A question: Can we support multiple pipelines? For example, I have three source tables and just one sink table. So, I will write three CTAS: - CTAS IF NOT EXISTS 1 - CTAS IF NOT EXISTS 2 - CTAS IF NOT EXIST

Re: Re: [ANNOUNCE] New Apache Flink Committer - Rui Li

2021-04-21 Thread Jingsong Li
Congratulations Rui! Best, Jingsong On Thu, Apr 22, 2021 at 11:52 AM Yun Gao wrote: > Congratulations Rui! > > Best, > Yun > > > -- > Sender:Nicholas Jiang > Date:2021/04/22 11:26:05 > Recipient: > Theme:Re: [ANNOUNCE] New Apache

Re: [DISCUSS] Simplify SQL lookup join (temporal join with latest) syntax

2021-04-20 Thread Jingsong Li
+1 for simplifying. We already have a thread of this topic: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Make-Temporal-Join-syntax-easier-to-use-td47296.html FYI. Best, Jingsong On Tue, Apr 20, 2021 at 4:55 PM Gyula Fóra wrote: > Hi All! > > Playing around with the SQ

Re: [DISCUSS] FLIP-154: SQL Implicit Type Coercion

2021-03-14 Thread Jingsong Li
11, 2021 at 3:41 PM Danny Chan wrote: > Thanks for the feedback, Jingsong ~ > > This design mainly follows the behaviors of PostgreSQL and SQL-SERVER, > because their rules are more in line with the SQL standard. > > I have fixed the WIKI and add more details about the diff in i

Re: [Parquet support]

2021-03-11 Thread Jingsong Li
> https://github.com/apache/flink/pull/14961 > > > > If I have time, I'll also tackle the other parquet tickets that I > > opened lately > > > > Best > > > > Etienne > > > > On 25/02/2021 08:34, Jingsong Li wrote: > >> Hi Etienne, &

Re: Upgrade calcite version

2021-03-11 Thread Jingsong Li
Hi, Yes, as Danny said, it is very hard work... A suggestion is that you can cherry-pick some bugfixs from the new Calcite version to your own internal Calcite branch, if you just want to fix some bugs. Best, Jingsong On Thu, Mar 11, 2021 at 2:28 PM Danny Chan wrote: > Hi Sheng ~ > > It is a

Re: [DISCUSS] FLIP-154: SQL Implicit Type Coercion

2021-03-10 Thread Jingsong Li
Thanks Danny for starting this discussion. Big +1 for Implicit Type Coercion, in my opinion, it is very useful for writing SQL conveniently. I think there are two orthogonal things to discuss here: 1.[Matrix] Which types and which types can be implicitly converted. 2.[Strategies] In different cas

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Jingsong Li
+1 Let's go straight to the right behavior. Drop the option for the wrong behavior. Best, Jingsong On Tue, Mar 9, 2021 at 4:29 PM Timo Walther wrote: > Hi Leonard, > > I'm fine with dropping the old buggy behavior immediatly. Users can > still implement a UDF with the old bavhior if needed. I

Re: [VOTE] FLIP-162: Consistent Flink SQL time function behavior

2021-03-03 Thread Jingsong Li
+1 Thanks Leonard for driving this topic. Best, Jingsong On Wed, Mar 3, 2021 at 4:44 PM Kurt Young wrote: > +1 (binding) > > Best, > Kurt > > > On Wed, Mar 3, 2021 at 3:43 PM Timo Walther wrote: > > > +1 (binding) > > > > Regards, > > Timo > > > > On 03.03.21 04:14, Jark Wu wrote: > > > +1 (b

Re: [Parquet support]

2021-02-24 Thread Jingsong Li
ang wrote: > > Hi Jingsong, > > > > Thanks for pointing this out. Actually, I planned to work on changing > > interfaces ParquetTableSource and ParquetInputFormat. > > After refactoring the code, I may also help to fix the issue in > > https://issues.apache.org/

Re: [Parquet support]

2021-02-23 Thread Jingsong Li
Hi Etienne, Thanks for your reporting. There are indeed many problems. There is no doubt that we need to improve our current format implementation. But ParquetTableSource and ParquetInputFormat are legacy implementations with legacy interfaces. We have introduced new interfaces for execution and

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-02 Thread Jingsong Li
Thanks Rui for the proposal, I think this FLIP is required by many users, and it is very good to traditional Hive users. I have some confusion: # Version Which Hive version do you want to choose? Maybe, Hive 3.X and Hive 2.X have some differences? # Hive Codes Can you evaluate how much code we

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-02-01 Thread Jingsong Li
+1 for the default "auto" to the "table.exec.time-function-evaluation". >From the definition of these functions, in my opinion: - Batch is the instant execution of all records, which is the meaning of the word "BATCH", so there is only one time at query-start. - Stream only executes a single recor

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-01 Thread Jingsong Li
Thanks for the proposal, yes, sql-client is too outdated. +1 for improving it. About "SET" and "RESET", Why not be "SET" and "UNSET"? Best, Jingsong On Mon, Feb 1, 2021 at 2:46 PM Rui Li wrote: > Thanks Shengkai for the update! The proposed changes look good to me. > > On Fri, Jan 29, 2021 at

Re: [ANNOUNCE] Welcome Danny Cranmer as a new Apache Flink Committer

2021-01-12 Thread Jingsong Li
Congrats, Danny! Best, Jingsong On Tue, Jan 12, 2021 at 7:55 PM Leonard Xu wrote: > Congratulations Danny! > > Best, > Leonard > > 在 2021年1月12日,19:44,Wei Zhong 写道: > > > > Congratulations Danny! > > > > Best, > > Wei > > > >> 在 2021年1月12日,19:09,hailongwang <18868816...@163.com> 写道: > >> > >> C

Re: Support local aggregate push down for Blink batch planner

2021-01-06 Thread Jingsong Li
at present. >>> >>> For core points (b & c): I think we can change the interface to be: >>> ``` >>> >>> boolean applyAggregates(int[] groupingFields, List >>> aggregateExpressions, DataType producedDataType, List >>> groupingSets);

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
t the existing > connector will produce wrong results > when upgrading to new Flink versions (as we are pushing > grouping_sets/filter_args, but connector ignores it). > I think for these cases, providing a new default method to override might > be a better choice. > > Best, >

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
Hi, I'm also curious about aggregate with filter (COUNT(1) FILTER(WHERE d > 1)). Can we push it down? I'm not sure that a single call expression can express it, and how we should embody it and convey it to users. Best, Jingsong On Wed, Jan 6, 2021 at 1:36 PM Jingsong Li wrot

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
as a cost of losing user convenience. > Foremost, we don't see any parameters to add in the future. Do you know > any potential parameters? > > Best, > Jark > > On Wed, 6 Jan 2021 at 10:28, Jingsong Li wrote: > >> Hi Sebastian, >> >> Well, I mean: >> &

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
nterface, > and holds the > `List < CallExpression > aggregateExpressions, int[] GroupingFields, > DataType producedDataType` > fields? > > Looking forward to your further feedback or guidance. > > Jingsong Li 于2021年1月5日周二 下午2:44写道: > >> Thanks for your proposal! Seb

Re: Support local aggregate push down for Blink batch planner

2021-01-04 Thread Jingsong Li
Thanks for your proposal! Sebastian. +1 for SupportsAggregatePushDown. The above wonderful discussion has solved many of my concerns. ## Semantic problems We may need to add some mechanisms or comments, because as far as I know, the semantics of each database is actually different, which may nee

Re: [DISCUSS] Allow streaming operators to use managed memory

2021-01-04 Thread Jingsong Li
+1 for allowing streaming operators to use managed memory. The memory use of streams requires some hierarchy, and the bottom layer is undoubtedly the current StateBackend. Let the stream operators freely use the managed memory, which will make the memory management model to be unified and give the

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Jingsong Li
Thanks for volunteering as our release manager Xintong. +1 for releasing Flink 1.12.1 soon. I think https://issues.apache.org/jira/browse/FLINK-20665 should be addressed, I marked it as a Blocker. Best, Jingsong On Fri, Dec 18, 2020 at 11:16 AM Yang Wang wrote: > Hi David, > > I will take a lo

Re: Hive Streaming write compaction

2020-11-18 Thread Jingsong Li
Hi Chen, Table Filesystem/Hive sink file compaction has been merged into master, detail in [1]. It is included in Flink 1.12. Hope you can have a try and test. [1]https://issues.apache.org/jira/browse/FLINK-19345 Best, Jingsong On Thu, Nov 19, 2020 at 2:31 PM Chen Qin wrote: > Hi there, > >

Re: Re: Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)

2020-11-11 Thread Jingsong Li
+1 On Thu, Nov 12, 2020 at 2:07 PM hailongwang <18868816...@163.com> wrote: > > > +1(no-binding) > > > > > 在 2020-11-11 19:44:40,"刘大龙" 写道: > > > >+1 > > > >> -原始邮件- > >> 发件人: "Timo Walther" > >> 发送时间: 2020-11-11 18:55:06 (星期三) > >> 收件人: dev@flink.apache.org > >> 抄送: > >> 主题: Re: [VOTE]

Re: [VOTE] Remove flink-connector-filesystem module.

2020-10-29 Thread Jingsong Li
+1 On Thu, Oct 29, 2020 at 7:59 PM Chesnay Schepler wrote: > +1 > > On 10/29/2020 9:18 AM, Kostas Kloudas wrote: > > Hi all, > > > > Following the discussion in [1], I would like to start a vote on > > removing the flink-connector-filesystem module which includes the > > BucketingSink. > > > > T

Re: [ANNOUNCE] New Apache Flink Committer - Congxian Qiu

2020-10-29 Thread Jingsong Li
Congratulations! On Thu, Oct 29, 2020 at 3:56 PM Yuan Mei wrote: > Congratulations! > > On Thu, Oct 29, 2020 at 3:53 PM Till Rohrmann > wrote: > > > Congratulations Congxian! Great to have you as a committer now :-) > > > > Cheers, > > Till > > > > On Thu, Oct 29, 2020 at 8:33 AM Benchao Li wr

Re: [DISCUSS] Remove flink-connector-filesystem module.

2020-10-28 Thread Jingsong Li
+1 to remove the Bucketing Sink. Thanks for the effort on ORC and `HadoopPathBasedBulkFormatBuilder`, I think it's safe to get rid of the old Bucketing API with them. Best, Jingsong On Thu, Oct 29, 2020 at 3:06 AM Kostas Kloudas wrote: > Thanks for the discussion! > > From this thread I do not

Re: [DISCUSS] Releasing Apache Flink 1.11.3

2020-10-28 Thread Jingsong Li
+1 to backport the FLIP-27 adjustments to 1.11.x. If possible, that would be great. Many people are looking forward to the FLIP-27 interface, but they don't want to take the risk to upgrade to 1.12 (And wait 1.12). After all, 1.11 is a relatively stable version. Best, Jingsong On Thu, Oct 29, 20

Re: [VOTE] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2020-10-25 Thread Jingsong Li
+1 Best, Jingsong On Mon, Oct 26, 2020 at 11:52 AM Zhijiang wrote: > Thanks for driving this improvement, Yingjie! > > +1 (binding) > > Best, > Zhijiang > > > -- > From:Kurt Young > Send Time:2020年10月26日(星期一) 11:41 > To:dev > Sub

Re: [VOTE]FLIP-149: Introduce the upsert-kafka connector

2020-10-23 Thread Jingsong Li
+1 On Fri, Oct 23, 2020 at 3:52 PM Konstantin Knauf wrote: > +1 > > On Fri, Oct 23, 2020 at 9:36 AM Jark Wu wrote: > > > +1 > > > > On Fri, 23 Oct 2020 at 15:25, Shengkai Fang wrote: > > > > > Hi, all, > > > > > > I would like to start the vote for FLIP-149[1], which is discussed and > > > rea

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-23 Thread Jingsong Li
9 PM Kurt Young wrote: > To be precise, it means the Kakfa topic should set the configuration > "cleanup.policy" to "compact" not "delete". > > Best, > Kurt > > > On Fri, Oct 23, 2020 at 4:01 PM Jingsong Li > wrote: > > > I

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-23 Thread Jingsong Li
; > >> [1] > >> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-149%3A+Introduce+the+upsert-kafka+Connector > >> > >> Shengkai Fang 于2020年10月23日周五 下午2:55写道: > >> > >>> Hi, all. > >>> It seems we have reached

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Jingsong Li
impala_upsert.html > [6]: > > https://help.sap.com/viewer/7c78579ce9b14a669c1f3295b0d8ca16/Cloud/en-US/ea8b6773be584203bcd99da76844c5ed.html > [7]: https://phoenix.apache.org/atomic_upsert.html > [8]: > > https://docs.oracle.com/en/database/other-databases/nosql-database/1

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Jingsong Li
t;>>>>>>>>> Hi devs, > >>>>>>>>>>> > >>>>>>>>>>> As many people are still confused about the difference option > >>>>>>>>> behaviours > >>>>>>>>&g

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-20 Thread Jingsong Li
a while since it will prevent a lot of > duplicate > > > > > merging efforts. > > > > > > > > > > Regarding the date: I'm fine with the proposed date but I can also > > see > > > > > that extending it to the end of the week co

Re: Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-19 Thread Jingsong Li
is indeed an urgent need. > For example, In my inner flink jobs (1w+), we need to provide the ability > for users to customize parallelism, and the parallelism for Kafka always > is the number of partitions. > > > Look forwart to it ! > Best, > Hailong Wang > > 在 2020

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-19 Thread Jingsong Li
Hi Robert, Thanks for your detailed explanation. At present, we are preparing or participating in Flink forward, so +1 for appropriate extension of deadline. Best, Jingsong On Mon, Oct 19, 2020 at 5:36 PM Kurt Young wrote: > Can we change the freeze date to October 30th (Friday next week)? It

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-19 Thread Jingsong Li
Thanks Shengkai for your proposal. +1 for this feature. > Future Work: Support bounded KTable source I don't think it should be a future work, I think it is one of the important concepts of this FLIP. We need to understand it now. Intuitively, a ktable in my opinion is a bounded table rather th

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-18 Thread Jingsong Li
>> > >>> 2020年10月16日 上午10:05,Danny Chan 写道: > >>> > >>> +1, nice job ! > >>> > >>> Best, > >>> Danny Chan > >>> 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: > >>>> Hi all, > >>>> > >&

[VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Jingsong Li
Hi all, I would like to start the vote for FLIP-146 [1], which is discussed and reached consensus in the discussion thread [2]. The vote will be open until 20th Oct. (72h, exclude weekends), unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FL

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-13 Thread Jingsong Li
idual > operations in the DataStream API because it again takes freedom away > from the framework. So if it's really sth that users need we should go > ahead. > > Best, > Aljoscha > > On 09.10.20 13:57, Jingsong Li wrote: > > Hi Aljoscha, > > > > I want to

Re: [DISCUSS] Remove flink-connector-filesystem module.

2020-10-13 Thread Jingsong Li
Hi, I share a concern: Although we now support ORC Writer. It's not easy to support. We need to override something for ORC classes. Note that we are using a newer version of ORC, which is not forward compatible. Therefore, the data written by users using Flink Orc writer may not be readable by o

Re: [VOTE] FLIP-145: Support SQL windowing table-valued function

2020-10-11 Thread Jingsong Li
+1 Best, Jingsong On Sun, Oct 11, 2020 at 8:02 AM hailongwang <18868816...@163.com> wrote: > +1(non-binding) > > > > Best, > Hailong Wang > At 2020-10-10 17:06:07, "Jark Wu" wrote: > >Hi all, > > > >I would like to start the vote for FLIP-145 [1], which is discussed and > >reached consensus in

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-10 Thread Jingsong Li
+1 for voting. Thanks Jark for driving. +1 for TVF, It has been put forward by theory and supported by calcite. It will greatly enhance the window related operations. My personal feeling is that after TVF, the following operations can be similar to the traditional batch SQL, as long as the window

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-09 Thread Jingsong Li
bleSource > >>or ScanRuntimeProvider? > >>2. `scan.infer-parallelism.enabled` doesn't seem very useful to me. > From > >>a user's perspective, parallelism is either set by > `scan.parallelism`, or > >>automatically decided by F

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-09 Thread Jingsong Li
Congratulations, Zhu Zhu! On Fri, Oct 9, 2020 at 3:08 PM Zhijiang wrote: > Congratulations and welcome, Zhu Zhu! > > Best, > Zhijiang > -- > From:Yun Tang > Send Time:2020年10月9日(星期五) 14:20 > To:dev@flink.apache.org > Subject:Re: [

Re: [VOTE] FLIP-143: Unified Sink API

2020-09-28 Thread Jingsong Li
+1 (binding) Best, Jingsong On Mon, Sep 28, 2020 at 3:21 AM Kostas Kloudas wrote: > +1 (binding) > > @Steven Wu I think there will be opportunities to fine tune the API > during the implementation. > > Cheers, > Kostas > > On Sun, Sep 27, 2020 at 7:56 PM Steven Wu wrote: > > > > +1 (non-bindin

Re: [DISCUSS][Code-Style] The approach to implement singleton pattern

2020-09-25 Thread Jingsong Li
Hi, thanks for starting this discussion. I am +1 for using the private constructor for util class. We don't need to change it. I think few libraries use the enum, such as guava, common-utils, or even JDK, the private constructor is widely used. I don't quite understand why a util class is an enu

Re: [DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

2020-09-25 Thread Jingsong Li
t 2:49 AM Rui Li wrote: > > > >> +1 > >> > >> On Thu, Sep 24, 2020 at 2:59 PM Timo Walther > wrote: > >> > >>> +1 > >>> > >>> On 24.09.20 06:54, Jark Wu wrote: > >>>> +1 to move it there. > >>

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-25 Thread Jingsong Li
Hi Aljoscha, Thank you for your feedback, ## Connector parallelism Requirements: Set parallelism by user specified or inferred by connector. How to configure parallelism in DataStream: In the DataStream world, the only way to configure parallelism is `SingleOutputStreamOperator.setParallelism`.

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-24 Thread Jingsong Li
your > opinion. > > > > > > Regarding the interface `DataStreamScanProvider`, a concrete example > > would > > > help the discussion. What kind > > > of scenarios do you want to support? And what kind of connectors need > > such > > > an int

Re: [VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-24 Thread Jingsong Li
+1 (binding) Best, Jingsong On Thu, Sep 24, 2020 at 4:18 PM Kurt Young wrote: > +1 (binding) > > Best, > Kurt > > > On Thu, Sep 24, 2020 at 4:01 PM Timo Walther wrote: > > > Hi all, > > > > after the discussion in [1], I would like to open a second voting thread > > for FLIP-136 [2] which cove

[DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

2020-09-23 Thread Jingsong Li
Hi devs and users: After the 1.11 release, I heard some voices recently: How can't Hive's documents be found in the "Table & SQL Connectors". Actually, Hive's documents are in the "Table API & SQL". Since the "Table & SQL Connectors" document was extracted separately, Hive is a little out of plac

Re: Re: [DISCUSS] Support source/sink parallelism config in Flink sql

2020-09-23 Thread Jingsong Li
Hi , I have started a discussion about improving the new TableSource and TableSink: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-146-Improve-new-TableSource-and-TableSink-interfaces-td45161.html It includes parallelism setting, welcome to join the discussion and look

[DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-23 Thread Jingsong Li
Hi all, I'd like to start a discussion about improving the new TableSource and TableSink interfaces. Most connectors have been migrated to FLIP-95, but there are still the Filesystem and Hive that have not been migrated. They have some requirements on table connector API. And users also have some

Re: [DISCUSS] What is the correct site version?

2020-09-17 Thread Jingsong Li
ld update the "version" in docs/_config.yml in release-1.11 > branch. > > Best, > Jark > > On Fri, 18 Sep 2020 at 14:00, Chesnay Schepler wrote: > >> > Do we need to change this site version after releasing minor/bugfix >> >> versions? >>

[DISCUSS] What is the correct site version?

2020-09-17 Thread Jingsong Li
Hi Dev, Flink 1.11.2 has been released, (thanks ZhuZhu) But I found the site version on https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar is still 1.11.0. (1.10 is the same) For example: https://repo.maven.apache.org/maven2/org/apache/flink/flink-

Re: [VOTE] FLIP-107: Handling of metadata in SQL connectors

2020-09-17 Thread Jingsong Li
+1 It is very useful Best, Jingsong On Thu, Sep 17, 2020 at 11:12 AM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Tue, 15 Sep 2020 at 10:32, Leonard Xu wrote: > > > +1(non-binding) > > > > Leonard > > > > > 在 2020年9月12日,21:46,Danny Chan 写道: > > > > > > +1, non-binding ~ > > > > > > K

Re: [ANNOUNCE] Apache Flink 1.11.2 released

2020-09-16 Thread Jingsong Li
Thanks ZhuZhu for driving the release. Best, Jingsong On Thu, Sep 17, 2020 at 1:29 PM Zhu Zhu wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.11.2, which is the second bugfix release for the Apache Flink 1.11 > series. > > Apache Flink® is an open-s

Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He

2020-09-15 Thread Jingsong Li
Congratulations :) Best, Jingsong On Wed, Sep 16, 2020 at 12:45 PM Dian Fu wrote: > Congratulations, well deserved! > > Regards, > Dian > > > 在 2020年9月16日,下午12:36,Guowei Ma 写道: > > > > Congratulations :) > > > > Best, > > Guowei > > > > > > On Wed, Sep 16, 2020 at 12:19 PM Jark Wu wrote: > >

Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise

2020-09-15 Thread Jingsong Li
Congratulations Arvid ! Best, Jingsong On Tue, Sep 15, 2020 at 3:27 PM Dawid Wysakowicz wrote: > Congratulations Arvid! Very well deserved! > > Best, > > Dawid > > On 15/09/2020 04:38, Zhijiang wrote: > > Hi all, > > > > On behalf of the PMC, I’m very happy to announce Arvid Heise as a new > Fl

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-08 Thread Jingsong Li
11:08 AM Jingsong Li wrote: > Hi Zhu Zhu, > > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166 > > Will fix it soon. > > Best, > Jingsong > > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu wrote: > >> Hi All, >> >> Since there a

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-08 Thread Jingsong Li
14942 into 1.11.2. FLINK-14942 (this > >>> fixes a > >>> >> bug introduce in 1.11.0), there is a pr for it already. > >>> >> Best, > >>> >> Congxian > >>> >> > >>> >> > >>> >> Zhou, Brian 于2020

Re: [DISCUSS] Introduce partitioning strategies to Table/SQL

2020-09-02 Thread Jingsong Li
> separate FLIP for the parititioning topic. I'm currently working on an > update to FLIP-107 and would suggest to remove the paritioning topic > there. FLIP-107 will only focus on accessing metadata and expressing > key/value formats. > > What do you think? > > Regards,

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Jingsong Li
> a Row has two modes represented by an internal boolean flag `hasFieldOrder` +1 confusion with Dawid that what's the result when index-based setters and name-based setters are mixed used. And name-based setters look like append instead of set. It reminds me of Avro's `GenericRecord`, We should s

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Jingsong Li
Thanks Timo for driving. My first impression is, can we not deprecate these API? - StreamTableEnvironment.fromDataStream(DataStream): Table - StreamTableEnvironment.fromDataStream(DataStream, Expression...): Table - StreamTableEnvironment.createTemporaryView(String, DataStream, Expression...): Uni

Re: [DISCUSS] Introduce partitioning strategies to Table/SQL

2020-08-31 Thread Jingsong Li
titioning information is > > correct?! > > > > 2) I suppose it is up to the connector implementation whether/how to > > interpret the partition information. How will this work? > > > > 3) For Kafka, I suppose, the most common partitioning strategy is by key. > > FLIP

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

2020-08-25 Thread Jingsong Li
to the factory context sounds reasonable to > me, because the factory context should describe the full content of create > table DDL. > > Best, > Jark > > > On Tue, 25 Aug 2020 at 16:01, Jingsong Li wrote: > > > Hi Dawid, > > > > But the temporary table does

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

2020-08-25 Thread Jingsong Li
Hi Dawid, But the temporary table does not belong to Catalog, actually Catalog doesn't know the existence of the temporary table. Let the table factory of catalog to create source/sink sounds a little sudden. If we want to make temporary tables belong to Catalog, I think we need to involve catalo

[DISCUSS] Introduce partitioning strategies to Table/SQL

2020-08-24 Thread Jingsong Li
Hi all, ## Motivation FLIP-63 [1] introduced initial support for PARTITIONED BY clause to an extent that let us support Hive's partitioning. But this partition definition is completely specific to Hive/File systems, with the continuous development of the system, there are new requirements: - FLI

Re: [DISCUSS] Introduce SupportsParallelismReport and SupportsStatisticsReport for Hive and Filesystem

2020-08-04 Thread Jingsong Li
lism setting for the source operator, or do you want to let the > source operator set the parallelism? > > Best, > Kurt > > > On Fri, Jul 31, 2020 at 1:43 PM Jingsong Li > wrote: > > > Hi, thanks for your responses. > > > > To Benchao: > > > >

Re: [DISCUSS] Introduce SupportsParallelismReport and SupportsStatisticsReport for Hive and Filesystem

2020-07-30 Thread Jingsong Li
n is what the oldStats parameter is used for? > > > > 3) Internal or Public. I don't think we should mark them as internal. > They > > are currently only used by internal > > connectors doesn't mean this interface should be internal. I can imagine > &

Re: [DISCUSS] Align the semantic of returning null from InputFormat#nextRecord

2020-07-29 Thread Jingsong Li
o it's > common that we encounters dirty > data and want to skip it like in DeserializationSchema. > That's why I prefer #2. Anyway, If we are gonna to deprecate InputFormat in > the near future, I think there > is no need to change this in the community. > > > Jings

<    1   2   3   4   5   6   7   >