Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Gyula Fóra
I agree to create a new RC based on this find. And 48h voting time seems reasonable as all functional testing will carry over to this new candidate. Gyula On Wed, Jun 1, 2022 at 7:56 AM Yang Wang wrote: > Thanks all for your testing and patience. > > And sorry for I have to cancel this VOTE

[jira] [Created] (FLINK-27860) List the CSS/docs dependencies in the NOTICE

2022-05-31 Thread Yang Wang (Jira)
Yang Wang created FLINK-27860: - Summary: List the CSS/docs dependencies in the NOTICE Key: FLINK-27860 URL: https://issues.apache.org/jira/browse/FLINK-27860 Project: Flink Issue Type: Bug

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Yang Wang
Thanks all for your testing and patience. And sorry for I have to cancel this VOTE since @Márton Balassi found a license issue. We do not list the CSS/docs dependencies in the NOTICE file of source release[1]. I will create another release candidate today including this fix. Given that there's

Re: [DISCUSS] FLIP-231: Introduce SupportStatisticReport to support reporting statistics from source connectors

2022-05-31 Thread godfrey he
Hi, Jing. Thanks for the suggestion, I have updated the doc and will continue to optimize the code in subsequent PR. Best, Godfrey Jing Ge 于2022年6月1日周三 04:41写道: > > Hi Godfrey, > > Thanks for clarifying it. I personally prefer the new change you suggested. > > Would you please help to

Re: [VOTE] FLIP-235: Hybrid Shuffle Mode

2022-05-31 Thread weijie guo
Hi all, The voting time for FLIP-235 has passed. I'm closing the vote now. There were 5 votes, 3 of which are binding: - Aitozi - Jing Ge - Xintong Song (binding) - Yangze Guo (binding) - Yingjie Cao (binding) There were no disapproving votes. Thus, FLIP-235 has been accepted. Thanks everyone

[jira] [Created] (FLINK-27859) Hive dialect should support create table that begins with digits

2022-05-31 Thread luoyuxia (Jira)
luoyuxia created FLINK-27859: Summary: Hive dialect should support create table that begins with digits Key: FLINK-27859 URL: https://issues.apache.org/jira/browse/FLINK-27859 Project: Flink

[jira] [Created] (FLINK-27858) Hive source support filter push down for parquet format

2022-05-31 Thread luoyuxia (Jira)
luoyuxia created FLINK-27858: Summary: Hive source support filter push down for parquet format Key: FLINK-27858 URL: https://issues.apache.org/jira/browse/FLINK-27858 Project: Flink Issue Type:

[jira] [Created] (FLINK-27857) Hive source support filter push down for orc format

2022-05-31 Thread luoyuxia (Jira)
luoyuxia created FLINK-27857: Summary: Hive source support filter push down for orc format Key: FLINK-27857 URL: https://issues.apache.org/jira/browse/FLINK-27857 Project: Flink Issue Type:

Re: About Native Deployment's Autoscaling implementation

2022-05-31 Thread Talat Uyarer
Hi Yang and Gyula, Yang, Could you give a little bit more information ? What prevents us from changing task managers' count ? I am aware of ActiveResourceManager of Flink. But Flink only calls resources when it initializes a cluster. If we set - jobmanager.scheduler: adaptive -

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Jim Busche
Hi Yang, Thank you for RC3 +1 (not-binding) * Podman builds look great with your .dockerignore file and “COPY . .” Thank you for the fix. * Twistlock security scans look clean of both your “ghcr.io/apache/flink-kubernetes-operator:52b50c2 image” as well as a locally built image

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-31 Thread Jing Ge
Hi Jark, Thanks for clarifying it. It would be fine. as long as we could provide the no-cache solution. I was just wondering if the client side cache could really help when HBase is used, since the data to look up should be huge. Depending how much data will be cached on the client side, the data

Re: [DISCUSS] FLIP-231: Introduce SupportStatisticReport to support reporting statistics from source connectors

2022-05-31 Thread Jing Ge
Hi Godfrey, Thanks for clarifying it. I personally prefer the new change you suggested. Would you please help to understand one more thing? The else if (filterPushDownSpec != null) branch is the only branch that doesn't have to check if the newTableStat has been calculated previously. The reason

[jira] [Created] (FLINK-27856) Adding pod template without spec crashes job manager

2022-05-31 Thread Jeesmon Jacob (Jira)
Jeesmon Jacob created FLINK-27856: - Summary: Adding pod template without spec crashes job manager Key: FLINK-27856 URL: https://issues.apache.org/jira/browse/FLINK-27856 Project: Flink Issue

[jira] [Created] (FLINK-27855) Job Manager fails to recover with S3 storage and HA enabled

2022-05-31 Thread Sushant (Jira)
Sushant created FLINK-27855: --- Summary: Job Manager fails to recover with S3 storage and HA enabled Key: FLINK-27855 URL: https://issues.apache.org/jira/browse/FLINK-27855 Project: Flink Issue

[jira] [Created] (FLINK-27854) Refactor FlinkImageBuilder and FlinkContainerBuilder

2022-05-31 Thread Alexander Fedulov (Jira)
Alexander Fedulov created FLINK-27854: - Summary: Refactor FlinkImageBuilder and FlinkContainerBuilder Key: FLINK-27854 URL: https://issues.apache.org/jira/browse/FLINK-27854 Project: Flink

[jira] [Created] (FLINK-27853) FLIP-229: Introduces Join Hint for Flink SQL Batch Job

2022-05-31 Thread xuyang (Jira)
xuyang created FLINK-27853: -- Summary: FLIP-229: Introduces Join Hint for Flink SQL Batch Job Key: FLINK-27853 URL: https://issues.apache.org/jira/browse/FLINK-27853 Project: Flink Issue Type: New

Re: [DISCUSS] FLIP-231: Introduce SupportStatisticReport to support reporting statistics from source connectors

2022-05-31 Thread godfrey he
Hi Jark and Jing, +1 to use "report" instead of "collect". > // only filter push down (*the description means > partitionPushDownSpec == null but misses the case of > partitionPushDownSpec != null*) `if (partitionPushDownSpec != null && filterPushDownSpec == null)` this branch is only consider

[jira] [Created] (FLINK-27852) Contribute k8s operator to OperatorHub

2022-05-31 Thread Jira
Márton Balassi created FLINK-27852: -- Summary: Contribute k8s operator to OperatorHub Key: FLINK-27852 URL: https://issues.apache.org/jira/browse/FLINK-27852 Project: Flink Issue Type:

Re: [VOTE] FLIP-235: Hybrid Shuffle Mode

2022-05-31 Thread Yingjie Cao
+1 Best, Yingjie Jing Ge 于2022年5月30日周一 05:02写道: > +1 (non-binding) > > Best Regards, > Jing > > On Sun, May 29, 2022 at 5:16 AM Aitozi wrote: > > > +1 (non-binding) > > > > Best, > > Aitozi > > > > Yangze Guo 于2022年5月27日周五 11:17写道: > > > > > +1 > > > > > > Best, > > > Yangze Guo > > > > > >

Re: [DISCUSS] Bi-Weekly Flink Community Sync Meeting

2022-05-31 Thread Konstantin Knauf
Hi everyone, can you be more specific what you mean by "current topics in the Flink Community"? Shouldn't asynchronous communication be the default, and if that doesn't work, we consider a synchronous channel? Cheers, Konstantin Am Di., 31. Mai 2022 um 13:54 Uhr schrieb Jing Ge : > +1 >

Re: [DISCUSS] Bi-Weekly Flink Community Sync Meeting

2022-05-31 Thread Chesnay Schepler
I'm not necessarily against collecting ideas/topics around the community side of things and discussing them in some way, but Im very much not a friend of immediately starting out such an effort with synchronous meetings. IMO we should only setup such meetings after we have identified a clear

Re: [DISCUSS] Bi-Weekly Flink Community Sync Meeting

2022-05-31 Thread Jing Ge
+1 Sounds good! Thanks Robert! On Tue, May 31, 2022 at 1:46 PM Márton Balassi wrote: > Hi Robert, > > Thanks for the suggestion +1 from me. You already listed the topic of the > timezone on the wiki that I wanted to bring up. > > On Tue, May 31, 2022 at 9:38 AM Robert Metzger > wrote: > > > Hi

Re: [DISCUSS] Bi-Weekly Flink Community Sync Meeting

2022-05-31 Thread Márton Balassi
Hi Robert, Thanks for the suggestion +1 from me. You already listed the topic of the timezone on the wiki that I wanted to bring up. On Tue, May 31, 2022 at 9:38 AM Robert Metzger wrote: > Hi everyone, > > We currently have a bi-weekly release sync meeting on Google Meet every > Tuesday at 9am

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Robert Metzger
I renamed the channel. On Tue, May 31, 2022 at 10:54 AM Xintong Song wrote: > Ok, then let's try it out. > > Best, > > Xintong > > > > On Tue, May 31, 2022 at 4:24 PM Robert Metzger > wrote: > > > +1 to merging #contribution-helps with #dev. > > > > I'm pretty sure we'll have to revisit this

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Yang Wang
+1 (binding) Successfully verified the following: - Verify that the checksums and GPG files - Verify that the source distributions do not contain any binaries - Build binary and image from release source - Verify the NOTICE and licenses in the docker image - Verify the helm chart values with

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Xintong Song
Ok, then let's try it out. Best, Xintong On Tue, May 31, 2022 at 4:24 PM Robert Metzger wrote: > +1 to merging #contribution-helps with #dev. > > I'm pretty sure we'll have to revisit this once we have a bit > more experience with running the slack community. > > On Tue, May 31, 2022 at

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Robert Metzger
+1 to merging #contribution-helps with #dev. I'm pretty sure we'll have to revisit this once we have a bit more experience with running the slack community. On Tue, May 31, 2022 at 10:07 AM Jark Wu wrote: > I'm fine with the #dev channel. > I remembered in the previous discussion that most

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Nicholas Jiang
Hi Yang! +1 (not-binding) Verified the following successfully: - Build from source, build container from source - Run the examples for application, session and session job deployments successfully and without any errors. Regards, Nicholas Jiang On 2022/05/31 06:26:02 Yang Wang wrote: > Hi

[jira] [Created] (FLINK-27851) Join can't access the pk from source table

2022-05-31 Thread xuyang (Jira)
xuyang created FLINK-27851: -- Summary: Join can't access the pk from source table Key: FLINK-27851 URL: https://issues.apache.org/jira/browse/FLINK-27851 Project: Flink Issue Type: Bug

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Jark Wu
I'm fine with the #dev channel. I remembered in the previous discussion that most people are positive about the dev channel as long as the discussions are properly reflected back to the mailing lists. If we create the dev channel, maybe we can merge #contribution-helps into it as well? Just

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Gyula Fóra
I agree with Robert. I think discussing implementation ideas etc on the dev channel briefly before posting discussions on the ML can make the design discussions much more productive as the initial iterations can be often slow and cumbersome via email alone. I understand the general sentiment

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Gyula Fóra
Hi Yang! +1 (binding) Successfully verified the following: - Signatures, Hashes - No binaries in source release - Helm Repo works, Helm install works, docker image matches release commit tag - Build from source, build container from source - Submit example application, session and session job

Re: [DISCUSS] Initializing Apache Flink Slack

2022-05-31 Thread Robert Metzger
Thanks for your input. I share the concern that we'll potentially have too important discussions in Slack. However, I would still like to try it out. There are valid use-cases, such as requests for VOTEs on releases, or briefly floating an idea before opening a thread on dev@. If we find out that

[DISCUSS] Bi-Weekly Flink Community Sync Meeting

2022-05-31 Thread Robert Metzger
Hi everyone, We currently have a bi-weekly release sync meeting on Google Meet every Tuesday at 9am CEST / 3pm China Standard Time / 7am UTC. I would like to propose extending the purpose of the meeting to a general "Flink Community Sync" meeting, to discuss current topics in the Flink community.

[jira] [Created] (FLINK-27850) Hive dialect supports "add file xxx"

2022-05-31 Thread luoyuxia (Jira)
luoyuxia created FLINK-27850: Summary: Hive dialect supports "add file xxx" Key: FLINK-27850 URL: https://issues.apache.org/jira/browse/FLINK-27850 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Planning Flink 1.16 with the community

2022-05-31 Thread Robert Metzger
> > Is the feature freeze deadline on July 25 fixed or will it be adjusted > accordingly? Ideally we try not to push the deadline too much into the future. So I prefer to consider it fixed. On Mon, May 30, 2022 at 7:08 PM Jing Ge wrote: > Thanks Robert for the reminder. Thanks Martijn for

Re: [DISCUSS] FLIP-228: Support Within between events in CEP Pattern

2022-05-31 Thread Nicholas Jiang
Hi Martijn, Sorry for later reply. This feature is only supported in DataStream and doesn't be supported in MATCH_RECOGNIZE because the SQL syntax of MATCH_RECOGNIZE does not contain the semantics of this feature, which requires modification of the SQL syntax. The support above MATCH_RECOGNIZE

[jira] [Created] (FLINK-27849) Harden correctness for non-deterministic updates present in the changelog pipeline

2022-05-31 Thread lincoln lee (Jira)
lincoln lee created FLINK-27849: --- Summary: Harden correctness for non-deterministic updates present in the changelog pipeline Key: FLINK-27849 URL: https://issues.apache.org/jira/browse/FLINK-27849

[jira] [Created] (FLINK-27848) ZooKeeperLeaderElectionDriver keeps writing leader information, using up zxid

2022-05-31 Thread Xintong Song (Jira)
Xintong Song created FLINK-27848: Summary: ZooKeeperLeaderElectionDriver keeps writing leader information, using up zxid Key: FLINK-27848 URL: https://issues.apache.org/jira/browse/FLINK-27848

Re: [DISCUSS] FLIP-228: Support Within between events in CEP Pattern

2022-05-31 Thread Nicholas Jiang
Hi Yue, Thanks for providing the benefit of this feature. After this feature is merged, you are welcome to try this feature in business scenarios. Regards, Nicholas Jiang On 2022/05/06 11:31:48 yue ma wrote: > hi Nicholas, > > Thanks for bringing this discussion, we also think it's a useful

Re: [DISCUSS] FLIP-228: Support Within between events in CEP Pattern

2022-05-31 Thread Nicholas Jiang
Hi Dian, Thanks for your feedback about supprting the within between events feature. I have updated the FLIP for introduction of 'Pattern#within(WithinType withInType, Time windowTime)' interface. Regarding your comments, I have the following thoughts: - Regarding the API, the name

Re: [VOTE] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-31 Thread Yun Gao
+1 (binding) Thanks Lincoln for driving the FLIP and discussion. Best, Yun Gao -- From:Jingsong Li Send Time:2022 May 31 (Tue.) 11:19 To:dev Subject:Re: [VOTE] FLIP-232: Add Retry Support For Async I/O In DataStream API +1

Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-05-31 Thread Yun Gao
Hi, Regarding the drop operation, with some offline discussion with Dalong and Zhu, we think that listening in the client side might be problematic since it would exit after submitting the jobs in detached mode, thus the operation might need to be in the JobMaster side. For the listener

Re: [DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-31 Thread Jingsong Li
Lincoln Thanks! Looks very good. Best, Jingsong On Tue, May 31, 2022 at 12:09 PM Lincoln Lee wrote: > Thanks Jingsong for the reminder! There is indeed some cleanup work to be > done, I'll update the doc today. > > Best, > Lincoln Lee > > > Jingsong Li 于2022年5月31日周二 11:29写道: > > > Hi

[VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #3

2022-05-31 Thread Yang Wang
Hi everyone, Please review and vote on the release candidate #3 for the version 1.0.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of