[jira] [Created] (FLINK-32090) Python API for enabling and configuring file merging snapshot

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32090: --- Summary: Python API for enabling and configuring file merging snapshot Key: FLINK-32090 URL: https://issues.apache.org/jira/browse/FLINK-32090 Project: Flink

[jira] [Created] (FLINK-32091) Add necessary metrics for file-merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32091: --- Summary: Add necessary metrics for file-merging Key: FLINK-32091 URL: https://issues.apache.org/jira/browse/FLINK-32091 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32089) Do fast copy in best-effort during first checkpoint after restoration

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32089: --- Summary: Do fast copy in best-effort during first checkpoint after restoration Key: FLINK-32089 URL: https://issues.apache.org/jira/browse/FLINK-32089 Project: Flink

[jira] [Created] (FLINK-32092) Integrate snapshot file-merging with existing IT cases

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32092: --- Summary: Integrate snapshot file-merging with existing IT cases Key: FLINK-32092 URL: https://issues.apache.org/jira/browse/FLINK-32092 Project: Flink Issue

[jira] [Created] (FLINK-32087) Space amplification statistics of file merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32087: --- Summary: Space amplification statistics of file merging Key: FLINK-32087 URL: https://issues.apache.org/jira/browse/FLINK-32087 Project: Flink Issue Type:

[jira] [Created] (FLINK-32088) Re-uploading in state file-merging for space amplification control

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32088: --- Summary: Re-uploading in state file-merging for space amplification control Key: FLINK-32088 URL: https://issues.apache.org/jira/browse/FLINK-32088 Project: Flink

[jira] [Created] (FLINK-32086) Cleanup non-reported managed directory on exit of TM

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32086: --- Summary: Cleanup non-reported managed directory on exit of TM Key: FLINK-32086 URL: https://issues.apache.org/jira/browse/FLINK-32086 Project: Flink Issue

[jira] [Created] (FLINK-32084) Migrate current file merging of channel state into the file merging framework

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32084: --- Summary: Migrate current file merging of channel state into the file merging framework Key: FLINK-32084 URL: https://issues.apache.org/jira/browse/FLINK-32084 Project:

[jira] [Created] (FLINK-32085) Implement and migrate batch uploading in changelog files into the file merging framework

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32085: --- Summary: Implement and migrate batch uploading in changelog files into the file merging framework Key: FLINK-32085 URL: https://issues.apache.org/jira/browse/FLINK-32085

[jira] [Created] (FLINK-32083) Chinese translation of documentation of checkpoint file-merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32083: --- Summary: Chinese translation of documentation of checkpoint file-merging Key: FLINK-32083 URL: https://issues.apache.org/jira/browse/FLINK-32083 Project: Flink

[jira] [Created] (FLINK-32082) Documentation of checkpoint file-merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32082: --- Summary: Documentation of checkpoint file-merging Key: FLINK-32082 URL: https://issues.apache.org/jira/browse/FLINK-32082 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32080) Restoration of FileMergingSnapshotManager

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32080: --- Summary: Restoration of FileMergingSnapshotManager Key: FLINK-32080 URL: https://issues.apache.org/jira/browse/FLINK-32080 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32081) Compatibility between file-merging on and off across job runs

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32081: --- Summary: Compatibility between file-merging on and off across job runs Key: FLINK-32081 URL: https://issues.apache.org/jira/browse/FLINK-32081 Project: Flink

[jira] [Created] (FLINK-32079) Read/write checkpoint metadata of merged files

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32079: --- Summary: Read/write checkpoint metadata of merged files Key: FLINK-32079 URL: https://issues.apache.org/jira/browse/FLINK-32079 Project: Flink Issue Type:

[jira] [Created] (FLINK-32078) Implement private state file merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32078: --- Summary: Implement private state file merging Key: FLINK-32078 URL: https://issues.apache.org/jira/browse/FLINK-32078 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32077) Implement shared state file merging

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32077: --- Summary: Implement shared state file merging Key: FLINK-32077 URL: https://issues.apache.org/jira/browse/FLINK-32077 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32076) Add file pool for concurrent file reusing

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32076: --- Summary: Add file pool for concurrent file reusing Key: FLINK-32076 URL: https://issues.apache.org/jira/browse/FLINK-32076 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32075) Delete merged files on checkpoint abort

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32075: --- Summary: Delete merged files on checkpoint abort Key: FLINK-32075 URL: https://issues.apache.org/jira/browse/FLINK-32075 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32074) Support file merging across checkpoints

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32074: --- Summary: Support file merging across checkpoints Key: FLINK-32074 URL: https://issues.apache.org/jira/browse/FLINK-32074 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32072) Create and wire FileMergingSnapshotManager with TaskManagerServices

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32072: --- Summary: Create and wire FileMergingSnapshotManager with TaskManagerServices Key: FLINK-32072 URL: https://issues.apache.org/jira/browse/FLINK-32072 Project: Flink

[jira] [Created] (FLINK-32073) Implement file merging in snapshot

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32073: --- Summary: Implement file merging in snapshot Key: FLINK-32073 URL: https://issues.apache.org/jira/browse/FLINK-32073 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32070) FLIP-306 Unified File Merging Mechanism for Checkpoints

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32070: --- Summary: FLIP-306 Unified File Merging Mechanism for Checkpoints Key: FLINK-32070 URL: https://issues.apache.org/jira/browse/FLINK-32070 Project: Flink Issue

[jira] [Created] (FLINK-32071) Implement the snapshot manager for merged checkpoint files in TM

2023-05-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-32071: --- Summary: Implement the snapshot manager for merged checkpoint files in TM Key: FLINK-32071 URL: https://issues.apache.org/jira/browse/FLINK-32071 Project: Flink

[RESULT][VOTE] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-12 Thread Zakelly Lan
Hi everyone, Happy to announce that FLIP-306[1] has been approved! According to the vote thread[2], there are 7 approving votes, out of which 5 are binding: * Yanfei Lei * Yuan Mei (binding) * Hangxiang Yu * Rui Fan (binding) * Yu Li (binding) * Yun Tang (binding) * Piotr Nowojski (binding) And

Re: [NOTICE] Flink master branch now uses Maven 3.8.6

2023-05-12 Thread David Anderson
Chesnay, thank you for all your hard work on this! David On Fri, May 12, 2023 at 4:03 PM Chesnay Schepler wrote: > > > What happened? > > I have just merged the last commits to properly support Maven 3.3+ on > the Flink master branch. > > mvnw and CI have been updated to use Maven 3.8.6.

Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers

2023-05-12 Thread Venkatakrishnan Sowrirajan
Thanks for the FLIP, Archit. +1 from me as well. This would be very useful for us and others in the community given the same issue was raised earlier as well. Regards Venkata krishnan On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > Thanks for the FLIP, Archit. > > The motivation sounds

Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers

2023-05-12 Thread Becket Qin
Thanks for the FLIP, Archit. The motivation sounds reasonable and it looks like a straightforward proposal. +1 from me. Thanks, Jiangjie (Becket) Qin On Fri, May 12, 2023 at 1:30 AM Archit Goyal wrote: > Hi all, > > I am opening this thread to discuss the proposal to support Yarn ACLs to >

[VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #2

2023-05-12 Thread Hao t Chang
Successfully built and deployed the bundle from the flink-kubernetes-operator-1.5.0-rc2 Steps: Create a fresh KinD cluster (v0.18.0 go1.20.2 linux/amd64) wget https://dist.apache.org/repos/dist/dev/flink/flink-kubernetes-operator-1.5.0-rc2/flink-kubernetes-operator-1.5.0-src.tgz tar xzvf

Re:Scala Compilation Errors with Flink code in IntelliJ. Builds fine with Maven command line.

2023-05-12 Thread Brandon Wright
Thank you so much Wencong Liu, that fixed it!!! Sent with Proton Mail secure email. --- Original Message --- On Thursday, May 11th, 2023 at 11:11 PM, Wencong Liu wrote: > > > Hi Brandon Wright, > > > I think you could try the following actions in IntelliJ IDE: > First, execute

[VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #2

2023-05-12 Thread Gyula Fóra
Hi Everyone, Please review and vote on the release candidate #2 for the version 1.5.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of

[jira] [Created] (FLINK-32069) jobClient.getJobStatus() can return status RUNNING for finished insert operation

2023-05-12 Thread Aleksandr Iushmanov (Jira)
Aleksandr Iushmanov created FLINK-32069: --- Summary: jobClient.getJobStatus() can return status RUNNING for finished insert operation Key: FLINK-32069 URL: https://issues.apache.org/jira/browse/FLINK-32069

[NOTICE] Flink master branch now uses Maven 3.8.6

2023-05-12 Thread Chesnay Schepler
What happened? I have just merged the last commits to properly support Maven 3.3+ on the Flink master branch. mvnw and CI have been updated to use Maven 3.8.6. What does this mean for me? * You can now use Maven versions beyond 3.2.5 (duh). o Most versions should work, but

[jira] [Created] (FLINK-32068) flink-connector-jdbc support clickhouse

2023-05-12 Thread leishuiyu (Jira)
leishuiyu created FLINK-32068: - Summary: flink-connector-jdbc support clickhouse Key: FLINK-32068 URL: https://issues.apache.org/jira/browse/FLINK-32068 Project: Flink Issue Type: New Feature

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #1

2023-05-12 Thread Gyula Fóra
Hi all! I am closing this vote now in light of the above mentioned blocker issue: https://issues.apache.org/jira/browse/FLINK-32067 The fix is on the way, I will create another RC later today. Cheers, Gyula On Fri, May 12, 2023 at 10:20 AM Gyula Fóra wrote: > The error only happens in cases

[jira] [Created] (FLINK-32067) When no pod template configured, an invalid null pod template is configured

2023-05-12 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-32067: -- Summary: When no pod template configured, an invalid null pod template is configured Key: FLINK-32067 URL: https://issues.apache.org/jira/browse/FLINK-32067 Project:

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #1

2023-05-12 Thread Gyula Fóra
The error only happens in cases when a podTemplate is not defined. The checkpointing/HA examples run well. Adding a simple (dummy) podtemplate to the basic.yaml makes it work: podTemplate: spec: containers: - name: flink-main-container I suspect it could be related to some

Re: [DISCUSS] Planning Flink 2.0

2023-05-12 Thread Xintong Song
Thanks all for the feedback. I'd like to provide some status updates for transparency. As previously discussed, we have assembled the release management team and have been discussing how to push things forward. We'd like to split this into several separate tracks, and prepare some draft

[jira] [Created] (FLINK-32066) Flink Ci service on Azure stops responding to pull requests

2023-05-12 Thread Wencong Liu (Jira)
Wencong Liu created FLINK-32066: --- Summary: Flink Ci service on Azure stops responding to pull requests Key: FLINK-32066 URL: https://issues.apache.org/jira/browse/FLINK-32066 Project: Flink

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #1

2023-05-12 Thread Gyula Fóra
Very strange, I also get an error when running the examples from the RC. It's different but could be related: Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://10.96.0.1/api/v1/namespaces/default/pods. Message: PodList in version "v1" cannot be

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #1

2023-05-12 Thread Márton Balassi
Hi Jim and Ted, Thanks for the quick response. For Openshift issue I would assume that adding the RBAC suggested here [1] would solve the problem, it seems fine to me. For the missing taskmanager could you please share the relevant logs from your jobmanager pod that is already show running?

[VOTE] Apache Flink Kubernetes Operator Release 1.5.0, release candidate #1

2023-05-12 Thread Hao t Chang
Seems missing taskmanager pod. I tried the following: kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml helm install 1.5rc1

[jira] [Created] (FLINK-32065) Got NoSuchFileException when initialize source function.

2023-05-12 Thread Spongebob (Jira)
Spongebob created FLINK-32065: - Summary: Got NoSuchFileException when initialize source function. Key: FLINK-32065 URL: https://issues.apache.org/jira/browse/FLINK-32065 Project: Flink Issue