[jira] [Created] (FLINK-26040) PrometheusReporterEndToEndITCase hang on azure pipeline

2022-02-08 Thread Yun Gao (Jira)
Yun Gao created FLINK-26040: --- Summary: PrometheusReporterEndToEndITCase hang on azure pipeline Key: FLINK-26040 URL: https://issues.apache.org/jira/browse/FLINK-26040 Project: Flink Issue Type:

[jira] [Created] (FLINK-26039) Incorrect value getter in map unnest table function

2022-02-08 Thread Han (Jira)
Han created FLINK-26039: --- Summary: Incorrect value getter in map unnest table function Key: FLINK-26039 URL: https://issues.apache.org/jira/browse/FLINK-26039 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-26038) Support delay message on Pulsar sink

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26038: --- Summary: Support delay message on Pulsar sink Key: FLINK-26038 URL: https://issues.apache.org/jira/browse/FLINK-26038 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-26037) TaskManagerRunnerTest JVM crash with exit code 239

2022-02-08 Thread Yun Gao (Jira)
Yun Gao created FLINK-26037: --- Summary: TaskManagerRunnerTest JVM crash with exit code 239 Key: FLINK-26037 URL: https://issues.apache.org/jira/browse/FLINK-26037 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-26036) LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory timeout on azure

2022-02-08 Thread Yun Gao (Jira)
Yun Gao created FLINK-26036: --- Summary: LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory timeout on azure Key: FLINK-26036 URL: https://issues.apache.org/jira/browse/FLINK-26036

[jira] [Created] (FLINK-26035) Rework loader-bundle into separate module

2022-02-08 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-26035: Summary: Rework loader-bundle into separate module Key: FLINK-26035 URL: https://issues.apache.org/jira/browse/FLINK-26035 Project: Flink Issue

[DISCUSS]Support the merge statement in FlinkSQL

2022-02-08 Thread zhou chao
Hi, devs! Jingfeng and I would like to start a discussion about the MERGE statement, and the discussion consists of two parts. In the first part, we want to explore and collect the cases and motivations of the MERGE statement users. In the second part, we want to find out the possibility for

[jira] [Created] (FLINK-26034) Add maven wapper for flink

2022-02-08 Thread Aiden Gong (Jira)
Aiden Gong created FLINK-26034: -- Summary: Add maven wapper for flink Key: FLINK-26034 URL: https://issues.apache.org/jira/browse/FLINK-26034 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread Gen Luo
Hi Chesney and Piotr, I have seen some jobs with tens of independent vertices that process data for the same business. The sub jobs should be started or stopped together. Splitting them into separate jobs means the user has to manage them separately. But in fact the jobs were running in per-job

[jira] [Created] (FLINK-26033) In KafkaConnector, when 'sink.partitioner' is configured as 'round-robin', it does not take effect

2022-02-08 Thread shizhengchao (Jira)
shizhengchao created FLINK-26033: Summary: In KafkaConnector, when 'sink.partitioner' is configured as 'round-robin', it does not take effect Key: FLINK-26033 URL:

[jira] [Created] (FLINK-26032) log job info in the ContextEnvironment

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26032: --- Summary: log job info in the ContextEnvironment Key: FLINK-26032 URL: https://issues.apache.org/jira/browse/FLINK-26032 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-26031) Support projection pushdown on keys and values in sst file readers

2022-02-08 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-26031: --- Summary: Support projection pushdown on keys and values in sst file readers Key: FLINK-26031 URL: https://issues.apache.org/jira/browse/FLINK-26031 Project: Flink

[jira] [Created] (FLINK-26030) Set FLINK_LIB_DIR to lib under working dir in YARN containers

2022-02-08 Thread Biao Geng (Jira)
Biao Geng created FLINK-26030: - Summary: Set FLINK_LIB_DIR to lib under working dir in YARN containers Key: FLINK-26030 URL: https://issues.apache.org/jira/browse/FLINK-26030 Project: Flink

[jira] [Created] (FLINK-26029) Generalize the checkpoint protocol of OperatorCoordinator.

2022-02-08 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-26029: Summary: Generalize the checkpoint protocol of OperatorCoordinator. Key: FLINK-26029 URL: https://issues.apache.org/jira/browse/FLINK-26029 Project: Flink

[DISCUSS]Support the merge statement in FlinkSQL

2022-02-08 Thread cao zou
Hi, devs! Jingfeng and I would like to start a discussion about the MERGE statement, and the discussion consists of two parts. In the first part, we want to explore and collect the cases and motivations of the MERGE statement users. In the second part, we want to find out the possibility for

[jira] [Created] (FLINK-26028) Write documentation for new PulsarSink

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26028: --- Summary: Write documentation for new PulsarSink Key: FLINK-26028 URL: https://issues.apache.org/jira/browse/FLINK-26028 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-26026) Test FLIP-191 PulsarSink

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26026: --- Summary: Test FLIP-191 PulsarSink Key: FLINK-26026 URL: https://issues.apache.org/jira/browse/FLINK-26026 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-26027) Add FLIP-33 metrics to new PulsarSink

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26027: --- Summary: Add FLIP-33 metrics to new PulsarSink Key: FLINK-26027 URL: https://issues.apache.org/jira/browse/FLINK-26027 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-26025) Replace MockPulsar with new Pulsar test tools based on PulsarStandalone

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26025: --- Summary: Replace MockPulsar with new Pulsar test tools based on PulsarStandalone Key: FLINK-26025 URL: https://issues.apache.org/jira/browse/FLINK-26025 Project: Flink

[jira] [Created] (FLINK-26024) Create a PulsarSerializationSchema for better records serialization

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26024: --- Summary: Create a PulsarSerializationSchema for better records serialization Key: FLINK-26024 URL: https://issues.apache.org/jira/browse/FLINK-26024 Project: Flink

[jira] [Created] (FLINK-26023) Create a Pulsar sink config model for matching ProducerConfigurationData

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26023: --- Summary: Create a Pulsar sink config model for matching ProducerConfigurationData Key: FLINK-26023 URL: https://issues.apache.org/jira/browse/FLINK-26023 Project:

[jira] [Created] (FLINK-26022) Implement at-least-once and exactly-once Pulsar Sink

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26022: --- Summary: Implement at-least-once and exactly-once Pulsar Sink Key: FLINK-26022 URL: https://issues.apache.org/jira/browse/FLINK-26022 Project: Flink Issue

[jira] [Created] (FLINK-26021) Pulsar topic deduplicated in both sink and source connector

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26021: --- Summary: Pulsar topic deduplicated in both sink and source connector Key: FLINK-26021 URL: https://issues.apache.org/jira/browse/FLINK-26021 Project: Flink

[jira] [Created] (FLINK-26020) Unified Pulsar Connector config model

2022-02-08 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-26020: --- Summary: Unified Pulsar Connector config model Key: FLINK-26020 URL: https://issues.apache.org/jira/browse/FLINK-26020 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-26019) Changelogged PriorityQueue elements recovered out-of-order

2022-02-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-26019: - Summary: Changelogged PriorityQueue elements recovered out-of-order Key: FLINK-26019 URL: https://issues.apache.org/jira/browse/FLINK-26019 Project: Flink

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-08 Thread Galen Warren
I'm ready to pick this one up, I have some code that's working locally. Shall I create a PR? On Wed, Feb 2, 2022 at 3:17 PM Igal Shilman wrote: > Great, ping me when you would like to pick this up. > > For the related issue, I think that can be a good addition indeed! > > On Wed, Feb 2, 2022

[jira] [Created] (FLINK-26018) Late events in the new KafkaSource

2022-02-08 Thread Jun Qin (Jira)
Jun Qin created FLINK-26018: --- Summary: Late events in the new KafkaSource Key: FLINK-26018 URL: https://issues.apache.org/jira/browse/FLINK-26018 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-26017) Add debug log message when marking a job result as dirty

2022-02-08 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-26017: - Summary: Add debug log message when marking a job result as dirty Key: FLINK-26017 URL: https://issues.apache.org/jira/browse/FLINK-26017 Project: Flink

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-08 Thread Gyula Fóra
I agree with flink-kubernetes-operator as the repo name :) Don't have any better idea Gyula On Sat, Feb 5, 2022 at 2:41 AM Thomas Weise wrote: > Hi, > > Thanks for the continued feedback and discussion. Looks like we are > ready to start a VOTE, I will initiate it shortly. > > In parallel it

[jira] [Created] (FLINK-26016) FileSystemLookupFunction does not produce correct results when hive table uses columnar storage

2022-02-08 Thread jinfeng (Jira)
jinfeng created FLINK-26016: --- Summary: FileSystemLookupFunction does not produce correct results when hive table uses columnar storage Key: FLINK-26016 URL: https://issues.apache.org/jira/browse/FLINK-26016

[jira] [Created] (FLINK-26015) FileSystemJobResultStore fails to access Minio

2022-02-08 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-26015: - Summary: FileSystemJobResultStore fails to access Minio Key: FLINK-26015 URL: https://issues.apache.org/jira/browse/FLINK-26015 Project: Flink Issue Type:

[jira] [Created] (FLINK-26014) Document how to use the working directory for faster local recoveries

2022-02-08 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-26014: - Summary: Document how to use the working directory for faster local recoveries Key: FLINK-26014 URL: https://issues.apache.org/jira/browse/FLINK-26014 Project:

[jira] [Created] (FLINK-26013) Develop ArchUnit test for Flink core and runtime

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26013: --- Summary: Develop ArchUnit test for Flink core and runtime Key: FLINK-26013 URL: https://issues.apache.org/jira/browse/FLINK-26013 Project: Flink Issue Type:

[jira] [Created] (FLINK-26012) Develop ArchUnit test for Flink table

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26012: --- Summary: Develop ArchUnit test for Flink table Key: FLINK-26012 URL: https://issues.apache.org/jira/browse/FLINK-26012 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-26011) Develop ArchUnit test for formats

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26011: --- Summary: Develop ArchUnit test for formats Key: FLINK-26011 URL: https://issues.apache.org/jira/browse/FLINK-26011 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-26010) Develop ArchUnit test for filesystems

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26010: --- Summary: Develop ArchUnit test for filesystems Key: FLINK-26010 URL: https://issues.apache.org/jira/browse/FLINK-26010 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-26009) ArchUnit for Flink

2022-02-08 Thread Jing Ge (Jira)
Jing Ge created FLINK-26009: --- Summary: ArchUnit for Flink Key: FLINK-26009 URL: https://issues.apache.org/jira/browse/FLINK-26009 Project: Flink Issue Type: Improvement Reporter: Jing

[jira] [Created] (FLINK-26008) [FLIP-171] Update Kinesalite docker container reference

2022-02-08 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-26008: - Summary: [FLIP-171] Update Kinesalite docker container reference Key: FLINK-26008 URL: https://issues.apache.org/jira/browse/FLINK-26008 Project: Flink

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread 丛鹏
hi guys,If I understand it correctly, will only some checkpoints be recovered when there is an error in the Flink batch? Piotr Nowojski 于2022年2月8日周二 19:05写道: > Hi, > > I second Chesnay's comment and would like to better understand the > motivation behind this. At the surface it sounds to me

[jira] [Created] (FLINK-26007) Use durationType instead of stringType for time related config options

2022-02-08 Thread Marios Trivyzas (Jira)
Marios Trivyzas created FLINK-26007: --- Summary: Use durationType instead of stringType for time related config options Key: FLINK-26007 URL: https://issues.apache.org/jira/browse/FLINK-26007

[jira] [Created] (FLINK-26006) KinesisFirehoseSinkITCase leaks resources

2022-02-08 Thread Jira
David Morávek created FLINK-26006: - Summary: KinesisFirehoseSinkITCase leaks resources Key: FLINK-26006 URL: https://issues.apache.org/jira/browse/FLINK-26006 Project: Flink Issue Type: Bug

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread Piotr Nowojski
Hi, I second Chesnay's comment and would like to better understand the motivation behind this. At the surface it sounds to me like this might require quite a bit of work for a very narrow use case. At the same time I have a feeling that Yuan, you are mixing this feature request (checkpointing

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread Chesnay Schepler
Could someone expand on these operational issues you're facing when achieving this via separate jobs? I feel like we're skipping a step, arguing about solutions without even having discussed the underlying problem. On 08/02/2022 11:25, Gen Luo wrote: Hi, @Yuan Do you mean that there should

[jira] [Created] (FLINK-26005) TableEnvironment.createTemporarySystemFunction cause NPE when using leftOuterLateralJoin

2022-02-08 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-26005: - Summary: TableEnvironment.createTemporarySystemFunction cause NPE when using leftOuterLateralJoin Key: FLINK-26005 URL: https://issues.apache.org/jira/browse/FLINK-26005

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread Gen Luo
Hi, @Yuan Do you mean that there should be no shared state between source subtasks? Sharing state between checkpoints of a specific subtask should be fine. Sharing state between subtasks of a task can be an issue, no matter whether it's a source. That's also what I was afraid of in the previous

[jira] [Created] (FLINK-26004) Introduce ForwardForLocalKeyByPartitioner

2022-02-08 Thread Lijie Wang (Jira)
Lijie Wang created FLINK-26004: -- Summary: Introduce ForwardForLocalKeyByPartitioner Key: FLINK-26004 URL: https://issues.apache.org/jira/browse/FLINK-26004 Project: Flink Issue Type: Sub-task

Re: 来自高大余的邮件

2022-02-08 Thread Caizhi Weng
Hi! Thanks for your interest in contributing to Flink! Currently there is no need (and actually there does not exist) a contributor permission. Feel free to open or reply to a JIRA ticket you're interested in, or to discuss in the mailing list with others. If you'd like to be assigned a JIRA

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-08 Thread Yuan Mei
Hey Folks, Thanks for the discussion! *Motiviation and use cases* I think motiviation and use cases are very clear and I do not have doubts on this part. A typical use case is ETL with two-phase-commit, hundreds of partitions can be blocked by a single straggler (a single task's checkpoint

来自高大余的邮件

2022-02-08 Thread 高大余
Hi Guys, I want to contribute to Apache Flink. Would you please give me the permission as a contributor? My JIRA ID is dygao.

[jira] [Created] (FLINK-26003) Use Jackson serialization for persisting TaskExecutor state to working directory

2022-02-08 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-26003: - Summary: Use Jackson serialization for persisting TaskExecutor state to working directory Key: FLINK-26003 URL: https://issues.apache.org/jira/browse/FLINK-26003

[jira] [Created] (FLINK-26002) Add test coverage for native format job upgrades

2022-02-08 Thread Anton Kalashnikov (Jira)
Anton Kalashnikov created FLINK-26002: - Summary: Add test coverage for native format job upgrades Key: FLINK-26002 URL: https://issues.apache.org/jira/browse/FLINK-26002 Project: Flink

[jira] [Created] (FLINK-26001) Implement ProjectableDecodingFormat for avro bulk format

2022-02-08 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-26001: --- Summary: Implement ProjectableDecodingFormat for avro bulk format Key: FLINK-26001 URL: https://issues.apache.org/jira/browse/FLINK-26001 Project: Flink Issue

Re: [RESULT][VOTE] FLIP-211: Kerberos delegation token framework

2022-02-08 Thread Gabor Somogyi
David, Thanks for making the design better! I count on you to pinpoint bugs as early as possible in the upcoming PRs. I'm just testing the first one on cluster, hope I can file it in the couple of days... BR, G On Tue, Feb 8, 2022 at 9:27 AM David Morávek wrote: > Thanks Gabor for driving

Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15

2022-02-08 Thread Konstantin Knauf
Thank you everyone, I've closed the vote and created a ticket for deprecation [1] and dropping [2] and linked the current blockers for dropping it to the latter. Please if or when you encounter new blockers link them to [2]. [1] https://issues.apache.org/jira/browse/FLINK-25999 [2]

Re: [RESULT][VOTE] FLIP-211: Kerberos delegation token framework

2022-02-08 Thread David Morávek
Thanks Gabor for driving this, I think the change is going to be really valuable for some of the enterprise users. Best, D. On Tue, Feb 8, 2022 at 8:33 AM Gabor Somogyi wrote: > Hi devs, > > FLIP-211 [1] Has been accepted. > There were 3 binding votes and 2 non-binding in favor. > None

[RESULT] [VOTE] Deprecate Per-Job Mode

2022-02-08 Thread Konstantin Knauf
Hi everyone, The vote on deprecating per-job mode in Flink 1.15 has been unanimously approved in [1]. I've created a ticket for deprecation [2] and dropping [3] and linked the current blockers for dropping it to the latter. Binding +1 Thomas Weise Xintong Song Yang Wang Jing Zhang Till Rohrmann

[jira] [Created] (FLINK-26000) Drop Per-Job Mode

2022-02-08 Thread Konstantin Knauf (Jira)
Konstantin Knauf created FLINK-26000: Summary: Drop Per-Job Mode Key: FLINK-26000 URL: https://issues.apache.org/jira/browse/FLINK-26000 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-25999) Deprecate Per-Job Mode

2022-02-08 Thread Konstantin Knauf (Jira)
Konstantin Knauf created FLINK-25999: Summary: Deprecate Per-Job Mode Key: FLINK-25999 URL: https://issues.apache.org/jira/browse/FLINK-25999 Project: Flink Issue Type: Improvement