Re: [DISCUSS] Releasing Flink ML 2.2.0

Thanks everyone for the comments! We will go ahead to release Flink ML 2.2.0. Please see here for the release plan. Best Regards, Dong On Fri, Mar 31, 2023 at 6:50 PM Yu Li wrote: > +1. Great to know the (exciting) progress

[jira] [Created] (FLINK-31718) pulsar connector v3.0 branch's CI is not working properly

Weijie Guo created FLINK-31718: -- Summary: pulsar connector v3.0 branch's CI is not working properly Key: FLINK-31718 URL: https://issues.apache.org/jira/browse/FLINK-31718 Project: Flink Issue

[jira] [Created] (FLINK-31717) Unit tests running with local kube config

Matyas Orhidi created FLINK-31717: - Summary: Unit tests running with local kube config Key: FLINK-31717 URL: https://issues.apache.org/jira/browse/FLINK-31717 Project: Flink Issue Type: New

RE: [VOTE] Release flink-connector-aws, 4.1.0 for Flink 1.17

+1 (non-binding) + verified hashes and signatures + checked local build of website pull request and approved On 2023/04/03 16:19:00 Danny Cranmer wrote: > Hi everyone, > Please review and vote on the binaries for flink-connector-aws version > 4.1.0-1.17, as follows: > [ ] +1, Approve the release

[jira] [Created] (FLINK-31716) Event UID field is missing the first time that an event is consumed

Rodrigo Meneses created FLINK-31716: --- Summary: Event UID field is missing the first time that an event is consumed Key: FLINK-31716 URL: https://issues.apache.org/jira/browse/FLINK-31716 Project:

TaskManager OutOfMemoryError. Metaspace out of memory with python udf

Hi Flink community, I am running a session cluster with 1gb of jvm metaspace. Each time I submit and cancel the flink job with python udf I am noticing that the metaspace is gradually increasing until it eventually kills the task manager due to an out of memory exception. To reproduce this error

[jira] [Created] (FLINK-31715) Warning - 'An illegal reflective access operation has occurred'

Feroze Daud created FLINK-31715: --- Summary: Warning - 'An illegal reflective access operation has occurred' Key: FLINK-31715 URL: https://issues.apache.org/jira/browse/FLINK-31715 Project: Flink

Re: [DISCUSS] Status of Statefun Project

Thanks for bringing this up. I'm currently using Statefun, and I've made a few small code contributions over time. All of my PRs have been merged into master and most have been released, but a few haven't been part of a release yet. Most recently, I helped upgrade Statefun to be compatible with

[VOTE] Release flink-connector-aws, 4.1.0 for Flink 1.17

Hi everyone, Please review and vote on the binaries for flink-connector-aws version 4.1.0-1.17, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The v4.1.0 source release has already been approved [1], this vote is to distribute the

Re: [DISCUSS] Releasing connectors for Flink 1.17

Hi everyone, I started the release process for flink-connector-aws and flink-connector-pulsar. Unfortunately the Pulsar connector does not build against Flink 1.17 at the v3.0.0 tag, errors below. The nightly is passing since the code has been updated to support 1.17. The PulsarFetcherManagerBase

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

Hi Timo, Thanks for your valuable feedback. Let me explain the design details. > However, actually fine-grained state TTL should already be possible today. I don't fully understand where your proposed StateMetadata is located? Would this be a value of @ExecNodeMetadata, StreamExecNode, or

Re: [DISCUSS] FLIP-306: Unified File Merging Mechanism for Checkpoints

Hi Zakelly, Thanks for driving this, this proposal enables the files merging of different types of states to be grouped under a unified framework. I think it has the added benefit of lightening the load on JM. As FLINK-26590[1] described, triggered checkpoints can be delayed by discarding

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

Hi Jane, Thanks for your clarification. +1 for using this feature for advanced use cases. Best regards, Jing On Mon, Apr 3, 2023 at 4:43 PM Jane Chan wrote: > Hi Yisha, > > Thanks for your detailed explanation. Here are my thoughts. > > > I'm interested in how to design a graphical interface

[jira] [Created] (FLINK-31714) Conjars.org has died

Niels Basjes created FLINK-31714: Summary: Conjars.org has died Key: FLINK-31714 URL: https://issues.apache.org/jira/browse/FLINK-31714 Project: Flink Issue Type: Bug Reporter:

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

Hi Yisha, Thanks for your detailed explanation. Here are my thoughts. > I'm interested in how to design a graphical interface to help users to maintain their custom fine-grained configuration between their job versions. Regarding the graphical IDE, this was mentioned to address the concern of

[jira] [Created] (FLINK-31713) k8s operator should gather job version metrics

Márton Balassi created FLINK-31713: -- Summary: k8s operator should gather job version metrics Key: FLINK-31713 URL: https://issues.apache.org/jira/browse/FLINK-31713 Project: Flink Issue

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

Hi Jane, thanks for proposing this FLIP. More state insights and fine-grained state TTL are a frequently requested feature for Flink SQL. Eventually, we need to address this. I agree with the previous responses that doing this with a hint might cause more confusion than it actually helps.

[ANNOUNCE] Starting with Flink 1.18 Release Sync

Hi everyone, As a fresh start of the Flink release 1.18, I'm happy to share with you that the first release sync meeting of 1.18 will happen tomorrow on Tuesday, April 4th at 10am (UTC+2) / 4pm (UTC+8). Welcome and feel free to join us and share your ideas about the new release cycle! Details of

[jira] [Created] (FLINK-31712) Allow skipping of archunit tests for nightly connector builds

Martijn Visser created FLINK-31712: -- Summary: Allow skipping of archunit tests for nightly connector builds Key: FLINK-31712 URL: https://issues.apache.org/jira/browse/FLINK-31712 Project: Flink

[jira] [Created] (FLINK-31711) OpenAPI spec omits complete-statement request body

Chesnay Schepler created FLINK-31711: Summary: OpenAPI spec omits complete-statement request body Key: FLINK-31711 URL: https://issues.apache.org/jira/browse/FLINK-31711 Project: Flink

[jira] [Created] (FLINK-31710) Remove and rely on the curator dependency that's provided by flink

Matthias Pohl created FLINK-31710: - Summary: Remove and rely on the curator dependency that's provided by flink Key: FLINK-31710 URL: https://issues.apache.org/jira/browse/FLINK-31710 Project: Flink

[jira] [Created] (FLINK-31709) JobResultStore and ExecutionGraphInfoStore could be merged

Matthias Pohl created FLINK-31709: - Summary: JobResultStore and ExecutionGraphInfoStore could be merged Key: FLINK-31709 URL: https://issues.apache.org/jira/browse/FLINK-31709 Project: Flink

[jira] [Created] (FLINK-31708) RuntimeException/KryoException thrown when deserializing an empty protobuf record

shen created FLINK-31708: Summary: RuntimeException/KryoException thrown when deserializing an empty protobuf record Key: FLINK-31708 URL: https://issues.apache.org/jira/browse/FLINK-31708 Project: Flink

[jira] [Created] (FLINK-31707) Constant string cannot be used as input arguments of Pandas UDAF

Dian Fu created FLINK-31707: --- Summary: Constant string cannot be used as input arguments of Pandas UDAF Key: FLINK-31707 URL: https://issues.apache.org/jira/browse/FLINK-31707 Project: Flink

[jira] [Created] (FLINK-31706) The default source parallelism should be the same as execution's default parallelism under adaptive batch scheduler

Yun Tang created FLINK-31706: Summary: The default source parallelism should be the same as execution's default parallelism under adaptive batch scheduler Key: FLINK-31706 URL:

[jira] [Created] (FLINK-31705) Remove Conjars

Martijn Visser created FLINK-31705: -- Summary: Remove Conjars Key: FLINK-31705 URL: https://issues.apache.org/jira/browse/FLINK-31705 Project: Flink Issue Type: Technical Debt

Re: [DISCUSS] Releasing connectors for Flink 1.17

Hi Denny, I could help with JDBC connectors. Thanks~ Best, Hang Danny Cranmer 于2023年4月3日周一 17:57写道： > Hi everyone, > > I have created Jiras to investigate each failing build. Thanks Andriy, I > have assigned the Elasticsearch/Opensearch ones to you. Let me know if you > do not have capacity.

[jira] [Created] (FLINK-31704) Pulsar docs should be pulled from dedicated branch

Danny Cranmer created FLINK-31704: - Summary: Pulsar docs should be pulled from dedicated branch Key: FLINK-31704 URL: https://issues.apache.org/jira/browse/FLINK-31704 Project: Flink Issue

[jira] [Created] (FLINK-31703) Update Flink docs for AWS v4.1.0

Danny Cranmer created FLINK-31703: - Summary: Update Flink docs for AWS v4.1.0 Key: FLINK-31703 URL: https://issues.apache.org/jira/browse/FLINK-31703 Project: Flink Issue Type: Improvement

[DISCUSS] FLIP-306: Unified File Merging Mechanism for Checkpoints

Hi everyone, I would like to open a discussion on providing a unified file merging mechanism for checkpoints[1]. Currently, many files are uploaded to the DFS during checkpoints, leading to the 'file flood' problem when running intensive workloads in a cluster. To tackle this problem, various

[jira] [Created] (FLINK-31702) Integrate Opensearch connector docs into Flink docs v1.17/master

Danny Cranmer created FLINK-31702: - Summary: Integrate Opensearch connector docs into Flink docs v1.17/master Key: FLINK-31702 URL: https://issues.apache.org/jira/browse/FLINK-31702 Project: Flink

Re: [DISCUSS] Releasing connectors for Flink 1.17

Hi everyone, I have created Jiras to investigate each failing build. Thanks Andriy, I have assigned the Elasticsearch/Opensearch ones to you. Let me know if you do not have capacity. - Elasticsearch: https://issues.apache.org/jira/browse/FLINK-31696 - Opensearch:

[jira] [Created] (FLINK-31701) RabbitMQ nightly CI failure

Danny Cranmer created FLINK-31701: - Summary: RabbitMQ nightly CI failure Key: FLINK-31701 URL: https://issues.apache.org/jira/browse/FLINK-31701 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-31700) MongoDB nightly CI failure

Danny Cranmer created FLINK-31700: - Summary: MongoDB nightly CI failure Key: FLINK-31700 URL: https://issues.apache.org/jira/browse/FLINK-31700 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-31699) JDBC nightly CI failure

Danny Cranmer created FLINK-31699: - Summary: JDBC nightly CI failure Key: FLINK-31699 URL: https://issues.apache.org/jira/browse/FLINK-31699 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-31698) Cassandra nightly CI failure

Danny Cranmer created FLINK-31698: - Summary: Cassandra nightly CI failure Key: FLINK-31698 URL: https://issues.apache.org/jira/browse/FLINK-31698 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-31697) OpenSearch nightly CI failure

Danny Cranmer created FLINK-31697: - Summary: OpenSearch nightly CI failure Key: FLINK-31697 URL: https://issues.apache.org/jira/browse/FLINK-31697 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-31696) ElasticSearch nightly CI failure

Danny Cranmer created FLINK-31696: - Summary: ElasticSearch nightly CI failure Key: FLINK-31696 URL: https://issues.apache.org/jira/browse/FLINK-31696 Project: Flink Issue Type: Technical

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

Hi João, Thanks for creating this FLIP! I'm overall +1 for it to unblock the migration of sinks to SinkV2. Yet I think it's better to let the `ReadableExecutionConfig` extend `ExecutionConfig`, because otherwise we have to introduce a new method

Re: [ANNOUNCE] Apache flink-connector-mongodb v1.0.0 released

Good to know & Congrats! Many thanks to Danny, Chesnay, Martijn and Ross! Best, Jiabao > 2023年4月3日 下午3:56，Danny Cranmer 写道： > > The Apache Flink community is very happy to announce the release of Apache > flink-connector-mongodb v1.0.0 > > Apache Flink® is an open-source stream processing

[jira] [Created] (FLINK-31695) Calling `bin/flink stop` when flink-shaded-hadoop-2-uber-2.8.3-10.0.jar is in lib directory thwos NoSuchMethodError

Caizhi Weng created FLINK-31695: --- Summary: Calling `bin/flink stop` when flink-shaded-hadoop-2-uber-2.8.3-10.0.jar is in lib directory thwos NoSuchMethodError Key: FLINK-31695 URL:

[DISCUSS] Status of Statefun Project

Hi everyone, I want to open a discussion on the status of the Statefun Project [1] in Apache Flink. As you might have noticed, there hasn't been much development over the past months in the Statefun repository [2]. There is currently a lack of active contributors and committers who are able to

Re: [blog article] Howto create a batch source with the new Source framework

Hi all, I just published the last article of the series about creating a batch source with the new Source framework. This one is about testing the source. Can you tell me if you think it would make sense to publish both articles to the official Flink blog as they could serve as a detailed

[jira] [Created] (FLINK-31694) Bump ua-parser-js from 0.7.31 to 0.7.33

Martijn Visser created FLINK-31694: -- Summary: Bump ua-parser-js from 0.7.31 to 0.7.33 Key: FLINK-31694 URL: https://issues.apache.org/jira/browse/FLINK-31694 Project: Flink Issue Type:

[jira] [Created] (FLINK-31693) Bump http-cache-semantics from 4.1.0 to 4.1.1 in

Martijn Visser created FLINK-31693: -- Summary: Bump http-cache-semantics from 4.1.0 to 4.1.1 in Key: FLINK-31693 URL: https://issues.apache.org/jira/browse/FLINK-31693 Project: Flink Issue

[DISCUSS] FLINK-31553

Hello to all, I'd like to discuss an issue with flink-jdbc-connector: https://issues.apache.org/jira/browse/FLINK-31553. Currently, the Dialect/Catalog/TypeMappers, etc. are chosen automatically based on the JDBC connection string prefix. For example, if `jdbc:postgresql://` is used, then the

[jira] [Created] (FLINK-31692) Integrate MongoDB connector docs into Flink website

Danny Cranmer created FLINK-31692: - Summary: Integrate MongoDB connector docs into Flink website Key: FLINK-31692 URL: https://issues.apache.org/jira/browse/FLINK-31692 Project: Flink Issue

[ANNOUNCE] Apache flink-connector-aws v4.1.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-aws v4.1.0 Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is available for download

[ANNOUNCE] Apache flink-connector-mongodb v1.0.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-mongodb v1.0.0 Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is available for

Re: [DISCUSS] FLIP 295: Support persistence of Catalog configuration and asynchronous registration

Hi everyone, Thank you all for your interest in this DISCUSS. @Shammon > How to handle a catalog with the same name that exists for both of them? I believe this is a crucial point. Based on my current personal understanding, the Map catalogs will serve as a cache for instantiated catalogs and

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

Hi Jane, Thanks for driving this FLIP. I think the compiled plan solution and the hint solution do not conflict, the two can exist at the same time. The compiled plan solution can address the need of advanced users and the platform users which all stateful operators' state TTL can be defined by

Re: [VOTE] Release flink-connector-kafka, release candidate #1

+1. Thanks, Gordon! Am Mo., 3. Apr. 2023 um 06:37 Uhr schrieb Tzu-Li (Gordon) Tai < tzuli...@apache.org>: > Hi Martijn, > > Since this RC vote was opened, we had three critical bug fixes that was > merged for the Kafka connector: > >- https://issues.apache.org/jira/browse/FLINK-31363 >-