Issue in seeing job manager and task manager logs in history sever

Hi Everyone, I am Sriram Ganesh part of the Microsoft Flink platform team. I recently went through the history server setup in Flink. I found an issue in the history server that was not showing the job manager and task manager logs. We are running clusters in session mode and using native

Re: [DISCUSS] FLIP-235: Hybrid Shuffle Mode

Hi All, Thank you for your attention and feedback. Do you have any other comments? If there are no other questions, I'll vote on FLIP-235 tomorrow. Best regards, Weijie Aitozi 于2022年5月20日周五 13:22写道： > Hi Xintong > Thanks for your detailed explanation, I misunderstand the spill > behavior

[jira] [Created] (FLINK-27761) Add database and table field in canal-json sink

Zhiwen Sun created FLINK-27761: -- Summary: Add database and table field in canal-json sink Key: FLINK-27761 URL: https://issues.apache.org/jira/browse/FLINK-27761 Project: Flink Issue Type:

[jira] [Created] (FLINK-27760) NPE is thrown when executing PyFlink jobs in batch mode

Dian Fu created FLINK-27760: --- Summary: NPE is thrown when executing PyFlink jobs in batch mode Key: FLINK-27760 URL: https://issues.apache.org/jira/browse/FLINK-27760 Project: Flink Issue Type:

Re: About Native Deployment's Autoscaling implementation

Thanks for the interesting discussion. Compared with reactive mode, leveraging the flink-kubernetes-operator to do the job restarting/upgrading is another solution for auto-scaling. Given that fully restarting a Flink application on K8s is not too slow, this is a reasonable way. Really hope we

Re:Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Martijn Thanks for your reply! I looked at the SQL standard, CTAS is part of the SQL standard. Feature T172 is "AS subquery clause in table definition". -- Best regards, Mang Zhang At 2022-05-04 21:49:00, "Martijn Visser" wrote: >Hi everyone, > >Can we identify if this proposed

[jira] [Created] (FLINK-27759) Rethink how to get the git commit id for docker image in Flink Kubernetes operator

Yang Wang created FLINK-27759: - Summary: Rethink how to get the git commit id for docker image in Flink Kubernetes operator Key: FLINK-27759 URL: https://issues.apache.org/jira/browse/FLINK-27759

Re: About Native Deployment's Autoscaling implementation

Hi Talat! It would be great to have a HPA that works based on some flink throughput/backlog metrics. I wonder how you are going to access the Flink metrics in the HPA, we might need some integration with the k8s metrics system. In any case whether we need a FLIP or not depends on the complexity,

[jira] [Created] (FLINK-27758) [JUnit5 Migration] Module: flink-table-runtime

Sergey Nuyanzin created FLINK-27758: --- Summary: [JUnit5 Migration] Module: flink-table-runtime Key: FLINK-27758 URL: https://issues.apache.org/jira/browse/FLINK-27758 Project: Flink Issue

Re: About Native Deployment's Autoscaling implementation

Hi Gyula, This seems very promising for initial scaling. We are using Flink Kubernetes Operators. Most probably we are very early adapters for it :) Let me try it. Get back to you soon. My plan is building a general purpose CPU and backlog/throughput base autoscaling for Flink. I can create a

[jira] [Created] (FLINK-27757) Elasticsearch connector should not use flink-table-planner but flink-table-planner-loader

Martijn Visser created FLINK-27757: -- Summary: Elasticsearch connector should not use flink-table-planner but flink-table-planner-loader Key: FLINK-27757 URL: https://issues.apache.org/jira/browse/FLINK-27757

Re: [DISCUSS] Docker images for SNAPSHOT

Hi everyone, Glad to inform you that the Github Actions workflow [1][2] that publishes SNAPSHOT Flink images to GHCR [3] is now up and running. Best, Alexander Fedulov [1] https://github.com/apache/flink-docker/pull/116/files [2] https://github.com/apache/flink-docker/actions/runs/2376596377

[GitHub] [flink-connector-rabbitmq] MartijnVisser commented on pull request #1: [FLINK-20628] RabbitMQ Connector using FLIP-27 Source API

MartijnVisser commented on PR #1: URL: https://github.com/apache/flink-connector-rabbitmq/pull/1#issuecomment-1135867243 @pscls I think you've done a good job already with the root-pom; it looks like the one we currently have for Elasticsearch. I've just approved the run, so we can also

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

Hi Lincoln, I have taken a look at FLIP-234, and I agree with you that the connectors can provide both async and sync runtime providers simultaneously instead of one of them. At that point, "lookup.async" looks redundant. If this option is planned to be removed in the long term, I think it makes

Re: Access to confluence

Hi Alexander, I have assigned the permission to you. Best, Jark On Mon, 23 May 2022 at 22:38, Александр Смирнов wrote: > Hi Jark and other PMC members! > > Сould you give me access to the editing of the confluence page with > FLIP-221 [1]? I would like to make small changes that were

Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi Mang and Godfrey, I'm okay with `Table#saveAs(tablePath)`, but it would be better to consider overriding table options and create or replace table. So we may also need to provide methods for the returned object: - #option(key, value) - #create() - #createIfNotExist() - #createOrReplace() //

[jira] [Created] (FLINK-27756) Fix Intermittingly failing test in `AsyncSinkWriterTest`

Ahmed Hamdy created FLINK-27756: --- Summary: Fix Intermittingly failing test in `AsyncSinkWriterTest` Key: FLINK-27756 URL: https://issues.apache.org/jira/browse/FLINK-27756 Project: Flink Issue

Re: All flink-ci fail

Hi Chesnay Schepler Thanks for your information. Best wishes fanrui On Tue, May 24, 2022 at 4:21 PM Chesnay Schepler wrote: > See https://issues.apache.org/jira/browse/FLINK-27751 > > On 24/05/2022 08:12, rui fan wrote: > > Hi masters, > > > > All compile_ci of flink-ci[1] failed today,

[jira] [Created] (FLINK-27755) Introduce a filesystem catalog for table store

Jingsong Lee created FLINK-27755: Summary: Introduce a filesystem catalog for table store Key: FLINK-27755 URL: https://issues.apache.org/jira/browse/FLINK-27755 Project: Flink Issue Type:

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #1

Hi Yang, Thanks for the work! I successfully verified these items: 1. Verify that the checksums and GPG files are intact 2. Verify that the source distributions do not contain any binaries 3. Build the source distribution to ensure all source files 4. Validate the Maven artifacts do not contain

[jira] [Created] (FLINK-27754) Getting Flink config fails if multiple files in /lib match flink-dist-*

Rudi Kershaw created FLINK-27754: Summary: Getting Flink config fails if multiple files in /lib match flink-dist-* Key: FLINK-27754 URL: https://issues.apache.org/jira/browse/FLINK-27754 Project:

Re: All flink-ci fail

See https://issues.apache.org/jira/browse/FLINK-27751 On 24/05/2022 08:12, rui fan wrote: Hi masters, All compile_ci of flink-ci[1] failed today, seemingly for the same reason: 05:03:16,982 ERROR org.apache.flink.tools.ci.suffixcheck.ScalaSuffixChecker [] - Violations found: Scala-free

[jira] [Created] (FLINK-27753) Scala suffix checks should check for maven errors

Chesnay Schepler created FLINK-27753: Summary: Scala suffix checks should check for maven errors Key: FLINK-27753 URL: https://issues.apache.org/jira/browse/FLINK-27753 Project: Flink

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #1

Thanks Gyula for sharing the feedback. I have created two tickets[1][2] to fix the problems and will integrate them in the next release candidate. I am not closing this vote and waiting for other verification comments. [1]. https://issues.apache.org/jira/browse/FLINK-27746 [2].

[jira] [Created] (FLINK-27752) Change 'path' to 'root-path' in table store

Jingsong Lee created FLINK-27752: Summary: Change 'path' to 'root-path' in table store Key: FLINK-27752 URL: https://issues.apache.org/jira/browse/FLINK-27752 Project: Flink Issue Type:

Re:Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Jark Thanks for your reply and suggest! >With this interface, we can easily support LEVEL-2 semantics by calling >`Catalog#dropTable` in the >`JobListener#onJobFailed`. We can also support LEVEL-3 by introducing >`StagingTableCatalog` like Spark, >calling `StagedTable#commitStagedChanges()` in

Re:Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Godfrey Thanks for your reply and suggest! I adjusted the style of FLIP. There are still two questions that need to be discussed: 1. The new API we introduced to CTAS, in Table or Tableenvalonment? 2. The Drop Table Operation in JM is everyone's consensus, more details design at runtime need

[jira] [Created] (FLINK-27751) Scala suffix check failed

Huang Xingbo created FLINK-27751: Summary: Scala suffix check failed Key: FLINK-27751 URL: https://issues.apache.org/jira/browse/FLINK-27751 Project: Flink Issue Type: Bug

[DISCUSS] Releasing Flink 1.14.5

Hi all, I would like to start discussing releasing Flink 1.14.5. It has already been more than two months since we released 1.14.4. There are currently 62 tickets[1] already resolved for 1.14.5, some of them quite severe. Currently, there are no issues marked as critical or blocker for 1.14.5.

All flink-ci fail

Hi masters, All compile_ci of flink-ci[1] failed today, seemingly for the same reason: 05:03:16,982 ERROR org.apache.flink.tools.ci.suffixcheck.ScalaSuffixChecker [] - Violations found: Scala-free module 'flink-hadoop-compatibility' is referenced with scala suffix in