Re: [DISCUSS] Dashboard/HistoryServer authentication

gt; should >> > be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or >> OAuth2 >> > been considered instead of Kerberos? How do you see the market moving? >> But >> > as I said before, in my opinion we can generate more value by investing >>

Re: Unsubscribe Dev-Mail

Hi, you have to send a mail to dev-unsubscr...@flink.apache.org in order to unsubscribe from Flink's dev ML. Cheers, Till On Wed, Jun 16, 2021 at 12:04 PM 雒正林 wrote: > Unsubscribe

Re: [ANNOUNCE] New PMC member: Arvid Heise

Congratulations, Arvid! Cheers, Till On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG wrote: > Congratulations, Arvid! > > Nicholas Jiang 于2021年6月16日周三 下午7:25写道： > > > Congratulations, Arvid! > > > > > > > > -- > > Sent from: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ > > >

Re: [ANNOUNCE] New PMC member: Xintong Song

Congratulations, Xintong! Cheers, Till On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG wrote: > Congratulations, Xintong! > > > Jiayi Liao 于2021年6月16日周三 下午7:30写道： > > > > > > > > > > Congratulations Xintong! > > > > > > On Wed, Jun 1

Re: [DISCUSS] Feedback Collection Jira Bot

cument a future effort? Maybe for the example of > DataStream replacing DataSet how would this look like in Jira? > > Cheers, > > Konstantin > > > On Tue, Jun 8, 2021 at 11:31 AM Till Rohrmann > wrote: > >> I like this idea. It would then be the responsibility of

Re: Apply for FLIP Wiki Page Edit Permission

Done. You should now have the access rights. Cheers, Till On Tue, Jun 15, 2021 at 10:18 AM Roc Marshal wrote: > Hi, > > > > > I want to contribute to Flink, but I can't create FLIP wiki page now. > Would someone please give me the confluence permission ? > > My Confluence ID is roc-marshal. Ful

Re: Add control mode for flink

Thanks for starting this discussion. I do see the benefit of dynamically configuring your Flink job and the cluster running it. Some of the use cases which were mentioned here are already possible. E.g. adjusting the log level dynamically can be done by configuring an appropriate logging backend an

Re: Flink TM Heartbeat Timeout

Hi Lu, longer heartbeat timeouts will have the effect that a loss of component (e.g. a TaskManager) will take longer to be detected. This will affect the recovery speed of your application in case of such a situation. On the upside, longer heartbeat timeouts allow working on less reliable infrastr

Re: [DISCUSS] Definition of idle partitions

Thanks for providing these details Gordon. I have to admit that I do not fully follow the reasoning why periodic watermark generators forced us to define idleness for records. Is it because the idleness was generated based on the non-availability of more data in the sources and not in the watermark

Re: [VOTE] Watermark propagation with Sink API

+1, Cheers, Till On Thu, Jun 10, 2021 at 4:59 AM Tzu-Li (Gordon) Tai wrote: > +1 > > On Thu, Jun 10, 2021 at 9:04 AM jincheng sun > wrote: > > > +1 (binding) // Sorry for the late reply. > > > > Best, > > Jincheng > > > > > > Piotr Nowojski 于2021年6月9日周三 下午10:04写道： > > > > > Thanks for drivin

Re: [DISCUSS] Dashboard/HistoryServer authentication

oxy implementation (Nginx and Squid) but according to > our analysis and testing it's not suitable for the mentioned use-cases. > Please take a look at the rejected alternatives for detailed explanation. > > Thanks for your time in advance! > > BR, > G > > > On Fr

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

precated // In the future we could remove this interface >> interface BoundedOneInput extends BoundedInput {} >> >> interface BoundedMultiInput extends BoundedInput { >> void endInput(int i); >> >> default void endInput() {} // For compatibility >>

Re: Flink 1.14. Bi-weekly 2021-06-08

Thanks for this update Joe, Dawid and Xintong. This is super helpful! Cheers, Till On Tue, Jun 8, 2021 at 4:18 PM Johannes Moser wrote: > Hi, > > Today we had our first bi-weekly. > Here’s a short summary of what has been discussed. > > Please watch the 1.14. Release page [1] to stay up to date

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

Great :-) On Tue, Jun 8, 2021 at 1:11 PM Yingjie Cao wrote: > Hi Till, > > Thanks for the suggestion. The blog post is already on the way. > > Best, > Yingjie > > Till Rohrmann 于2021年6月8日周二 下午5:30写道： > >> Thanks for the update Yingjie. Would it make sense to

Re: [DISCUSS] Watermark propagation with Sink API

Hi everyone, I do agree that Flink's definition of idleness is not fully thought through yet. Consequently, I would feel a bit uneasy to make it part of Flink's API right now. Instead, defining the proper semantics first and then exposing it sounds like a good approach forward. Hence, +1 for optio

Re: [DISCUSS] Limit size of already processed files in File Source SplitEnumerator

Hi Tianxin, thanks for starting this discussion. I am pulling in Arvid who works on Flink's connectors. I think the problem you are describing can happen. >From what I understand you are proposing to keep track of the watermark of processed file input splits and then filter out splits based on t

Re: [DISCUSS] Feedback Collection Jira Bot

omment on test instabilities? Would test > > > > instabilities always get a fixVersion then? > > > > > > > > Background: Test instabilities are supposed to be Critical. Critical > > > > tickets are deprioritized if they are unassigned and have not &g

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

Thanks for the update Yingjie. Would it make sense to write a short blog post about this feature including some performance improvement numbers? I think this could be interesting to our users. Cheers, Till On Mon, Jun 7, 2021 at 4:49 AM Jingsong Li wrote: > Thanks Yingjie for the great effort!

Re: [DISCUSS] Support specifying custom transactional.id prefix in FlinkKafkaProducer

Thanks for starting this discussion Wenhao. I've given you permission to create a FLIP. Cheers, Till On Sat, Jun 5, 2021 at 9:48 AM Wenhao Ji wrote: > Hi everyone, > > Currently, the "transactional.id"s of the Kafka producers in > FlinkKafkaProducer are generated based on the task name. This me

Re: [DISCUSS] Dashboard/HistoryServer authentication

It would increase the code footprint but this is true for many features > (as a side note I'm here to maintain it) > > Feel free to add your points because it only represents a single viewpoint. > Also if you have any better option for strong authentication please share > it an

Re: [DISCUSS] Dashboard/HistoryServer authentication

n consider sidecar proxies etc if we can solve it in such a > simple way as proposed by G. > > Cheers > Gyula > > On Fri, 4 Jun 2021 at 10:03, Till Rohrmann wrote: > >> I am not saying that we shouldn't add a strong authentication mechanism >> if there a

Re: [DISCUSS] Dashboard/HistoryServer authentication

wn challenges too. Thanks for bringing it up. > > Happy to have this added to the rejected alternative list so that we have > the full picture documented. > > On Thu, Jun 3, 2021 at 5:52 PM Till Rohrmann wrote: > >> I guess the idea would then be to let the proxy do the authenti

Re: [DISCUSS] Dashboard/HistoryServer authentication

ly. > > BR, > G > > > On Thu, Jun 3, 2021 at 12:49 PM Till Rohrmann > wrote: > >> Thanks for the information Gabor. If it is about securing the >> communication between the REST client and the REST server, then Flink >> already supports enabling mutual S

Re: [DISCUSS] Dashboard/HistoryServer authentication

e users who has > hard requirements on this. They're doing security audits and if they fail > then it's deal breaking. > That is why we have added kerberos at the first place. Unfortunately we > can't name them in this public list, however > the customers who specifical

Re: [Discuss] Planning Flink 1.14

Thanks for volunteering as our release managers Xintong, Dawid and Joe! Thanks for starting the discussion about the release date Kurt. Personally, I prefer in general shorter release cycles as it allows us to deliver features faster and people feel less pressure to merge half-done features last m

Re: recover from svaepoint

tead of > >> this.producerIdAndEpoch = ProducerIdAndEpoch.NONE; > > > On Thu, Jun 3, 2021 at 12:11 AM Till Rohrmann > wrote: > >> Thanks for the update. Skimming over the code it looks indeed that we are >> overwriting the values of the static value ProducerIdAndEpoch.NONE.

Re: [DISCUSS] Dashboard/HistoryServer authentication

edit >> >> >> >> On Tue, Jun 1, 2021 at 11:47 AM Chesnay Schepler >> wrote: >> >>> There's a related effort: >>> https://issues.apache.org/jira/browse/FLINK-21108 >>> >>> On 6/1/2021 10:14 AM, Till Rohrmann wrote: >

Re: recover from svaepoint

more familiar could double check this part of the code. Concerning the required changing of the UID of an operator Piotr, is this a known issue and documented somewhere? I find this rather surprising from a user's point of view. Cheers, Till On Thu, Jun 3, 2021 at 8:53 AM Till Rohrmann

Re: recover from svaepoint

enqueueRequest(handler); return handler.result; }, State.INITIALIZING); } On Wed, Jun 2, 2021 at 3:55 PM Piotr Nowojski wrote: > Hi, > > I think there is no generic way. If this error has happened indeed after > starting a second job from the same savepoint, or som

Re: recover from svaepoint

The error message says that we are trying to reuse a transaction id that is currently being used or has expired. I am not 100% sure how this can happen. My suspicion is that you have resumed a job multiple times from the same savepoint. Have you checked that there is no other job which has been re

Re: Getting Abstract method Error with flink 1.13.0 and 1.12.0

Hi Dipanjan, this type of question is best sent to Flink's user mailing list because there are a lot more people using Flink who could help you. The dev mailing list is intended to be used for development discussions. Cheers, Till On Tue, Jun 1, 2021 at 1:31 PM Dipanjan Mazumder wrote: > Hi ,

Re: The partition tracker should support remote shuffle properly

Hi Jin, thanks for starting this discussion and the initiative to implement a remote shuffle service. It has always been the idea of the ShuffleService abstraction to make this possible and we probably have overlooked some details. What I have understood from your description, you would like to i

Re: How to use or configure flink checkpointing with siddhi internal state

Hi Dipanjan, I am assuming that you are using the flink-siddhi library [1]. I am not an expert but it looks as if the AbstractSiddhiOperator overrides the snapshotState [2] method to store the Siddhi state in Flink. [1] https://github.com/haoch/flink-siddhi [2] https://github.com/haoch/flink-sidd

Re: [DISCUSS] Dashboard/HistoryServer authentication

Hi Gabor, welcome to the Flink community! Thanks for sharing this proposal with the community Márton. In general, I agree that authentication is missing and that this is required for using Flink within an enterprise. The thing I am wondering is whether this feature strictly needs to be implemented

Re: [ANNOUNCE] Apache Flink 1.13.1 released

Thanks for the great work Dawid and to everyone who has contributed to this release. Cheers, Till On Mon, May 31, 2021 at 10:25 AM Yangze Guo wrote: > Thanks, Dawid for the great work, thanks to everyone involved. > > Best, > Yangze Guo > > On Mon, May 31, 2021 at 4:14 PM Youngwoo Kim (김영우) >

Re: [DISCUSS] Moving to JUnit5

Thanks for joining the discussion Qingsheng. In general, I am not opposed to upgrading our testing library to JUnit 5. Also, the idea of starting with individual modules and do it incrementally sounds reasonable. However, before starting to do it like this, the community should agree that we want

[jira] [Created] (FLINK-22745) MesosWorkerStore is started with an illegal namespace

Till Rohrmann created FLINK-22745: - Summary: MesosWorkerStore is started with an illegal namespace Key: FLINK-22745 URL: https://issues.apache.org/jira/browse/FLINK-22745 Project: Flink

Re: Issue with using siddhi extension function with flink

Hi Dipanjan, Please double check whether the libraries are really contained in the job jar you are submitting because if the library is contained in this jar, then it should be on the classpath and you should be able to load it. Cheers, Till On Thu, May 20, 2021 at 3:43 PM Dipanjan Mazumder wro

[jira] [Created] (FLINK-22721) Breaking HighAvailabilityServices interface by adding new method

Till Rohrmann created FLINK-22721: - Summary: Breaking HighAvailabilityServices interface by adding new method Key: FLINK-22721 URL: https://issues.apache.org/jira/browse/FLINK-22721 Project: Flink

Re: [DISCUSS] Feedback Collection Jira Bot

I like this idea. +1 for your proposal Konstantin. Cheers, Till On Wed, May 19, 2021 at 1:30 PM Konstantin Knauf wrote: > Hi everyone, > > Till and I recently discussed whether we should disable the > "stale-blocker", "stale-critical", "stale-major" and "stale-minor" rules > for tickets that ha

Re: [DISCUSS] Component labels in PR/commit messages

I think a big problem with the component labels is that there is a) no defined set of labels b) no way to enforce the usage of them c) no easy way to figure out which label to use Due to these problems they are used very inconsistently in the project. I do agree with Arvid's observation that the

Re: Status of a savepoint operation returns Completed but an error was thrown

Hi Diogo, the idea is that a savepoint operation can also fail. The status only denotes whether the savepoint operation is still in-progress or completed because it is an asynchronous operation. A savepoint operation can be completed if it succeeded or if it failed. The failure cause should tell y

[jira] [Created] (FLINK-22636) Group job specific ZooKeeper HA services under common jobs/ zNode

Till Rohrmann created FLINK-22636: - Summary: Group job specific ZooKeeper HA services under common jobs/ zNode Key: FLINK-22636 URL: https://issues.apache.org/jira/browse/FLINK-22636 Project: Flink

Re: [DISCUSS] Releasing Flink 1.12.4

+1, thanks Arvid. Cheers, Till On Mon, May 10, 2021 at 5:08 AM Thomas Weise wrote: > +1 > > On Fri, May 7, 2021 at 7:00 AM Konstantin Knauf wrote: > > > +1. Thanks Arvid. > > > > On Fri, May 7, 2021 at 3:56 PM Chesnay Schepler > > wrote: > > > > > +1 for a releasing 1.12.4 ASAP. > > > > > > O

Re: Task Local Recovery with mountable disks in the cloud

t; ticket do? > > Thanks, > Sonam > -- > *From:* Till Rohrmann > *Sent:* Monday, April 26, 2021 10:24 AM > *To:* dev > *Cc:* u...@flink.apache.org ; Sonam Mandal < > soman...@linkedin.com> > *Subject:* Re: Task Local Recovery with mou

[jira] [Created] (FLINK-22597) JobMaster cannot be restarted

Till Rohrmann created FLINK-22597: - Summary: JobMaster cannot be restarted Key: FLINK-22597 URL: https://issues.apache.org/jira/browse/FLINK-22597 Project: Flink Issue Type: Bug

Re: Configmaps not deleted Kubernetes HA

Hi Enrique, I think you are actually seeing a mixture of FLINK-20219 and FLINK-20695. If any of these problems is solved, then the problem should be gone. Also note that the K8s HA services won't clean up the ConfigMaps if you delete the deployment as documented here [1]. [1] https://ci.apache.or

Re: Configmaps not deleted Kubernetes HA

Hi Enrique, I think you are running into this problem FLINK-20695 [1]. In a nutshell, Flink only deletes the config maps when it shuts down at the moment. We want to change this with the next release. [1] https://issues.apache.org/jira/browse/FLINK-20695 Cheers, Till On Wed, May 5, 2021 at 8:3

[jira] [Created] (FLINK-22576) Jira bot does not set the status to Open if it unassigns people

Till Rohrmann created FLINK-22576: - Summary: Jira bot does not set the status to Open if it unassigns people Key: FLINK-22576 URL: https://issues.apache.org/jira/browse/FLINK-22576 Project: Flink

Re: Presence of Jars in Flink reg security

Hi Prasanna, in the latest Flink version (1.13.0) I couldn't find these dependencies. Which version of Flink are you looking at? What you could check is whether one of these dependencies is contained in one of Flink's shaded dependencies [1]. [1] https://github.com/apache/flink-shaded Cheers, Ti

Re: [DISCUSS] Using timeouts in JUnit tests

r 29, 2021 at 4:51 PM Till Rohrmann > wrote: > > > I think for the maven tests we use this log4j.properties file [1]. > > > > [1] > https://github.com/apache/flink/blob/master/tools/ci/log4j.properties > > > > Cheers, > > Till > > > > On W

Re: [ANNOUNCE] Apache Flink 1.12.3 released

Great to hear. Thanks a lot for being our release manager Arvid and to everyone who has contributed to this release! Cheers, Till On Thu, Apr 29, 2021 at 4:11 PM Arvid Heise wrote: > Dear all, > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.12.3, which i

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

The reason why this feature is not part of Flink is that it is not trivial to solve. There are definitely different solutions to the problem. I think that a solution will probably be built around Flink and not integrated into it because it goes a bit beyond what Flink's current focus is. I can tell

[jira] [Created] (FLINK-22528) Document latency tracking metrics for state accesses

Till Rohrmann created FLINK-22528: - Summary: Document latency tracking metrics for state accesses Key: FLINK-22528 URL: https://issues.apache.org/jira/browse/FLINK-22528 Project: Flink Issue

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

Hi Bhagi, out of the box, Flink does not support this functionality. What you could try to do is to configure a stretch K8s cluster running on different DCs and then having an HA service (ZK or K8s) and blob storage being able to survive a DC outage. That way Flink should also be able to survive a

Re: [DISCUSS] Using timeouts in JUnit tests

> > for UnalignedCheckpointRescaleITCase, we test all known partitioners in > one > > pipeline for correctness. For higher parallelism, that means the test > runs > > over 1 minute regularly. If there is a global limit, I'd need to split > the > > test into sm

[jira] [Created] (FLINK-22502) DefaultCompletedCheckpointStore drops unrecoverable checkpoints silently

Till Rohrmann created FLINK-22502: - Summary: DefaultCompletedCheckpointStore drops unrecoverable checkpoints silently Key: FLINK-22502 URL: https://issues.apache.org/jira/browse/FLINK-22502 Project

[jira] [Created] (FLINK-22495) Document how to use the reactive mode on K8s

Till Rohrmann created FLINK-22495: - Summary: Document how to use the reactive mode on K8s Key: FLINK-22495 URL: https://issues.apache.org/jira/browse/FLINK-22495 Project: Flink Issue Type

Re: [DISCUSS] Using timeouts in JUnit tests

I think we do capture the INFO logs of the test runs on AZP. I am also not sure whether we really caught slow tests with Junit's timeout rule before. I think the default is usually to increase the timeout to make the test pass. One way to find slow tests is to measure the time and look at the outl

Re: [DISCUSS] Using timeouts in JUnit tests

that module), instead of having the test stall at a random > test. > > > On Mon, Apr 26, 2021 at 10:49 AM Till Rohrmann > wrote: > > > +1. I think this rule makes a lot of sense. > > > > Cheers, > > Till > > > > On Mon, Apr 26, 2021 at 10:08 AM Ar

Re: Task Local Recovery with mountable disks in the cloud

Hi Sonam, sorry for the late reply. We were a bit caught in the midst of the feature freeze for the next major Flink release. In general, I think it is a very good idea to disaggregate the local state storage to make it reusable across TaskManager failures. However, it is also not trivial to do.

[jira] [Created] (FLINK-22477) Remove DefaultSlotPoolServiceFactory and SlotPoolImpl

Till Rohrmann created FLINK-22477: - Summary: Remove DefaultSlotPoolServiceFactory and SlotPoolImpl Key: FLINK-22477 URL: https://issues.apache.org/jira/browse/FLINK-22477 Project: Flink

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

I should always be the first choice. > Could you help me to understand why the FlinkYarnSessionCli can be > activated? > > > Best, > Yangze Guo > > On Mon, Apr 26, 2021 at 4:48 PM Till Rohrmann > wrote: > > > > Hi Tony, > > > > I think you are ri

Re: [DISCUSS] Using timeouts in JUnit tests

+1. I think this rule makes a lot of sense. Cheers, Till On Mon, Apr 26, 2021 at 10:08 AM Arvid Heise wrote: > +1 from my side. > > We should probably double-check if we really need 4h timeouts on test tasks > in AZP. It feels like 2h be enough. > > On Mon, Apr 26, 2021 at 9:54 AM Dawid Wysakow

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

Hi Tony, I think you are right that Flink's cli does not behave super consistent at the moment. Case 2. should definitely work because `-t yarn-application` should overwrite what is defined in the Flink configuration. The problem seems to be that we don't resolve the configuration wrt the specifie

[jira] [Created] (FLINK-22474) Display job configuration in web-ui

Till Rohrmann created FLINK-22474: - Summary: Display job configuration in web-ui Key: FLINK-22474 URL: https://issues.apache.org/jira/browse/FLINK-22474 Project: Flink Issue Type

[jira] [Created] (FLINK-22431) AdaptiveScheduler does not log failure cause when recovering

Till Rohrmann created FLINK-22431: - Summary: AdaptiveScheduler does not log failure cause when recovering Key: FLINK-22431 URL: https://issues.apache.org/jira/browse/FLINK-22431 Project: Flink

[jira] [Created] (FLINK-22417) OrcFileSystemITCase.testOrcFilterPushDown fails on AZP

Till Rohrmann created FLINK-22417: - Summary: OrcFileSystemITCase.testOrcFilterPushDown fails on AZP Key: FLINK-22417 URL: https://issues.apache.org/jira/browse/FLINK-22417 Project: Flink

Re: Re: Re: [ANNOUNCE] New Apache Flink Committer - Rui Li

Congratulations Rui! Cheers, Till On Thu, Apr 22, 2021 at 4:10 PM 刘大龙 wrote: > Congratulations Rui! > > Best > > > -原始邮件- > > 发件人: "Benchao Li" > > 发送时间: 2021-04-22 14:43:33 (星期四) > > 收件人: dev > > 抄送: > > 主题: Re: Re: [ANNOUNCE] New Apache Flink Committer - Rui Li > > > > Congratulatio

Re: [jira] [Created] (FLINK-21986) taskmanager native memory not release timely after restart

Thanks for raising this issue Feifan. I think it is very important to fix it. I will take a look at your PR. Cheers, Till On Tue, Apr 20, 2021 at 5:35 AM zoltar9264 wrote: > > > Hi community, > I raised this issue about three weeks ago. After several weeks of > investigation, I found th

Re: confluence permission apply

Hi Roc, I have given you access to Flink's Confluence. Cheers, Till On Fri, Apr 16, 2021 at 4:18 PM Roc Marshal wrote: > Hi, > > > > > I want to contribute to Flink. Would someone please give me the confluence > permission ? > > My Confluence ID is roc-marshal. Full name is RocMarshal. > > My

Re: [DISCUSS] Releasing Flink 1.12.3

+1. Thanks for volunteering Arvid. Cheers, Till On Fri, Apr 16, 2021 at 9:50 AM Stephan Ewen wrote: > +1 > > Thanks for pushing this, Arvid, let's get this fix out asap. > > > > On Fri, Apr 16, 2021 at 9:46 AM Arvid Heise wrote: > > > Dear devs, > > > > Since we just fixed a severe bug that c

[jira] [Created] (FLINK-22306) KafkaITCase.testCollectingSchema failed on AZP

Till Rohrmann created FLINK-22306: - Summary: KafkaITCase.testCollectingSchema failed on AZP Key: FLINK-22306 URL: https://issues.apache.org/jira/browse/FLINK-22306 Project: Flink Issue Type

[jira] [Created] (FLINK-22295) Document sort-merge blocking shuffle

Till Rohrmann created FLINK-22295: - Summary: Document sort-merge blocking shuffle Key: FLINK-22295 URL: https://issues.apache.org/jira/browse/FLINK-22295 Project: Flink Issue Type: Sub-task

Re: [VOTE] Deprecating Mesos support

+1 Cheers, Till On Wed, Apr 14, 2021 at 10:10 AM Konstantin Knauf wrote: > +1 > > On Wed, Apr 14, 2021 at 9:46 AM Matthias Pohl > wrote: > > > Hi everyone, > > After the discussion of deprecating the Mesos support went on for a while > > now in [1] and considering that we already plan to retir

Re: Using FileSystem SQL Connector to Commit Hive Partition Exception

aYg&f=Images.zip&sz=42860073>Hi, > sorry for the debug log, I can take the photos instead of the log text > because of company policy, so if there is any log not clear, please tell > me. Thanks. > > > > > > > > > > On Apr 12, 2021, at 16:06, Till Rohrmann

Re: Add jobId and JobName variable to JobManager metrics in per-job deployment mode

Hi Lu, thanks for reaching out to the dev community. I would suggest keeping the discussion on the JIRA ticket. Cheers, Till On Tue, Apr 13, 2021 at 11:59 PM Lu Niu wrote: > Hi, Flink dev > > Could you share your thoughts about > https://issues.apache.org/jira/browse/FLINK-22164 ? > > context:

[jira] [Created] (FLINK-22255) AdaptiveScheduler improvements

Till Rohrmann created FLINK-22255: - Summary: AdaptiveScheduler improvements Key: FLINK-22255 URL: https://issues.apache.org/jira/browse/FLINK-22255 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-22254) Only trigger scale up if the resources have stabilized

Till Rohrmann created FLINK-22254: - Summary: Only trigger scale up if the resources have stabilized Key: FLINK-22254 URL: https://issues.apache.org/jira/browse/FLINK-22254 Project: Flink

Re: [DISCUSS] Flink should rely on flink-shaded SNAPSHOT artifacts

Thanks for creating this proposal Chesnay. I do understand the problem you want to fix. What I am wondering is why we don't release flink-shaded more often. Does the release process cause too much overhead? If this is the case, then we could look into what is causing the overhead and whether we ca

Re: handle SUSPENDED in ZooKeeperLeaderRetrievalService

to complete this feature. [1] https://issues.apache.org/jira/browse/FLINK-10052 Cheers, Till On Tue, Apr 13, 2021 at 8:25 AM Yang Wang wrote: > This might be related with FLINK-10052[1]. > Unfortunately, we do not have any progress on this ticket. > > cc @Till Rohrmann >

Re: Using FileSystem SQL Connector to Commit Hive Partition Exception

Hi Chenzuoli, the exception says that Flink could not allocate enough slots. Could you share the DEBUG logs of the run with us. They should contain the reason why the allocation of further resources failed. Cheers, Till On Sun, Apr 11, 2021 at 5:59 AM chenzuoli wrote: > > https://ci.apache.org

Re: Dev process: testing one suite only (and are our docs suggesting Java 8 accurate?)

min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 02:45 min > [INFO] Finished at: 2021-04-09T10:30:51+01:00 > [INFO] > ---

Re: Dev process: testing one suite only (and are our docs suggesting Java 8 accurate?)

Hi Adam, what works for me to run a single/set of tests is to use mvn verify -pl flink-runtime -Dtest='JobMaster*' -DfailIfNoTests=false -am I will add it to the wiki. Concerning FLINK-21672, I think it would be really great to not use vendor specific classes if possible. If you find a solution

[jira] [Created] (FLINK-22144) Test display last n exceptions/causes for job restarts in Web UI

Till Rohrmann created FLINK-22144: - Summary: Test display last n exceptions/causes for job restarts in Web UI Key: FLINK-22144 URL: https://issues.apache.org/jira/browse/FLINK-22144 Project: Flink

[jira] [Created] (FLINK-22142) Remove console logging for Kafka connector for AZP runs

Till Rohrmann created FLINK-22142: - Summary: Remove console logging for Kafka connector for AZP runs Key: FLINK-22142 URL: https://issues.apache.org/jira/browse/FLINK-22142 Project: Flink

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

Hi Becket, If I remember correctly, then we deliberately not documented the Kafka connector in the 1.12 release. Hence, from this point there should be no need to backport any fixes because users are not aware of this feature. On the other hand this also means that we should be able to break anyt

[jira] [Created] (FLINK-22135) Test the adaptive scheduler

Till Rohrmann created FLINK-22135: - Summary: Test the adaptive scheduler Key: FLINK-22135 URL: https://issues.apache.org/jira/browse/FLINK-22135 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-22134) Test the reactive mode

Till Rohrmann created FLINK-22134: - Summary: Test the reactive mode Key: FLINK-22134 URL: https://issues.apache.org/jira/browse/FLINK-22134 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-22067) SavepointWindowReaderITCase.testApplyEvictorWindowStateReader

Till Rohrmann created FLINK-22067: - Summary: SavepointWindowReaderITCase.testApplyEvictorWindowStateReader Key: FLINK-22067 URL: https://issues.apache.org/jira/browse/FLINK-22067 Project: Flink

Re: Flink job cannot find recover path after using entropy injection for s3 file systems

Thanks for creating this PR. I think it would be good to re-open the issue and post your analysis there together with the proposal for the fix. Cheers, Till On Wed, Mar 31, 2021 at 3:41 AM chenqin wrote: > link fix pr here https://github.com/apache/flink/pull/15442 > we might need someone help

Re: flink 1.11 class loading question

Hi Chen, with version 1.10 Flink introduced that RocksDB uses Flink's managed memory [1]. This shall prevent RocksDB from exceeding the memory limits of a process/container. Unfortunately, this is not yet perfect due to a problem in RocksDB [2]. Due to this fact, RocksDB can still exceed the manag

[jira] [Created] (FLINK-22051) Better document the distinction between stop-with-savepoint and stop-with-savepoint-with-drain

Till Rohrmann created FLINK-22051: - Summary: Better document the distinction between stop-with-savepoint and stop-with-savepoint-with-drain Key: FLINK-22051 URL: https://issues.apache.org/jira/browse/FLINK-22051

[jira] [Created] (FLINK-22048) Remove unnecessary akka.transport.* configuration options

Till Rohrmann created FLINK-22048: - Summary: Remove unnecessary akka.transport.* configuration options Key: FLINK-22048 URL: https://issues.apache.org/jira/browse/FLINK-22048 Project: Flink

[jira] [Created] (FLINK-22041) Configure DataStreamAllroundTest via Configuration instead of ParameterTool

Till Rohrmann created FLINK-22041: - Summary: Configure DataStreamAllroundTest via Configuration instead of ParameterTool Key: FLINK-22041 URL: https://issues.apache.org/jira/browse/FLINK-22041

Re: Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

Thanks a lot for all your input. To sum up the discussion so far: ## Final checkpoints We currently agree on favouring a single final checkpoint which can shut down the topology. In order to support this we need to be able to create a checkpoint after an operator has finished producing results.

Re: [DISCUSS] Feature freeze date for 1.13

+1 for the 31st of March for the feature freeze. Cheers, Till On Mon, Mar 29, 2021 at 10:12 AM Robert Metzger wrote: > +1 for March 31st for the feature freeze. > > > > On Fri, Mar 26, 2021 at 3:39 PM Dawid Wysakowicz > wrote: > > > Thank you Thomas! I'll definitely check the issue you linked.

Re: [DISCUSS] Apache Flink Jira Process

gt;>>> > >>>>>>>>>> Open > >>>>>>>>>> > >>>>>>>>>> Major > >>>>>>>>>> > >>>>>>>>>> auto-unassigned, auto-depriorit

Re: [VOTE] Apache Flink Jira Process (& Bot)

+1 for the proposal. Thanks for driving it Konstantin. Cheers, Till On Fri, Mar 26, 2021 at 9:58 AM Roman Khachatryan wrote: > +1, > > thanks for this improvement Konstantin! > > Regards, > Roman > > On Fri, Mar 26, 2021 at 8:49 AM Arvid Heise wrote: > > > > +1 from my side. > > Thanks for pr

Re: [BULK]Re: [SURVEY] Remove Mesos support

sos support ones more. Mesos is also already listed >> as deprecated in Flink's overall roadmap [1]. Maybe, it's time to align >> the >> documentation accordingly to make it more explicit? >> >> What do you think? >> >> Best, >> Matthia

<    1   2   3   4   5   6   7   8   9   10   >