Re: map concurrent modification exception analysis when checkpoint

But, I do not know why this leads to the job's failure and recovery since I have set the tolerable failed checkpoint to Integer.MAX_VALUE. Due to the failure, my task manager failed because of the task cancel timeout, and about 80% of task managers went down due to cancel timeout. yidan zhao 于202

Re: DataStream to Table API

Thank you, Caizhi, for looking into this and identifying the source of the bug. Is there a way to work around this at the API level until this bug is resolved? Can I somehow "inject" the type? Thanks a lot for your help, Matthias On Thu, Aug 19, 2021 at 10:15 PM Caizhi Weng wrote: > Hi! > > I'v

Re: map concurrent modification exception analysis when checkpoint

Ok, thanks. I have some result, and you can give some ensure. Here is the issue code: The async function's implementation. It do async redis query, and fill some data back. In code [ currentBatch.get(i).getD().put("ipLabel", objects.getResponses().get(i)); ] the getD() returns a map attr in Origin

RE: failures during job start

Thanks, will try that. From: Chesnay Schepler Sent: Friday, August 20, 2021 8:06 AM To: Colletta, Edward ; user@flink.apache.org Subject: Re: failures during job start NOTICE: This email is from an external sender - do not click on links or attachments unless you recognize the sender and know t

RE: Upgrading from Flink on YARN 1.9 to 1.11

Hi David, I was able to get this working using your suggestion: 1)Deploy a Flink YARN Session Cluster, noting the host + port of the session’s Job Manager. 2)Submit a Flink job using the session’s details, i.e submitting Flink job with ‘-m host:port’ option. Thanks for clearing things

aggregation, triggers, and no activity

I use event time,with Kafka as my source. The system that I am developing requires data to be aggregated every 15 minutes, thus I am using a Tumbling Event Time window. However, my system also is required to take action every 15 minutes even if there is activity. I need the elements collected in t

Re: failures during job start

I don't think there are any metrics; logging-wise you will need to do some detective work. We do know which tasks have started deployment by this message from the JobManager: ExecutionGraph [] - (/) () switched from SCHEDULED to DEPLOYING. We also know which have completed deployment by thi

Re: Pre shuffle aggregation in flink is not working

Hi Suman, > But I am always seeing the following code of ` *AbstractMapBundleOperator.java*` `*numOfElements` *is always 0. It is weird, please set a breakpoint at line ` *bundleTrigger.onElement(input);*` in `*processElement*` method to see what happens when a record is processed by `*processEle

Re: How can I achieve 'sink.partition-commit.policy.kind'='metastore,success-file' with batch Hive sink?

Hi Jingsong, I have created a JIRA ticket https://issues.apache.org/jira/browse/FLINK-23891. Best, Yik San On Fri, Aug 20, 2021 at 3:32 PM Yik San Chan wrote: > Hi Caizhi, > > Thanks for the work around! It should work fine. > > Hi Jingsong, > > Thanks for the suggestion. Before creating a JIR

Re: How can I achieve 'sink.partition-commit.policy.kind'='metastore,success-file' with batch Hive sink?

Hi Caizhi, Thanks for the work around! It should work fine. Hi Jingsong, Thanks for the suggestion. Before creating a JIRA ticket, I wonder if this is considered a valid ask at the first glance? If so, I will create a JIRA ticket. Best, Yik San On Fri, Aug 20, 2021 at 11:28 AM Jingsong Li wro

Re: submit new job is not working

thanks  @chesney its later On Friday, August 20, 2021, 03:05:26 AM AST, Chesnay Schepler wrote: Is the problem that previously uploaded jars are no longer available (which would be expected behavior), or that you cannot upload new jars? If it is the latter, could you use the develope

Re: submit new job is not working

Is the problem that previously uploaded jars are no longer available (which would be expected behavior), or that you cannot upload new jars? If it is the latter, could you use the developer tools of you browser to check what response the UI receives when attempting to upload the jar? On 20/08/2