Flink 1.4.0 release commit

2018-06-11 Thread Abdul Qadeer
Hi! I was trying to find out the commit from which 1.4.0 was released. The release was on 29th November 2017 but I am not able to find any commits around that date to verify this. Any help appreciated.

Test Bounce

2018-06-18 Thread Abdul Qadeer
Test

Metrics from detached execution mode

2018-06-18 Thread Abdul Qadeer
Hi! I am trying to fetch metrics provided by Beam SDK via Flink runner in detached mode, but looks like it is not supported yet. I understand from class DetachedJobExecutionResult that metrics are not supported to be extracted in detached mode job execution. Is this a limitation of Flink as a

Metrics from detached execution mode

2018-06-15 Thread Abdul Qadeer
Hi! I am trying to fetch metrics provided by Beam SDK via Flink runner in detached mode, but looks like it is not supported yet. I understand from class DetachedJobExecutionResult that metrics are not supported to be extracted in detached mode job execution. Is this a limitation of Flink as a

Multiple Task Slots support in Flink 1.5

2018-05-30 Thread Abdul Qadeer
Hi! I came across the following point in release notes of 1.5 version: "The allocation of TaskManagers with multiple slots is not fully supported yet." Does this mean the support for it will come as a

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Abdul Qadeer
acy mode, and >> everything works fine. >> >> For the new default mode, it also seems to works for us, so I'm not sure >> what is not supported. May be someone from Flink team could clarify. >> >> >> Best regards, >> >> Kien >> >> &

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Abdul Qadeer
i slot support for the SlotManager. > > Cheers, > Till > > On Fri, Jun 1, 2018 at 10:10 AM, Abdul Qadeer > wrote: > >> Hi Kien! >> >> Thank you for sharing your experience. Do you have Flink deployed in >> Standalone Cluster mode or K8S/Docker mode? >

Re: Taskmanager times out continuously for registration with Jobmanager

2018-10-11 Thread Abdul Qadeer
ith newer Flink versions > (1.5.4 or 1.6.1)? > > Cheers, > Till > > On Thu, Oct 11, 2018 at 9:24 AM Dawid Wysakowicz > wrote: > >> Hi Abdul, >> >> I've added Till and Gary to cc, who might be able to help you. >> >> Best, >> >> Dawid &g

Re: Taskmanager times out continuously for registration with Jobmanager

2018-10-12 Thread Abdul Qadeer
ome cache problem in Flink or Akka code JobManager is using? On Fri, Oct 12, 2018 at 1:05 AM Till Rohrmann wrote: > It is hard to tell without all logs but it could easily be a K8s setup > problem. Also problematic is that you are running a Flink version which is > no longer actively

Re: Taskmanager times out continuously for registration with Jobmanager

2018-10-12 Thread Abdul Qadeer
t;threadId":49,"threadPriority":5} A manual restart by docker restart or killing the JVM doesn't reproduce this problem. On Thu, Oct 11, 2018 at 11:15 AM Abdul Qadeer wrote: > Hi Till, > > I didn't try with newer versions as it is not possible to update the Fli

Taskmanager times out continuously for registration with Jobmanager

2018-10-10 Thread Abdul Qadeer
Hi, We are facing an issue in standalone HA mode in Flink 1.4.0 where Taskmanager restarts and is not able to register with the Jobmanager. It times out awaiting *AcknowledgeRegistration/AlreadyRegistered* message from Jobmanager Actor and keeps sending *RegisterTaskManager *message. The logs at

Re: AskTimeoutException

2019-04-12 Thread Abdul Qadeer
Hi Alex, The timeout shown in the exception is due to AkkaOptions.LOOKUP_TIMEOUT On Fri, 12 Apr 2019 at 09:45, Alex Soto wrote: > Hello, > > I am using Flink version 1.7.1. In a unit test, I create a local > environment: > > Configuration cfg = new Configuration(); >

Retention properties for CANCELED/FINISHED jobs

2019-05-29 Thread Abdul Qadeer
Hi! Is it possible to persist history of completed jobs across job manager restarts, without starting a history server? Also, is there a limit to how many jobs are stored in CANCELED state at jobmanager?

Need for user class path accessibility on all nodes

2019-06-17 Thread Abdul Qadeer
Hi! I was going through submission of a Flink program through CLI. I see that "--classpath " needs to be accessible from all nodes in the cluster as per documentation. As I understand the jar files are already part of the blob uploaded to JobManager from the CLI. The TaskManagers can download

Re: Need for user class path accessibility on all nodes

2019-06-18 Thread Abdul Qadeer
e to each node of cluster, the jobs depend on these jars could avoid >> deploying one by one. >> 2. Support NFS which is mentioned in option description of "--classpath". >> >> >> Abdul Qadeer 于2019年6月18日周二 上午11:45写道: >> >>> Hi Biao, >>>

Re: Need for user class path accessibility on all nodes

2019-06-17 Thread Abdul Qadeer
Hi Biao, I am aware of it - that's not my question. On Mon, Jun 17, 2019 at 7:42 PM Biao Liu wrote: > Hi Abdul, "--classpath " can be used for those are not included in > user jar. If all your classes are included in your jar passed to Flink, you > don't need this "

Re: Change Akka Ask Timeout for Job Submission Only

2019-12-19 Thread Abdul Qadeer
The relevant config here is "akka.ask.timeout". On Thu, Dec 19, 2019 at 6:51 PM tison wrote: > In previous version there is an "akka.client.timeout" option but it is > only used for timeout the future in client side so I don't think it change > akka scope timeout. &

Change Akka Ask Timeout for Job Submission Only

2019-12-19 Thread Abdul Qadeer
Hi! I am using Flink 1.8.3 and facing an issue where job submission through RestClusterClient times out on Akka (default value 10s). In previous Flink versions there was an option to set a different timeout value just for the submission client (ClusterClient config), but looks like it is not

Flink Job Cleanup Sequence

2020-02-06 Thread Abdul Qadeer
Hi! Is there a FLIP or doc describing how Flink (1.8+) cleans up the states of a Job upon cancellation in Zookeeper HA mode? I am trying to find the sequence of cleaning between SubmittedJobGraphs in Zookeeper, blobs in FS maintained by BlobStore etc.