Re: Job Manager and Task Manager unable to communicate

2023-11-27 Thread Benoit Tailhades
Hello, Tauseef, Can you give more details ? Are your task manager and job manager on the same vm ? How did you configure the Job manager address in the task manager conf file ? Did you modify the binding in configuration files ? Benoit Le lun. 27 nov. 2023 à 14:29, Tauseef Janvekar a écrit :

Re: Recursive Split Detection + same split optimization

2023-07-10 Thread Benoit Tailhades
already extracted ID into the >> state and storing it into my state backend. What do you think about this ? >> > > For choice 1, put the information in the enumerator's state. > For choice 2, no need to consider that issue. > > Best, > Hang > > Benoit Tailha

Recursive Split Detection + same split optimization

2023-07-09 Thread Benoit Tailhades
Hello Everyone, I am trying to implement a custom source where split detection is an expensive operation and I would like to benefit from the split reader results to build my next splits. Basically, my source is taking as input an id from my external system, let's name it ID1. >From ID1, I can