We're not actually adding listeners twice. We're adding listeners once and calling init twice. The only listener that is added when the controller joins the cluster is the controller listener for leader election. The other listeners are added only if the controller becomes leader (and are removed whenever a callback occurs and the controller is not leader). See DistributedLeaderElection.java. However, there is definitely a problem: if we're instantiating a controller within the HelixManager, then that controller will automatically become leader, preventing any customized controller instance from doing so. So there needs to be some way of specifying that a different controller is desired. A hacky way of doing this is joining as an administrator and setting up the controller and leader election manually. I imagine that the connection adds a GenericHelixController automatically to simplify the API, though perhaps Jason can comment more. Date: Sun, 26 Jan 2014 14:54:32 -0800 Subject: Re: Duplicate INIT in Controller From: [email protected] To: [email protected]
Hi, I see there are two issues 1. We are adding listeners twice, as jason mentioned the duplicate init will be filtered but we will be processing the callbacks twice. While this wont be a problem since multiple listeners wont be concurrently invoked, it might impact performance slightly. 2. Since we add the listeners (generichelixcontroller) automatically, that means users cant really add there own customized listeners. In our recipes we do something similar to what Vinayak posted. Is there any reason why we add the listeners automatically when helixcontroller joins the cluster. Does this mean that we will setup watches even if the controller does not become leader. thanks, Kishore G On Sun, Jan 26, 2014 at 1:53 PM, Zhen Zhang <[email protected]> wrote: Hi Vinayak, we will definitely fix the problem, but the duplicate Init will be filtered by CallbackHandler, so your custom listener will not be invoked by multiple init callbacks. May I know what strange behavior you are worrying about? thanks, jason On Jan 26, 2014 1:47 PM, "Vinayak Borkar" <[email protected]> wrote: Hi Kanak, I do agree that instantiating my own GenericHelixController is not required. However, I will need to instantiate my own controller in its place if I would like custom controller behavior and then this issue could lead to strange behavior. Vinayak On 1/26/14, 1:36 PM, Kanak Biscuitwala wrote: Hi Vinayak, Looking at the code, here are a couple notes: 1. Instantiating your own GenericHelixController is unnecessary as the Helix manager already does this. 2. Even considering #1, you're right: there are still repeat callbacks. Here are the tracebacks I generated: https://gist.github.com/kanakb/8639730 The sequence is: 1. ZKHelixManager#handleNewSession() calls handleNewSessionAsController() 1. handleNewSessionAsController adds a controller callback for leader election 2. CallbackHandler calls init on that controller callback 3. The controller callback adds all other callbacks, resulting in init being called for each 4. ZKHelixManager#handleNewSession calls initHandlers() 5. initHandlers() iterates over all currently registered callbacks and calls init() on them 6. CallbackHandler knows it's already received an init, so it simply returns without doing anything on those callbacks So yes, we do have repeat callback initializations for the controller and it should be possible to avoid the double inits. These are, however, harmless and inexpensive. Kanak > Date: Sun, 26 Jan 2014 12:13:16 -0800 > From: [email protected] > To: [email protected] > Subject: Duplicate INIT in Controller > > Hi, > > I am observing duplicate handlers being added to the controller and this > is leading to INIT being called multiple times on the same handlers. > > Here is my code to start the controler, which looks fairly standard: > > @Override > protected void doStart() throws SystemException { > hManager = HelixManagerFactory.getZKHelixManager(clusterName, > UUID.randomUUID().toString(), > InstanceType.CONTROLLER, zkAddr); > try { > hManager.connect(); > } catch (Exception e) { > throw new SystemException(e); > } > GenericHelixController controller = new GenericHelixController(); > HelixControllerMain.addListenersToController(hManager, controller); > } > > > > Here are the log messages due to duplicate calls: > > 2014-01-26 12:11:16,721 WARN [CallbackHandler] Skip processing > callbacks for listener: > org.apache.helix.messaging.handling.HelixTaskExecutor@7f3076b2, path: > /e78dead7-0c2f-4c5f-af7e-f25e8c170db1/CONTROLLER/MESSAGES, expected > types: [CALLBACK, FINALIZE] but was INIT > 2014-01-26 12:11:16,721 WARN [CallbackHandler] Skip processing > callbacks for listener: > org.apache.helix.controller.GenericHelixController@4db4bfda, path: > /e78dead7-0c2f-4c5f-af7e-f25e8c170db1/CONFIGS/PARTICIPANT, expected > types: [CALLBACK, FINALIZE] but was INIT > 2014-01-26 12:11:16,721 WARN [CallbackHandler] Skip processing > callbacks for listener: > org.apache.helix.controller.GenericHelixController@4db4bfda, path: > /e78dead7-0c2f-4c5f-af7e-f25e8c170db1/LIVEINSTANCES, expected types: > [CALLBACK, FINALIZE] but was INIT > 2014-01-26 12:11:16,721 WARN [CallbackHandler] Skip processing > callbacks for listener: > org.apache.helix.controller.GenericHelixController@4db4bfda, path: > /e78dead7-0c2f-4c5f-af7e-f25e8c170db1/IDEALSTATES, expected types: > [CALLBACK, FINALIZE] but was INIT > 2014-01-26 12:11:16,721 WARN [CallbackHandler] Skip processing > callbacks for listener: > org.apache.helix.controller.GenericHelixController@4db4bfda, path: > /e78dead7-0c2f-4c5f-af7e-f25e8c170db1/CONTROLLER, expected types: > [CALLBACK, FINALIZE] but was INIT > > > Thanks, > Vinayak
