Indeed they kill was commanded by nimbus. I see messages like the following before supervisor kills worker. Now the question is why nimbus believe worker executors are not alive?
Fang ===== 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[29 29] not alive 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[30 30] not alive 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[31 31] not alive 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[32 32] not alive 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[1 1] not alive 2015-06-04T00:35:52.229+0000 b.s.d.nimbus [INFO] Executor asyncVarGenTopology-2-1433377984:[33 33] not alive 2015-06-04T00:35:52.242+0000 b.s.s.EvenScheduler [INFO] Available slots: (["1fee065f-699e-427d-b626-7ae51132db12" 6702] ["1fee065f-699e-427d-b626-7ae51132db12" 6701] ["1fee065f-699e-427d-b626-7ae51132db12" 6700] ["029d3393-8bd8-4ea7-9f79-3b23f46aa289" 6702] ["029d3393-8bd8-4ea7-9f79-3b23f46aa289" 6701] ["029d3393-8bd8-4ea7-9f79-3b23f46aa289" 6700] ["87d03069-baec-4c9c-ba92-fc548cd07b7d" 6702] ["87d03069-baec-4c9c-ba92-fc548cd07b7d" 6701] ["87d03069-baec-4c9c-ba92-fc548cd07b7d" 6700] ["06d1b8d8-eb1c-473f-93cb-444d3c69b24a" 6702] ["06d1b8d8-eb1c-473f-93cb-444d3c69b24a" 6701] ["06d1b8d8-eb1c-473f-93cb-444d3c69b24a" 6700] ["8f2db495-c860-4ff2-99f9-11c74b6cf258" 6702] ["8f2db495-c860-4ff2-99f9-11c74b6cf258" 6701] ["8f2db495-c860-4ff2-99f9-11c74b6cf258" 6700]) 2015-06-04T00:35:52.242+0000 b.s.d.nimbus [INFO] Reassigning asyncVarGenTopology-2-1433377984 to 5 slots 2015-06-04T00:35:52.243+0000 b.s.d.nimbus [INFO] Reassign executors: [[2 2] [34 34] [3 3] [35 35] [4 4] [36 36] [5 5] [37 37] [6 6] [38 38] [7 7] [39 39] [8 8] [40 40] [9 9] [41 41] [10 10] [42 42] [11 11] [43 43] [12 12] [44 44] [13 13] [45 45] [14 14] [46 46] [15 15] [47 47] [16 16] [48 48] [17 17] [49 49] [18 18] [50 50] [19 19] [51 51] [20 20] [21 21] [22 22] [23 23] [24 24] [25 25] [26 26] [27 27] [28 28] [29 29] [30 30] [31 31] [32 32] [1 1] [33 33]] On Tue, Jun 2, 2015 at 3:08 PM, Jeffery Maass <[email protected]> wrote: > disallowed = the request to shut down came from nimbus. > > It takes a village. Also, it takes all of the logs..... > > Get them together to see a better picture: > Nimbus, Supervisor, Worker > > Thank you for your time! > > +++++++++++++++++++++ > Jeff Maass <[email protected]> > linkedin.com/in/jeffmaass > stackoverflow.com/users/373418/maassql > +++++++++++++++++++++ > > > On Tue, Jun 2, 2015 at 10:29 AM, Grant Overby (groverby) < > [email protected]> wrote: > >> 0.9.3.2.2 (hdp 2.2.4.2-2) >> >> *Grant Overby* >> Software Engineer >> Cisco.com <http://www.cisco.com/> >> [email protected] >> Mobile: *865 724 4910 <865%20724%204910>* >> >> >> >> Think before you print. >> >> This email may contain confidential and privileged material for the sole >> use of the intended recipient. Any review, use, distribution or disclosure >> by others is strictly prohibited. If you are not the intended recipient (or >> authorized to receive for the recipient), please contact the sender by >> reply email and delete all copies of this message. >> >> Please click here >> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for >> Company Registration Information. >> >> >> >> >> From: Harsha <[email protected]> >> Date: Tuesday, June 2, 2015 at 11:24 AM >> To: "Grant Overby (groverby)" <[email protected]>, " >> [email protected]" <[email protected]> >> Subject: Re: Supervisor believes worker has not started. >> >> Which version of storm are you using? >> >> -- >> Harsha >> >> >> On June 2, 2015 at 8:04:48 AM, Grant Overby (groverby) ( >> [email protected]) wrote: >> >> Same here. The worker isn’t committing suicide; its being murdered by >> the supervisor. >> >> *Grant Overby* >> Software Engineer >> Cisco.com <http://www.cisco.com/> >> [email protected] >> Mobile: *865 724 4910 <865%20724%204910>* >> >> >> >> Think before you print. >> >> This email may contain confidential and privileged material for the sole >> use of the intended recipient. Any review, use, distribution or disclosure >> by others is strictly prohibited. If you are not the intended recipient (or >> authorized to receive for the recipient), please contact the sender by >> reply email and delete all copies of this message. >> >> Please click here >> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for >> Company Registration Information. >> >> >> >> From: Fang Chen <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Tuesday, June 2, 2015 at 2:25 AM >> To: "[email protected]" <[email protected]> >> Subject: Re: Supervisor believes worker has not started. >> >> ld not find a workaround. In my case, the w >> >> >
