Thanks for the quick response. We actually tried lowering that to 4, but it didn¹t have any effect.
--- Brian O'Neill Chief Architect Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42> healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: SKG <[email protected]> Date: Thursday, November 7, 2013 at 12:45 PM To: <[email protected]> Cc: <[email protected]>, Brian O'Neill <[email protected]> Subject: Re: Worker stops heartbeating. May be the hearbeat thread is not processed by worker java process. Check this parameter in defaults.yaml of supervisor server. topology.receiver.buffer.size: 8 # setting it too high causes a lot of problems (heartbeat thread gets starved, throughput plummets) On Thursday, November 7, 2013 11:05:23 PM UTC+5:30, Brian O'Neill wrote: > > We are running on 0.9.0-rc2, and have workers that stop heart beating. > (tailing the worker log, we see the WorkerHeartbeat message stop) > > At the time the worker stops heart beating, memory seems fine, CPU seems fine. > (jconsole) > The VM seems to continue processing data and is healthy enough for us to take > a thread dump. (jstack) > Nothing immediately jumped out in the thread dump. > > The worker is eventually killed, and re-spawned. > > What is the best way to diagnose the issue? > > Any ideas? > > -brian > > --- > Brian O'Neill > Chief Architect > Health Market Science > The Science of Better Results > 2700 Horizon Drive King of Prussia, PA 19406 > M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42> > healthmarketscience.com > > > This information transmitted in this email message is for the intended > recipient only and may contain confidential and/or privileged material. If you > received this email in error and are not the intended recipient, or the person > responsible to deliver it to the intended recipient, please contact the sender > at the email above and delete this email and any attachments and destroy any > copies thereof. Any review, retransmission, dissemination, copying or other > use of, or taking any action in reliance upon, this information by persons or > entities other than the intended recipient is strictly prohibited. >
