Hi Lalit, I asked for thread dumps, not logs. To get a thread dump, use your jdk's "jstack" command on the agents process that won't shut down.
Thanks, Karl On Tue, Sep 16, 2014 at 4:51 AM, lalit jangra <[email protected]> wrote: > Hi Karl, > > Below are the logs i get when i try to stop agents. Please suggest. > > > [root@iwdc1preecma03 ecmadmin]# cd > /app/IW/MCF1.7/dist/multiprocess-zk-example > > [root@iwdc1preecma03 multiprocess-zk-example]# ./stop-agents.sh > > Configuration file successfully read > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT > > [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name= > iwdc1preecma03.iwater.ie > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.version=1.7.0_25 > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.vendor=Oracle Corporation > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.home=/app/IW/java/jre > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.class.path=.:../lib/mcf-pull-agent.jar:../lib/mcf-agents.jar:../lib/mcf-core.jar:../lib/hsqldb.jar:../lib/derbyLocale_zh_TW.jar:../lib/derbyLocale_zh_CN.jar:../lib/derbyLocale_ru.jar:../lib/derbyLocale_pt_BR.jar:../lib/derbyLocale_pl.jar:../lib/derbyLocale_ko_KR.jar:../lib/derbyLocale_ja_JP.jar:../lib/derbyLocale_it.jar:../lib/derbyLocale_hu.jar:../lib/derbyLocale_fr.jar:../lib/derbyLocale_es.jar:../lib/derbyLocale_de_DE.jar:../lib/derbyLocale_cs.jar:../lib/derbytools.jar:../lib/derbynet.jar:../lib/derby.jar:../lib/postgresql.jar:../lib/mail.jar:../lib/slf4j-simple.jar:../lib/slf4j-api.jar:../lib/velocity.jar:../lib/xml-apis.jar:../lib/xercesImpl.jar:../lib/xalan.jar:../lib/servlet-api.jar:../lib/serializer.jar:../lib/log4j.jar:../lib/commons-logging.jar:../lib/commons-lang.jar:../lib/commons-io.jar:../lib/httpclient.jar:../lib/httpcore.jar:../lib/commons-fileupload.jar:../lib/commons-el.jar:../lib/commons-collections.jar:../lib/commons-codec.jar:../lib/json-simple.jar:../lib/json.jar:../lib/zookeeper.jar:../lib/activation.jar:../lib/saaj-impl.jar:../lib/saaj-api.jar:../lib/wsdl4j.jar:../lib/wss4j.jar:../lib/axis.jar:../lib/axis-jaxrpc.jar:../lib/commons-discovery.jar:../lib/geronimo-javamail_1.4_spec.jar:../lib/castor.jar: > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.io.tmpdir=/tmp > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:java.compiler=<NA> > > [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name > =Linux > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:os.arch=amd64 > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:os.version=2.6.32-358.el6.x86_64 > > [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name > =root > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:user.home=/root > > [main] INFO org.apache.zookeeper.ZooKeeper - Client > environment:user.dir=/app/IW/MCF1.7/dist/multiprocess-zk-example > > [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, > connectString=iwdc1preecma03:2181,iwdc1preecma03:2182,iwdc1preecma03:2183,iwdc2preecma04:2181,iwdc2preecma04:2182,iwdc2preecma04:2183 > sessionTimeout=4000 > watcher=org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection$ZooKeeperWatcher@78618ac5 > > [main-SendThread(iwdc1preecma03.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > iwdc1preecma03.iwater.ie/10.231.72.24:2181. Will not attempt to > authenticate using SASL (unknown error) > > [main-SendThread(iwdc1preecma03.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > iwdc1preecma03.iwater.ie/10.231.72.24:2181, initiating session > > [main-SendThread(iwdc1preecma03.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Unable to read additional data from > server sessionid 0x0, likely server has closed socket, closing socket > connection and attempting reconnect > > [main-SendThread(iwdc2preecma04.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > iwdc2preecma04.iwater.ie/10.231.72.25:2182. Will not attempt to > authenticate using SASL (unknown error) > > [main-SendThread(iwdc2preecma04.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > iwdc2preecma04.iwater.ie/10.231.72.25:2182, initiating session > > [main-SendThread(iwdc2preecma04.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Unable to read additional data from > server sessionid 0x0, likely server has closed socket, closing socket > connection and attempting reconnect > > [main-SendThread(iwdc1preecma03.iwater.ie:2183)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > iwdc1preecma03.iwater.ie/10.231.72.24:2183. Will not attempt to > authenticate using SASL (unknown error) > > [main-SendThread(iwdc1preecma03.iwater.ie:2183)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > iwdc1preecma03.iwater.ie/10.231.72.24:2183, initiating session > > [main-SendThread(iwdc1preecma03.iwater.ie:2183)] INFO > org.apache.zookeeper.ClientCnxn - Unable to read additional data from > server sessionid 0x0, likely server has closed socket, closing socket > connection and attempting reconnect > > [main-SendThread(iwdc2preecma04.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > iwdc2preecma04.iwater.ie/10.231.72.25:2181. Will not attempt to > authenticate using SASL (unknown error) > > [main-SendThread(iwdc2preecma04.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > iwdc2preecma04.iwater.ie/10.231.72.25:2181, initiating session > > [main-SendThread(iwdc2preecma04.iwater.ie:2181)] INFO > org.apache.zookeeper.ClientCnxn - Unable to read additional data from > server sessionid 0x0, likely server has closed socket, closing socket > connection and attempting reconnect > > [main-SendThread(iwdc1preecma03.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > iwdc1preecma03.iwater.ie/10.231.72.24:2182. Will not attempt to > authenticate using SASL (unknown error) > > [main-SendThread(iwdc1preecma03.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > iwdc1preecma03.iwater.ie/10.231.72.24:2182, initiating session > > [main-SendThread(iwdc1preecma03.iwater.ie:2182)] INFO > org.apache.zookeeper.ClientCnxn - Session establishment complete on server > iwdc1preecma03.iwater.ie/10.231.72.24:2182, sessionid = > 0x2487d06d7540078, negotiated timeout = 4000 > > Shutdown signal sent > > [Shutdown thread] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x2487d06d7540078 closed > > [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut > down > > > On Mon, Sep 15, 2014 at 1:46 AM, Karl Wright <[email protected]> wrote: > >> Hi Lalit, >> >> Please obtain thread dumps from the agents processese that will not stop. >> >> Thanks, >> Karl >> >> >> On Sun, Sep 14, 2014 at 12:18 PM, lalit jangra <[email protected]> >> wrote: >> >>> Hi, >>> >>> I am using MCF 1.7 in clustered mode with 6 zookeeper nodes and two >>> agents running on separate machines. Once i start the cluster (running >>> agent1 on node1 and agent 2 on node 2), everything is fine but after a long >>> time say a day or two, if i try to kill agents or restart them, agents do >>> not get killed and using ./stop-agents.sh shows right logs but do not kill >>> agents. >>> >>> Please suggest. >>> >>> Regards, >>> Lalit. >>> >> >> > > > -- > Regards, > Lalit. >
