Hi Ted, I don't know how how to take stacktrace using jstack. But i tried using some commands but it fails. can you help me in this?
Thanks On Sun, Jan 10, 2016 at 11:10 PM, Ted Yu <[email protected]> wrote: > What I meant was to capture stack trace of the thrift server (using jstack > command) when your query is running. > > This would give us some more clue as to the state thrift server was in when > the problem happened. > > Cheers > > On Sun, Jan 10, 2016 at 9:11 AM, Rajeshkumar J < > [email protected]> > wrote: > > > Hi Ted, > > > > This is what I found in the Hbase thrift server logs > > > > 2016-01-09 06:27:43,139 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:java.library.path=;C:\Hadoop\bin > > 2016-01-09 06:27:43,139 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:java.io.tmpdir=C:\Windows\TEMP\ > > 2016-01-09 06:27:43,139 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:java.compiler=<NA> > > 2016-01-09 06:27:43,139 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:os.name=Windows Server 2012 > > 2016-01-09 06:27:43,139 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:os.arch=amd64 > > 2016-01-09 06:27:43,140 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:os.version=6.2 > > 2016-01-09 06:27:43,140 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:user.name=xxxx > > 2016-01-09 06:27:43,140 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:user.home=C:\ > > 2016-01-09 06:27:43,140 INFO [thrift-worker-0] zookeeper.ZooKeeper: > Client > > environment:user.dir=C:\HBase\bin > > 2016-01-09 06:27:43,142 INFO [thrift-worker-0] zookeeper.ZooKeeper: > > Initiating client connection, > > connectString=hostname:2181,hostname:2181,hostname:2181 > > sessionTimeout=90000 watcher=hconnection-0x45e35460x0, > > quorum=hostname:2181,hostname:2181,hostname:2181, baseZNode=/hbase > > 2016-01-09 06:27:43,339 INFO [thrift-worker-0-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Opening socket connection to server > > hostname/xx.xx.x.xx:2181. Will not attempt to authenticate using SASL > > (unknown error) > > 2016-01-09 06:27:43,342 INFO [thrift-worker-0-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Socket connection established to > > hostname/xx.xx.x.xx:2181, initiating session > > 2016-01-09 06:27:43,365 INFO [thrift-worker-0-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Session establishment complete on server > > hostname/xx.xx.x.xx:2181, sessionid = 0x1522613893e0004, negotiated > timeout > > = 40000 > > 2016-01-09 07:14:10,607 INFO > > [hostname,60000,1452337815006_splitLogManager__ChoreService_1] > > master.SplitLogManager$TimeoutMonitor: Chore: SplitLogManager Timeout > > Monitor missed its start time > > 2016-01-09 07:16:43,511 INFO > > [B.defaultRpcServer.handler=8,queue=2,port=60000] master.HMaster: > > Client=Administrator/null create 'chk2', {NAME => 'fa', > DATA_BLOCK_ENCODING > > => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => > '1', > > COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', > > KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => > 'false', > > BLOCKCACHE => 'true'} > > 2016-01-09 07:16:44,121 INFO [ProcedureExecutorThread-1] > > procedure2.ProcedureExecutor: Rolledback procedure CreateTableProcedure > > (table=chk2) user=Administrator (auth:SIMPLE) id=2 state=ROLLEDBACK > > exec-time=462msec exception=org.apache.hadoop.hbase.TableExistsException: > > chk2 > > 2016-01-09 07:49:55,574 INFO [ConnectionCache_ChoreService_1] > > client.ConnectionManager$HConnectionImplementation: Closing zookeeper > > sessionid=0x1522613893e0004 > > 2016-01-09 07:49:55,629 INFO [ConnectionCache_ChoreService_1] > > zookeeper.ZooKeeper: Session: 0x1522613893e0004 closed > > 2016-01-09 07:49:55,661 INFO [thrift-worker-0-EventThread] > > zookeeper.ClientCnxn: EventThread shut down > > 2016-01-09 18:40:29,469 INFO [thrift-worker-1] > > zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x591287f8 > > connecting to ZooKeeper > ensemble=hostname:2181,hostname:2181,hostname:2181 > > 2016-01-09 18:40:29,482 INFO [thrift-worker-1] zookeeper.ZooKeeper: > > Initiating client connection, > > connectString=hostname:2181,hostname:2181,hostname:2181 > > sessionTimeout=90000 watcher=hconnection-0x591287f80x0, > > quorum=hostname:2181,hostname:2181,hostname:2181, baseZNode=/hbase > > 2016-01-09 18:40:29,553 INFO [thrift-worker-1-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Opening socket connection to server > > hostname/xx.xx.x.xx:2181. Will not attempt to authenticate using SASL > > (unknown error) > > 2016-01-09 18:40:29,559 INFO [thrift-worker-1-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Socket connection established to > > hostname/xx.xx.x.xx:2181, initiating session > > 2016-01-09 18:40:29,569 INFO [thrift-worker-1-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Session establishment complete on server > > hostname/xx.xx.x.xx:2181, sessionid = 0x35226137e300008, negotiated > timeout > > = 40000 > > 2016-01-09 18:54:25,558 INFO [ConnectionCache_ChoreService_1] > > client.ConnectionManager$HConnectionImplementation: Closing zookeeper > > sessionid=0x35226137e300008 > > 2016-01-09 18:54:25,566 INFO [ConnectionCache_ChoreService_1] > > zookeeper.ZooKeeper: Session: 0x35226137e300008 closed > > 2016-01-09 18:54:25,578 INFO [thrift-worker-1-EventThread] > > zookeeper.ClientCnxn: EventThread shut down > > 2016-01-09 20:05:42,469 INFO [thrift-worker-13] > > zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x6305dbeb > > connecting to ZooKeeper > ensemble=hostname:2181,hostname:2181,hostname:2181 > > 2016-01-09 20:05:42,476 INFO [thrift-worker-13] zookeeper.ZooKeeper: > > Initiating client connection, > > connectString=hostname:2181,hostname:2181,hostname:2181 > > sessionTimeout=90000 watcher=hconnection-0x6305dbeb0x0, > > quorum=hostname:2181,hostname:2181,hostname:2181, baseZNode=/hbase > > 2016-01-09 20:05:42,507 INFO > [thrift-worker-13-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Opening socket connection to server > > hostname/xx.xx.x.xx:2181. Will not attempt to authenticate using SASL > > (unknown error) > > 2016-01-09 20:05:42,510 INFO > [thrift-worker-13-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Socket connection established to > > hostname/xx.xx.x.xx:2181, initiating session > > 2016-01-09 20:05:42,708 INFO > [thrift-worker-13-SendThread(hostname:2181)] > > zookeeper.ClientCnxn: Session establishment complete on server > > hostname/xx.xx.x.xx:2181, sessionid = 0x1522613893e0005, negotiated > timeout > > = 40000 > > 2016-01-09 20:15:45,541 INFO [ConnectionCache_ChoreService_1] > > client.ConnectionManager$HConnectionImplementation: Closing zookeeper > > sessionid=0x1522613893e0005 > > 2016-01-09 20:15:45,567 INFO [ConnectionCache_ChoreService_1] > > zookeeper.ZooKeeper: Session: 0x1522613893e0005 closed > > > > > > Thanks, > > Rajeshkumar J > > > > On Sun, Jan 10, 2016 at 10:31 PM, Ted Yu <[email protected]> wrote: > > > > > Can you take a stack trace of the thrift server and pastebin the trace > ? > > > > > > Thanks > > > > > > On Sun, Jan 10, 2016 at 8:56 AM, Rajeshkumar J < > > > [email protected]> > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > I am connecting via Hbase thrift server to access records in Hbase > and > > I > > > am > > > > doing this in C# and i am using range scan to get records from the > > table. > > > > This seems to be working for few hours but after few hours it didn't > > > > returned any records and then i searched hbase logs i found this > > > > > > > > 2016-01-04 00:44:54,880 INFO [ConnectionCleaner] > > > > client.HConnectionManager$HConnectionImplementation: Closing > zookeeper > > > > sessionid=0x3520b1721650006 > > > > 2016-01-04 00:44:54,891 INFO [ConnectionCleaner] > zookeeper.ZooKeeper: > > > > Session: 0x3520b1721650006 closed > > > > 2016-01-04 00:44:54,891 INFO [thrift-worker-0-EventThread] > > > > zookeeper.ClientCnxn: EventThread shut down > > > > > > > > And I am using hbase 0.98.13 > > > > > > > > I have asked this question before and some user suggested me to > upgrade > > > to > > > > new version so i have upgraded HBase to 1.1.2. But now it also it > > didn't > > > > return any records but different log generated as shown below > > > > > > > > 2016-01-09 20:05:42,708 INFO > [thrift-worker-13-SendThread(xxxx:2181)] > > > > zookeeper.ClientCnxn: Session establishment complete on server > > > > xxxx/xx.xx.x.xx:2181, sessionid = 0x1522613893e0005, negotiated > > timeout = > > > > 40000 > > > > 2016-01-09 20:15:45,541 INFO [ConnectionCache_ChoreService_1] > > > > client.ConnectionManager$HConnectionImplementation: Closing zookeeper > > > > sessionid=0x1522613893e0005 > > > > 2016-01-09 20:15:45,567 INFO [ConnectionCache_ChoreService_1] > > > > zookeeper.ZooKeeper: Session: 0x1522613893e0005 closed > > > > 2016-01-09 20:15:45,567 INFO [thrift-worker-13-EventThread] > > > > zookeeper.ClientCnxn: EventThread shut down > > > > > > > > Can any one help me in this? > > > > > > > > Regards, > > > > Rajeshkumar J > > > > > > > > > >
