Thanks for your respond,but Zk is working normally.





At 2017-08-10 15:21:49, "Billy Liu" <billy...@apache.org> wrote:
>The first WARN happened during ZK timeout. Please check the ZK connection
>first.
>
>2017-08-10 10:58 GMT+08:00 shendandan <shendandan2015y...@163.com>:
>
>> Hi!
>>    The process of kylin often hangs up,and the erroe in kylin,log:
>>
>>
>> 2017-08-10 10:14:22,292 INFO  [BadQueryDetector]
>> service.BadQueryDetector:151 : System free memory less than 100 MB. 1
>> queries running.
>> 2017-08-10 10:14:34,498 INFO  [Thread-11-EventThread]
>> state.ConnectionStateManager:228 : State change: SUSPENDED
>> 2017-08-10 10:14:34,506 INFO  [pool-7-thread-1]
>> threadpool.DefaultScheduler:118 : Job Fetcher: 0 should running, 0 actual
>> running, 0 ready, 63 already succeed, 3 error, 13 discarded, 0 others
>> 2017-08-10 10:14:34,999 INFO  [localhost-startStop-1-SendThread(
>> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening socket
>> connection to server hadoop-senior01.ctcf.com/10.1.8.90:2181. Will not
>> attempt to authenticate using SASL (unknown error)
>> 2017-08-10 10:14:34,999 INFO  [Thread-11-SendThread(hadoop-
>> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening socket
>> connection to server hadoop-senior03.ctcf.com/10.1.8.92:2181. Will not
>> attempt to authenticate using SASL (unknown error)
>> 2017-08-10 10:14:34,999 INFO  [localhost-startStop-1-SendThread(
>> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket
>> connection established, initiating session, client: /10.1.8.90:38229,
>> server: hadoop-senior01.ctcf.com/10.1.8.90:2181
>> 2017-08-10 10:14:34,999 INFO  [Thread-11-SendThread(hadoop-
>> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket connection
>> established, initiating session, client: /10.1.8.90:32962, server:
>> hadoop-senior03.ctcf.com/10.1.8.92:2181
>> 2017-08-10 10:14:35,001 INFO  [Thread-11-SendThread(hadoop-
>> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:1094 : Unable to reconnect
>> to ZooKeeper service, session 0x25dc4bd90cc002a has expired, closing socket
>> connection
>> 2017-08-10 10:14:35,002 INFO  [localhost-startStop-1-SendThread(
>> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:1094 : Unable to
>> reconnect to ZooKeeper service, session 0x35dc4bd90460028 has expired,
>> closing socket connection
>> 2017-08-10 10:14:35,013 WARN  [localhost-startStop-1-EventThread]
>> client.HConnectionManager$HConnectionImplementation:2468 : This client
>> just lost it's session with ZooKeeper, closing it. It will be recreated
>> next time someone needs it
>> org.apache.zookeeper.KeeperException$SessionExpiredException:
>> KeeperErrorCode = Session expired
>> at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(
>> ZooKeeperWatcher.java:517)
>> at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.
>> process(ZooKeeperWatcher.java:435)
>> at org.apache.zookeeper.ClientCnxn$EventThread.
>> processEvent(ClientCnxn.java:522)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>> 2017-08-10 10:14:35,013 INFO  [Thread-11-EventThread]
>> state.ConnectionStateManager:228 : State change: LOST
>> 2017-08-10 10:14:35,014 INFO  [localhost-startStop-1-EventThread]
>> client.HConnectionManager$HConnectionImplementation:1837 : Closing
>> zookeeper sessionid=0x35dc4bd90460028
>> 2017-08-10 10:14:35,014 WARN  [Thread-11-EventThread]
>> curator.ConnectionState:289 : Session expired event received
>> 2017-08-10 10:14:35,014 INFO  [localhost-startStop-1-EventThread]
>> zookeeper.ClientCnxn:512 : EventThread shut down
>> 2017-08-10 10:14:35,015 INFO  [Thread-11-EventThread]
>> zookeeper.ZooKeeper:438 : Initiating client connection, connectString=
>> hadoop-senior01.ctcf.com:2181,hadoop-senior02.ctcf.com:2181,
>> hadoop-senior03.ctcf.com:2181 sessionTimeout=60000
>> watcher=org.apache.curator.ConnectionState@4d6f4fa7
>> 2017-08-10 10:14:35,017 INFO  [Thread-11-SendThread(hadoop-
>> senior02.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening socket
>> connection to server hadoop-senior02.ctcf.com/10.1.8.91:2181. Will not
>> attempt to authenticate using SASL (unknown error)
>> 2017-08-10 10:14:35,017 INFO  [Thread-11-SendThread(hadoop-
>> senior02.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket connection
>> established, initiating session, client: /10.1.8.90:57503, server:
>> hadoop-senior02.ctcf.com/10.1.8.91:2181
>> 2017-08-10 10:14:35,021 INFO  [Thread-11-EventThread]
>> zookeeper.ClientCnxn:512 : EventThread shut down
>> 2017-08-10 10:14:35,021 INFO  [Thread-11-SendThread(hadoop-
>> senior02.ctcf.com:2181)] zookeeper.ClientCnxn:1235 : Session
>> establishment complete on server hadoop-senior02.ctcf.com/10.1.8.91:2181,
>> sessionid = 0x25dc4bd90cc002c, negotiated timeout = 40000
>> 2017-08-10 10:14:35,021 INFO  [Thread-11-EventThread]
>> state.ConnectionStateManager:228 : State change: RECONNECTED
>> 2017-08-10 10:15:09,204 INFO  [pool-7-thread-1]
>> threadpool.DefaultScheduler:118 : Job Fetcher: 0 should running, 0 actual
>> running, 0 ready, 63 already succeed, 3 error, 13 discarded, 0 others
>> 2017-08-10 10:15:26,969 INFO  [BadQueryDetector]
>> service.BadQueryDetector:179 : Slow query has been running 551.737 seconds
>> (project:eam_contract_baseinfo, thread: 0x4a, user:ADMIN) -- SELECT *
>> FROM "TN_QUERY"."BASEINFO_REPAY_HIS_FACT_PARTATION"
>> 2017-08-10 10:15:26,971 DEBUG [BadQueryDetector] 
>> badquery.BadQueryHistoryManager:84
>> : Loaded 10 Bad Query(s)
>> 2017-08-10 10:15:26,976 DEBUG [BadQueryDetector]
>> hbase.HBaseResourceStore:262 : Update row 
>> /bad_query/eam_contract_baseinfo.json
>> from oldTs: 1502331183691, to newTs: 1502331326972, operation result: true
>> 2017-08-10 10:15:26,977 INFO  [BadQueryDetector]
>> service.BadQueryDetector:169 : Problematic thread 0x4a
>> at org.apache.kylin.metadata.datatype.BigDecimalSerializer.deserialize(
>> BigDecimalSerializer.java:74)
>> at org.apache.kylin.metadata.datatype.BigDecimalSerializer.deserialize(
>> BigDecimalSerializer.java:33)
>> at org.apache.kylin.cube.gridtable.CubeCodeSystem.decodeColumnValue(
>> CubeCodeSystem.java:133)
>> at org.apache.kylin.gridtable.GTRecord.getValues(GTRecord.java:129)
>> at org.apache.kylin.storage.gtrecord.CubeTupleConverter.translateResult(
>> CubeTupleConverter.java:207)
>> at org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(
>> SegmentCubeTupleIterator.java:100)
>> at com.google.common.collect.Iterators$5.hasNext(Iterators.java:596)
>> at org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(
>> SequentialCubeTupleIterator.java:84)
>> at org.apache.kylin.query.enumerator.OLAPEnumerator.
>> moveNext(OLAPEnumerator.java:68)
>> at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.
>> next(Linq4j.java:673)
>>
>>
>> How can I solve the problem,thank you!

Reply via email to