liyang created KYLIN-1205:
-----------------------------
Summary: hbase RpcClient java.io.IOException: Unexpected closed
connection
Key: KYLIN-1205
URL: https://issues.apache.org/jira/browse/KYLIN-1205
Project: Kylin
Issue Type: Bug
Reporter: liyang
Assignee: liyang
during a scan of many rows(each row may be more than 64k), we encountered
"RpcClient java.io.IOException: Unexpected closed connection" a lot, this is
breaking kylin's normal functions.
if you open my log attached and grep "http-bio-7070-exec-11", you will find
this thread is:
1. launching several scans, this is because the kylin cube has several
segments, each segment correspond to a htable. A complete cube visit requires
to visit every htable related, so there're many scans established
2. One of the scanner(scanner_id: 538064847515747456 ) failed due
to:java.io.IOException: Unexpected closed connection
3. the error in step 2 lead to hbase client side/server side out of sync,
casuing org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException
4. hbase client retried with a different scanner id (still same htable, same
region) but still failed
can you please check at server side on what on earth is causing the "
Unexpected closed connection" it seems to be the root cause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
