meiyoula created SPARK-9585:
-------------------------------
Summary: HiveHBaseTableInputFormat can'be cached
Key: SPARK-9585
URL: https://issues.apache.org/jira/browse/SPARK-9585
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: meiyoula
Below exception occurs in Spark On HBase function.
{quote}
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException:
Task
org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@11c6577
rejected from java.util.concurrent.ThreadPoolExecutor@3414350b[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 17451]
{quote}
When an executor has many cores, the tasks belongs to same RDD will using the
same InputFormat. But the HiveHBaseTableInputFormat is not thread safety.
So I think we should add a config to enable cache InputFormat or not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
