[
https://issues.apache.org/jira/browse/HIVE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781388#comment-16781388
]
Zoltan Haindrich commented on HIVE-21312:
-----------------------------------------
with this declaration
{code}
private Queue<Map<String,Map<String,String>>> statsList;
{code}
statsList is a LIST?
this is the 3. time I'm saying this: Please use the executor's api to get the
return value and don't use other concurrent stuff for no good reason
{code}
leave statslist declaration as-is
[...]
+ for (final FileStatus file : status) {
+ futureList.add(pool.submit(new Callable<Void>() {
+ @Override public Void call() throws IOException {
+ Kryo kryo = null;
+ try (Input in = new Input(fs.open(file.getPath()))) {
+ kryo = SerializationUtilities.borrowKryo();
stats= kryo.readObject(in, statsMap.getClass());
Utilities.FILE_OP_LOGGER.trace("Read stats {}", stats);
+ return stats;
+ } finally {
+ SerializationUtilities.releaseKryo(kryo);
+ }
+ }
+ }));
[...]
+ for(Future<Void> future : futureList) {
+ statsList.add(future.get())
}
return true;
{code}
> FSStatsAggregator::connect is slow
> ----------------------------------
>
> Key: HIVE-21312
> URL: https://issues.apache.org/jira/browse/HIVE-21312
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Trivial
> Attachments: HIVE-21312.1.patch, HIVE-21312.2.patch,
> HIVE-21312.3.patch, HIVE-21312.4.patch, HIVE-21312.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
