[
https://issues.apache.org/jira/browse/TEZ-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Jaiprakash updated TEZ-1187:
-----------------------------------
Attachment: TEZ-1187.02.patch
Thanks [~sseth]. Sorry, I implemented most of the code. Majority of the changes
are for framework to propagate the service. The other places its replacing
Executors.newFixed* with context.createExecutorService(...);
* Simplified TezSharedExecutor, currently it has singleton for app use and
public constructor for test cases. Although its a singleton, I've chosen to
propagate it so that we can change in one place, if required.
* Removed ScheduledExecutorService capabilities since it was making
TezSharedExecutor too complex.
* Every ExecutorService returned by TezSharedExecutor have their own task queue
and ensures only the specified level of parallel tasks are executed.
* Created a method in TaskContext to be able to create an ExecutorService, this
is available in InputContext, ProcesserContext and OutputContext.
* Replaced creation of Executors.newFixed* with executorService from
TezSharedExecutors.
* Just started on tests for TezSharedExecutor, attaching patch to see if this
makes sense.
> Share Thread pools between different tasks
> ------------------------------------------
>
> Key: TEZ-1187
> URL: https://issues.apache.org/jira/browse/TEZ-1187
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Harish Jaiprakash
> Attachments: TEZ-1187.02.patch, TEZ-1187.WIP.01.patch
>
>
> Thread pools are used all over for fetchers, sort etc. When running a single
> task - this is already a problem, and gets worse when running multiple tasks.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
