[
https://issues.apache.org/jira/browse/TEZ-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4253:
------------------------------
Affects Version/s: 0.10.0
> Revert TEZ-4170
> ---------------
>
> Key: TEZ-4253
> URL: https://issues.apache.org/jira/browse/TEZ-4253
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Mustafa İman
> Assignee: Mustafa İman
> Priority: Major
> Fix For: 0.10.1
>
> Attachments: TEZ-4253.2.patch, TEZ-4253.2.patch, TEZ-4253.2.patch,
> TEZ-4253.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are two performance improvements in
> https://issues.apache.org/jira/browse/TEZ-4170
> # Move construction of InputInitializers to background thread
> # Remove RootInputInitializerManager's thread pool and move all threads
> using this executor to DagAppMaster's thread pool.
> Item 1: This is an incorrect optimization which may cause data races in
> VertexImpl regarding handling of events. This was mitigated in
> https://issues.apache.org/jira/browse/TEZ-4204 however the solution basically
> reverts back the initial optimization only with a more complicated approach.
> Apart from these, it unnecessarily complicates Tez application master. The
> scenario where this is useful is when a custom InputInitializer constructor
> contains a lot of heavyweight operations. However, the solution to this
> problem belongs to client application. Client application can easily move
> heavyweight operations to InputInitializer#initialize method.
> Item 2: The benefit of this is doubtful. Supposedly a root input initializer
> is offloaded to a cached thread in appcontext instead of creating a new one
> in RootInputInitializerManager. The number of threads of this pool is
> limited. When many root input initializer depends on InputInitializerEvents
> all threads may get blocked. In that case the rest of the vertices (which are
> supposed to send the InputInitializerEvents) cannot run. So we run into a
> deadlock.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
