[
https://issues.apache.org/jira/browse/TEZ-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247409#comment-14247409
]
Bikas Saha commented on TEZ-1789:
---------------------------------
Thanks for the reviews!
bq. Instead of Vertex.handleSpeculatorEvent. Should this just be
Vertex.getSpeculator()
Dont want to do this as IMO, for now, its better to keep the scope of
speculator within the vertex and let it do its thing. The speculator interfaces
are still not defined properly and probably will be when we write a better
speculator in the next version.
I will open a new jira to track changes to make our own version of the
dispatcher. sub-jiras to track naming of threads and handling multiple queues.
This is related not just to this jira but also to moving other processing off
the central dispatcher.
bq. This approach ends up using the queueing and threading mechanism from a new
dispatcher - which I think is also fine.
Yeah. I like this better because its common infra that everyone can use instead
of having to implement separately and also can end up using less threads as
opposed to the current scheme of 1 thread per entity.
I will commit this by eod unless there are further comments. Then I will follow
up on the dispatcher items.
> Move speculator processing off the central dispatcher
> -----------------------------------------------------
>
> Key: TEZ-1789
> URL: https://issues.apache.org/jira/browse/TEZ-1789
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1789.1.patch
>
>
> There may be a large number of such events and speculators may have expensive
> logic while handling them. Moving off the central dispatcher will keep this
> off critical execution path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
