[
https://issues.apache.org/jira/browse/LENS-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338341#comment-14338341
]
Amareshwari Sriramadasu commented on LENS-356:
----------------------------------------------
Here are some thoughts i have :
* Since estimate has to return few milli seconds, multiple thread context
switches should not make it actually take more time.
For multiple threads for estimate for each driver :
* Since the caller has to wait for cost to be estimated all drivers to be
finished, i'm thinking caller itself can estimate for some.
* Say we have a thread pool of size - 10, and two drivers in the server, and
caller estimates for one driver, then in parallel there can be 10 estimate
requests can be answered. If there are more queries being estimated, then
caller will pick next along with threads in pool, when is done for one driver.
* With N drivers in lens server, caller thread will estimate for one driver and
N-1 can be picked by threads in pool, if they are free.
I'll see if having caller to one estimate along with thread pool makes the code
complex and update again.
Thoughts?
> Run estimate for each driver in parallel
> -----------------------------------------
>
> Key: LENS-356
> URL: https://issues.apache.org/jira/browse/LENS-356
> Project: Apache Lens
> Issue Type: Sub-task
> Components: server
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Fix For: 2.1
>
>
> We are estimating the cost of query on each driver sequentially. Since each
> estimate is independent those can be parallelized.
> For each driver rewrite the query, call driver.estimate - need to parallelize
> this for all drivers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
