[
https://issues.apache.org/jira/browse/LENS-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339904#comment-14339904
]
Jaideep Dhok commented on LENS-356:
-----------------------------------
I don't think the solution to use caller thread is too complex.
{code}
// In the caller thread
Driver firstDriver = driverList.get(0);
if (driverList.size() > 1) {
CountDownLatch asyncEstimateLatch = new CountDownLatch(driverList.size() - 1);
List<Future> futureList = ...
for (int i = 1; i < driverList.size(); i++) {
// the runnable should decrement latch
futureList.add(estimatePool.submit(asyncRunnable));
}
// Estimate first driver
esitmate(firstDriver);
// Wait for other drivers, with timeout
asyncEstimateLatch.await(timeout, unit);
// Go through future list to get results of other estimate calls
for (Future f : futureList) {
// consume estimate result....
}
} else {
estimate(firstDriver);
}
{code}
I think we should utilize the request processing (Grizzly) threads, since
otherwise that thread is just waiting for the pool threads to finish. Also, the
request processing thread has been already allocated so we might as well
utilize it.
At a time we would be using N-1 from the estimate pool, in our case N = 2, so
only one thread from the pool.
About context switching, we need not worry about it. That cost should be in
order of microseconds if not nanoseconds. Unless we are switching hundreds
thousands of time during an estimate call there should be no reason to worry.
> Run estimate for each driver in parallel
> -----------------------------------------
>
> Key: LENS-356
> URL: https://issues.apache.org/jira/browse/LENS-356
> Project: Apache Lens
> Issue Type: Sub-task
> Components: server
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Fix For: 2.1
>
>
> We are estimating the cost of query on each driver sequentially. Since each
> estimate is independent those can be parallelized.
> For each driver rewrite the query, call driver.estimate - need to parallelize
> this for all drivers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
