[
https://issues.apache.org/jira/browse/TRAFODION-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643324#comment-14643324
]
Eric Owhadi commented on TRAFODION-1421:
----------------------------------------
Anoop, you are talking about "merging sorted streams": In what I was going to
implement the stream seen by ESP or Master executor would not be multiple
streams, but a single stream of unsorted data (not random data, but intermingle
of multiple regions scanned in parallel data in a single stream. So for
operators that needs sorted stream, that parallel scanner would not be
appropriate.
Hope this is still useful ? I guess it is since you would get multi-threading
parallelism on top of ESP (multi process parallelism)?
> Implement parallel Scanner primitive
> ------------------------------------
>
> Key: TRAFODION-1421
> URL: https://issues.apache.org/jira/browse/TRAFODION-1421
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp, sql-exe
> Reporter: Eric Owhadi
> Assignee: Eric Owhadi
> Labels: performance
> Fix For: 2.0-incubating
>
>
> ClientScanner API is serial, to conserve key ordering. However, many
> operators don't care about ordering and would rather get the scan result
> fast, regardless of order. This JIRA is about providing a parallel scanner,
> that would take care of splitting the work between all region servers evenly
> if possible. HBase had a parallel scanner in the pipe for quite some time
> HBAse-9272, but the work is stalled since october 2013. However, looking at
> the available code, look like a big part can be leveraged without requiring
> an HBase custom build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
