GitHub user eowhadi opened a pull request:
https://github.com/apache/incubator-trafodion/pull/422
[TRAFODION-1421] Implement parallel scanner primitive
new CQD: hbase_dop_parallel_scanner
- default 0.0 = disabled
- from 0.x to 1.0 -> the DOP will be dynamically assessed as a percentage
of region to scan. If 10 region, and cqd is .3 (30%), then 3 threads will be
used...
- from 2.0 and above, will take ceiling, and use it as fixed number of
threads.
Advantage of parallel scanner parallelism over ESP is on resource
consumption. Parallel scanner is just creating threads, while ESP are full
proceses wth high memory consumption.
For now, this feature is just a primitive that can be manually exercised.
The next obvious step is to make the compiler aware of it and pick it over ESP
DOP when appropriate...
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/eowhadi/incubator-trafodion parallelScan
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/422.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #422
----
commit a7f9a628179fcd4077a566f558e653b1380eabc6
Author: Eric Owhadi <[email protected]>
Date: 2016-04-11T16:59:23Z
[TRAFODION-1421] Implement parallel scanner primitive
new CQD: hbase_dop_parallel_scanner
- default 0.0 = disabled
- from 0.x to 1.0 -> the DOP will be dynamically assessed as a percentage
of region to scan. If 10 region, and cqd is .3 (30%), then 3 threads will be
used...
- from 2.0 and above, will take ceiling, and use it as fixed number of
threads.
Advantage of parallel scanner parallelism over ESP is on resource
consumption. Parallel scanner is just creating threads, while ESP are full
proceses wth high memory consumption.
For now, this feature is just a primitive that can be manually exercised.
The next obvious step is to make the compiler aware of it and pick it over ESP
DOP when appropriate...
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
