GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/incubator-trafodion/pull/929
[TRAFODION-2455] Add retry to row count estimation logic
This set of changes does the following:
1. Changes the stack from NATable::estimateHBaseRowCount on down to return
detailed error information about any failure.
2. Changes NATable::estimateHBaseRowCount to return a rowcount of 100
million instead of zero when an error occurs. It is safer to overestimate the
size of an object than to underestimate it.
3. Changes UPDATE STATISTICS to give an error 9252 "Unable to get row count
estimate: Error code $0int0, detail $1int1. Exception info (if any):
$0~string0" when an error occurs in or underneath
NATable::estimateHBaseRowCount instead of using a rowcount estimate of zero.
The information in this message gives details such as what error path was
taken, and any Java exception information that may be pertinent.
4. Adds a retry loop to HBaseClient.java method estimateRowCount so that we
retry if we encounter a FileNotFoundException. UPDATE STATISTICS will do up to
4 minutes worth of accumulated retries; normal compilation will do up to 5
seconds worth of accumulated retries. Wait times for retries start out at 2
seconds, doubling until topping out at 30 seconds.
5. Adds timestamps to messages in update statistics logging in local time.
To get local time, I needed to fix a bug in the monitor (process.cxx) that was
incorrectly setting an environment variable TZ to the empty string when TZ was
not defined in the monitor itself.
Note: The fix in process.cxx might (or might not!) fix a similar bug in DTM
logging where local timestamps are desired but UTC timestamps are produced.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/incubator-trafodion
Trafodion2455x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/929.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #929
----
commit 932c219f0fd2a49e1c53a666a1dcbcb3578c4f91
Author: Dave Birdsall <[email protected]>
Date: 2017-01-19T20:10:33Z
[TRAFODION-2440] Add retry to row count estimation logic
commit f1653636dd6d411fafe79697ee75c8763ff8edf2
Author: Dave Birdsall <[email protected]>
Date: 2017-01-24T00:09:08Z
Comment change
commit dbc6c0876ca5af07b63d68f388bad07195d106a5
Author: Dave Birdsall <[email protected]>
Date: 2017-01-25T22:40:56Z
[TRAFODION-2455] More refinements to row count estimation retry logic
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
