GitHub user liming01 opened a pull request:
https://github.com/apache/incubator-hawq/pull/1187
HAWQ-1408. Fixed crash when alloc not enough segno for write
In case of there is no reproduce steps, I only went through the code to
find out possible root cause. Now I get one possible cause. The possible
reason is related https://issues.apache.org/jira/browse/HAWQ-642.
This fix includes below changes:
(1) When keepHash, we can not guarantee that only generate remaining_num is
enough to alloc seg file for all segment_num. Because the next new generated
seg file number may have same hash key id with the old one.
(2) When !keepHash, now the remaining_num returned from addCandidateSegno()
is not precise. So we need to fix it to meet the need of HAWQ-642.
(3) At the final call to addCandidateSegno(), we should keep monitoring the
remaining_num instead of at the beginning because of the reason (1). So that
even the new seg file is not actually used by this query ( maybe for hash key
conflict), we can continue to alloc enough seg files for this query.
@ictmalili @zhangh43, please review it. Thanks.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liming01/incubator-hawq
mli/copy_crash_segno4write
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/1187.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1187
----
commit ecb2255f836ead8579f3b681336ede3ed767c0cf
Author: Ming LI <[email protected]>
Date: 2017-03-24T03:43:56Z
HAWQ-1408. Fixed crash when alloc not enough segno for write
This fix includes below changes:
(1) When keepHash, we can not guarantee that only generate remaining_num is
enough to alloc seg file for all segment_num. Because the next new generated
seg file number may have same hash key id with the old one.
(2) When !keepHash, now the remaining_num returned from addCandidateSegno()
is not precise. So we need to fix it to meet the need of HAWQ-642.
(3) At the final call to addCandidateSegno(), we should keep monitoring the
remaining_num instead of at the beginning because of the reason (1). So that
even the new seg file is not actually used by this query ( maybe for hash key
conflict), we can continue to alloc enough seg files for this query.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
