[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Reviewed-on: http://gerrit.cloudera.org:8080/14049 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 162 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Verified Vihang Karajgaonkar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 6 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 5: Code-Review+2 Carrying forward Anurag’s +1. I suspect the previous test failure is caused due to a race condition in the test itself. I will investigate it separately. This change is pretty simple and its scope is limited to event processor. I am going to go ahead and merge the change by self-reviewing in addition to Anurag’s +1 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 07:00:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 02:12:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4792/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 14 Aug 2019 22:03:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4775/ -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 13 Aug 2019 02:42:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4225/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:36:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4775/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:35:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Code-Review+1 Looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:17:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 162 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/4 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@181 PS3, Line 181: def wait_for_insert_event_processing(self, previous_event_id): > I like that you moved out EventProcessorUtils. However, this file heavily u yes, I agree. I am planning to clean it up as part of IMPALA-8795. This method had some custom logic in place where it was waiting for 2 events so didn't want to touch that to keep the scope minimum. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@200 PS3, Line 200: get_event_processor_metrics > Same as my above comment, this function can be removed here and modify this I am planning to clean up all the event polling tests together in IMPALA-8795 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@215 PS3, Line 215: get_last_synced_event_id > Same as previous comment. I am planning to clean up all the event polling tests together in IMPALA-8795 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@27 PS3, Line 27: class EventProcessorUtils(object): > flake8: E302 expected 2 blank lines, found 1 Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@30 PS3, Line 30: D > flake8: E303 too many blank lines (2) Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@50 PS3, Line 50: . > flake8: E131 continuation line unaligned for hanging indent Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@56 PS3, Line 56: @ > flake8: E303 too many blank lines (2) Done -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:55:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4224/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:51:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: Code-Review+1 +1 to the change in general. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:30:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (3 comments) Overall, looks good to me. I like moving EventProcessorUtils out. If not in this change, would be nice to create a JIRA to clean up test_event_processing.py. We can use EventProcessorUtils for insert events too. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@181 PS3, Line 181: def wait_for_insert_event_processing(self, previous_event_id): I like that you moved out EventProcessorUtils. However, this file heavily uses the same utils. Maybe we can create another JIRA to clean up this file. Especially, tests for insert events can be modified to use EventProcessorUtils functions. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@200 PS3, Line 200: get_event_processor_metrics Same as my above comment, this function can be removed here and modify this file to use EventProcessorUtils. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@215 PS3, Line 215: get_last_synced_event_id Same as previous comment. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:30:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@27 PS3, Line 27: class EventProcessorUtils(object): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@30 PS3, Line 30: D flake8: E303 too many blank lines (2) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@50 PS3, Line 50: . flake8: E131 continuation line unaligned for hanging indent http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@56 PS3, Line 56: @ flake8: E303 too many blank lines (2) -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:11:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4223/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 21:10:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 163 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/3 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 2: (8 comments) http://gerrit.cloudera.org:8080/#/c/14049/2/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/2/tests/custom_cluster/test_event_processing.py@156 PS2, Line 156: > line has trailing whitespace Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@22 PS2, Line 22: import logging > flake8: F401 'logging' imported but unused Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@28 PS2, Line 28: class EventProcessorUtils(object): > flake8: E302 expected 2 blank lines, found 1 Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@31 PS2, Line 31: @ > flake8: E301 expected 1 blank line, found 0 Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@47 PS2, Line 47: T > flake8: F821 undefined name 'TimeoutError' Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@49 PS2, Line 49: . > flake8: E131 continuation line unaligned for hanging indent Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@53 PS2, Line 53: > line has trailing whitespace Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@62 PS2, Line 62: > line has trailing whitespace Done -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:10:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14049 Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 162 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/2 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar
