Re: [Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Such a syntax error may occur if a script is updated while it's running. 2017-09-21 3:03 GMT+08:00 Tianyi Wang (Code Review): > Tianyi Wang has posted comments on this change. > > Change subject: IMPALA-5416: Fix an impala-shell command recursion bug > .. > > > Patch Set 4: > > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1238/ > > The build failed on building stage. The error message is > "./bin/jenkins/build-all-flag-combinations.sh: line 59: syntax error near > unexpected token `fi". Pretty strange since the script is not touched and > it builds on my machine. > > -- > To view, visit http://gerrit.cloudera.org:8080/8063 > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings > > Gerrit-MessageType: comment > Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 > Gerrit-PatchSet: 4 > Gerrit-Project: Impala-ASF > Gerrit-Branch: master > Gerrit-Owner: Tianyi Wang > Gerrit-Reviewer: Alex Behm > Gerrit-Reviewer: Impala Public Jenkins > Gerrit-Reviewer: Philip Zeyliger > Gerrit-Reviewer: Thomas Tauber-Marshall > Gerrit-Reviewer: Tianyi Wang > Gerrit-HasComments: No >
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8063 ) Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. IMPALA-5416: Fix an impala-shell command recursion bug Impala-shell crashes with 2 source commands on the same line and runs a command multiple times if it shares the same line with a source command. The bug is caused by a misuse of cmdqueue. The cmdqueue member of cmd.Cmd is used to execute commands not directly from user input in an event loop. When a 'source' is run, execute_query_list() is called which also executes the commands in cmdqueue, causing them to be executed twice. The fix is for execute_query_list() to not run the commands in cmdqueue. For the non-interactive case, where the event loop won't be run, we call execute_query_list() with cmdqueue so that the commands get run. A test case is added to test_shell_interactive.py. Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Reviewed-on: http://gerrit.cloudera.org:8080/8063 Reviewed-by: Alex BehmTested-by: Impala Public Jenkins --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 10 insertions(+), 4 deletions(-) Approvals: Alex Behm: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-Change-Number: 8063 Gerrit-PatchSet: 8 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8063 ) Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-Change-Number: 8063 Gerrit-PatchSet: 7 Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Thu, 21 Sep 2017 21:41:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8063 ) Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1252/ -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-Change-Number: 8063 Gerrit-PatchSet: 7 Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Thu, 21 Sep 2017 17:36:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/8063 ) Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-Change-Number: 8063 Gerrit-PatchSet: 7 Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Thu, 21 Sep 2017 17:36:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/8063 ) Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 7: Rebased again. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-Change-Number: 8063 Gerrit-PatchSet: 7 Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Thu, 21 Sep 2017 17:31:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 5: > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1242/ It seems to be related to IMPALA-5920. I will do a rebasing again. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1242/ -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1242/ -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Alex Behm has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 5: Code is rebased. Please build again. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 4: > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1238/ The build failed on building stage. The error message is "./bin/jenkins/build-all-flag-combinations.sh: line 59: syntax error near unexpected token `fi". Pretty strange since the script is not touched and it builds on my machine. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1238/ -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1238/ -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Alex Behm has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Thomas Tauber-Marshall has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 2: (1 comment) > Can this be tested in tests/shell/test_shell_interactive.py? Sure. A test case is extended. http://gerrit.cloudera.org:8080/#/c/8063/2//COMMIT_MSG Commit Message: PS2, Line 12: The cause is that there is a "cmdqueue" member in cmd library, which is : used to execute commands not directly from user input. When impala-shell : reads a line with multiple commands, it splits the line into multiple : queries and insert them into the queue, and then gives control back to : the eventloop in cmd library. The problem is that a source command : calls execute_query_list(), which executes queries in the cmdqueue as : well. So any query in the cmdqueue will be executed twice. And if there : is unfortunately a source command in the cmdqueue, it will call : execute_query_list() again, and there will be an infinite recursion. : The original purpose of running queued queries in execute_query_list() : is that in non-interactive mode, there is no event loop. And still, : there are queries like "use database" queued by connection setup : procedures, which need to be run before the user query. : This patch avoids running queries from the queue in : execute_query_list(), and for non-interactive mode, runs queued queries : in execute_queries_non_interactive_mode() instead. > Thanks for the explanation. Thanks for the suggestion. Commit message updated. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has uploaded a new patch set (#4). Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. IMPALA-5416: Fix an impala-shell command recursion bug Impala-shell crashes with 2 source commands on the same line and runs a command multiple times if it shares the same line with a source command. The bug is caused by a misuse of cmdqueue. The cmdqueue member of cmd.Cmd is used to execute commands not directly from user input in an event loop. When a 'source' is run, execute_query_list() is called which also executes the commands in cmdqueue, causing them to be executed twice. The fix is for execute_query_list() to not run the commands in cmdqueue. For the non-interactive case, where the event loop won't be run, we call execute_query_list() with cmdqueue so that the commands get run. A test case is added to test_shell_interactive.py. Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 10 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/8063/4 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has uploaded a new patch set (#3). Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. IMPALA-5416: Fix an impala-shell command recursion bug Impala-shell crashes with 2 source commands on the same line and runs a command multiple times if it shares the same line with a source command. The bug is caused by a misuse of cmdqueue. The cmdqueue member of cmd.Cmd is used to execute commands not directly from user input in an event loop. When a 'source' is run, execute_query_list() is called which also executes the commands in cmdqueue, causing them to be executed twice. The fix is for execute_query_list() to not run the commands in cmdqueue. For the non-interactive case, where the event loop won't be run, we call execute_query_list() with cmdqueue so that the commands get run. Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 10 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/8063/3 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Philip Zeyliger has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 2: Can this be tested in tests/shell/test_shell_interactive.py? -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Thomas Tauber-Marshall has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/8063/2//COMMIT_MSG Commit Message: PS2, Line 12: The cause is that there is a "cmdqueue" member in cmd library, which is : used to execute commands not directly from user input. When impala-shell : reads a line with multiple commands, it splits the line into multiple : queries and insert them into the queue, and then gives control back to : the eventloop in cmd library. The problem is that a source command : calls execute_query_list(), which executes queries in the cmdqueue as : well. So any query in the cmdqueue will be executed twice. And if there : is unfortunately a source command in the cmdqueue, it will call : execute_query_list() again, and there will be an infinite recursion. : The original purpose of running queued queries in execute_query_list() : is that in non-interactive mode, there is no event loop. And still, : there are queries like "use database" queued by connection setup : procedures, which need to be run before the user query. : This patch avoids running queries from the queue in : execute_query_list(), and for non-interactive mode, runs queued queries : in execute_queries_non_interactive_mode() instead. Thanks for the explanation. Hate to be picky, but this is more technical detail than I was looking for. The idea here is to give the reader a high level idea of what's going on while keeping it simple. Maybe something like (assuming I understand completely): The cmdqueue member of cmd.Cmd is used to execute commands not directly from user input in an event loop. When a 'source' is run, execute_query_list() is called which also executes the commands in cmdqueue, causing them to be executed twice. The fix is for execute_query_list() to not run the commands in cmdqueue. For the non-interactive case, where the event loop won't be run, we call execute_query_list() with cmdqueue so that the commands get run. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/8063/1//COMMIT_MSG Commit Message: PS1, Line 11: This patch fixes these bugs. > Can you describe briefly what the problem with the existing code was? Commit message updated. It is basically a wired usage of a queue. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has uploaded a new patch set (#2). Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. IMPALA-5416: Fix an impala-shell command recursion bug Impala-shell crashes with 2 source commands on the same line and runs a command multiple times if it shares the same line with a source command. The cause is that there is a "cmdqueue" member in cmd library, which is used to execute commands not directly from user input. When impala-shell reads a line with multiple commands, it splits the line into multiple queries and insert them into the queue, and then gives control back to the eventloop in cmd library. The problem is that a source command calls execute_query_list(), which executes queries in the cmdqueue as well. So any query in the cmdqueue will be executed twice. And if there is unfortunately a source command in the cmdqueue, it will call execute_query_list() again, and there will be an infinite recursion. The original purpose of running queued queries in execute_query_list() is that in non-interactive mode, there is no event loop. And still, there are queries like "use database" queued by connection setup procedures, which need to be run before the user query. This patch avoids running queries from the queue in execute_query_list(), and for non-interactive mode, runs queued queries in execute_queries_non_interactive_mode() instead. Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 --- M shell/impala_shell.py 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/8063/2 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Thomas Tauber-Marshall has posted comments on this change. Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/8063/1//COMMIT_MSG Commit Message: PS1, Line 11: This patch fixes these bugs. Can you describe briefly what the problem with the existing code was? Looking at the code change, my intuition is that the old and new code should be equivalent, but apparently that's not the case. -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi WangGerrit-Reviewer: Thomas Tauber-Marshall Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5416: Fix an impala-shell command recursion bug
Tianyi Wang has uploaded a new change for review. http://gerrit.cloudera.org:8080/8063 Change subject: IMPALA-5416: Fix an impala-shell command recursion bug .. IMPALA-5416: Fix an impala-shell command recursion bug Impala-shell crashes with 2 source commands on the same line and runs a command multiple times if it shares the same line with a source command. This patch fixes these bugs. Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 --- M shell/impala_shell.py 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/8063/1 -- To view, visit http://gerrit.cloudera.org:8080/8063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I453af2d4694d47e184031cb07ecd2af259ba20f3 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi Wang