IMPALA-7490: fix uninitialized variables in load-data.py
Fixes use of an uninitialized variable in bin/load-data.py
I found the following error message in a failed build, which is quite
misleading:
Traceback (most recent call last):
File "bin/load-data.py", line 495, in <module>
if __name__ == "__main__": main()
File "bin/load-data.py", line 459, in main
impala_exec_query_files_parallel(thread_pool, impala_create_files)
File "bin/load-data.py", line 297, in impala_exec_query_files_parallel
exec_query_files_parallel(thread_pool, query_files, 'impala')
File "bin/load-data.py", line 291, in exec_query_files_parallel
for result in thread_pool.imap_unordered(execution_function, query_files):
File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
raise value
UnboundLocalError: local variable 'query' referenced before assignment
The error is trown from the 'execution_function' which is
'exec_impala_query_from_file' in my case. Should not use 'query' if it's
undefined.
Change-Id: If0dd56a9b78a60b3a9f04d9f61e93b4b5d066b76
Reviewed-on: http://gerrit.cloudera.org:8080/11330
Reviewed-by: Quanlong Huang <huangquanl...@gmail.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/357c0a95
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/357c0a95
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/357c0a95
Branch: refs/heads/master
Commit: 357c0a959d1a3a9cb2662773a407acfee3ca6fe2
Parents: fe613e3
Author: stiga-huang <huangquanl...@gmail.com>
Authored: Sat Aug 25 19:09:23 2018 -0700
Committer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Committed: Sat Sep 1 12:00:43 2018 +0000
----------------------------------------------------------------------
bin/load-data.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/impala/blob/357c0a95/bin/load-data.py
----------------------------------------------------------------------
diff --git a/bin/load-data.py b/bin/load-data.py
index 7754147..8878372 100755
--- a/bin/load-data.py
+++ b/bin/load-data.py
@@ -195,6 +195,7 @@ def exec_impala_query_from_file(file_name):
is_success = True
impala_client = ImpalaBeeswaxClient(options.impalad,
use_kerberos=options.use_kerberos)
output_file = file_name + ".log"
+ query = None
with open(output_file, 'w') as out_file:
try:
impala_client.connect()
@@ -206,7 +207,10 @@ def exec_impala_query_from_file(file_name):
result = impala_client.execute(query)
out_file.write("{0}\n{1}\n".format(query, result))
except Exception as e:
- out_file.write("ERROR: {0}\n".format(query))
+ if query:
+ out_file.write("ERROR: {0}\n".format(query))
+ else:
+ out_file.write("Encounter errors before parsing any queries.\n")
traceback.print_exc(file=out_file)
is_success = False
