Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18329 )
Change subject: IMPALA-11192: Batch uploading files in test_scanner_fuzz.py ...................................................................... IMPALA-11192: Batch uploading files in test_scanner_fuzz.py test_scanner_fuzz.py runs much slower on ORC than other formats. The majority of the time is spent in uploading local files one by one to the hdfs table folder. The local files are copied from hdfs and randomly corrupted by the test. The directory layout remains the same as the table folder. There are no staging dirs that we should skip. So we can upload the whole local folder at once, which saves a lot of the test time. Tested locally and verified profiles of the succeeded queries. They all scan the expected number of rows. Change-Id: I504e160b84b3cc01d3be0b4e242d3c372692d181 Reviewed-on: http://gerrit.cloudera.org:8080/18329 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M tests/query_test/test_scanners_fuzz.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18329 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I504e160b84b3cc01d3be0b4e242d3c372692d181 Gerrit-Change-Number: 18329 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
