Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11800 )
Change subject: IMPALA-6374: fix handling of commas in .test files ...................................................................... IMPALA-6374: fix handling of commas in .test files The .test file parser implemented an unconventional method for parsing single-quoted strings in comma-separated value format. This didn't handle trailing commas in the string correctly. This commit switches to using a conventional method for parsing comma-separated value format: * Commas enclosed by single quotes are not treated as field separators * Single quotes can be escaped within a string by doubling them. I looked into using Python's .csv module for this, but it wouldn't work without modifying the test file format more because it automatically discards the quotes during parsing, which are actually semantically important in .test files. E.g. without the quotes we can't distinguish between the literal string 'regex:...' and the regex regex:.... Testing: Ran exhaustive tests and fixed .test files that required modifications. Will rerun before merging. Added a couple of tests to exercise edge cases in the test file parser. Change-Id: I18ddcb0440490ddf8184be66d3681038a1615dd9 Reviewed-on: http://gerrit.cloudera.org:8080/11800 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Tim Armstrong <[email protected]> --- M testdata/workloads/functional-query/queries/QueryTest/functions-ddl.test M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test M testdata/workloads/functional-query/queries/QueryTest/hbase-inserts.test M testdata/workloads/functional-query/queries/QueryTest/misc.test A testdata/workloads/functional-query/queries/QueryTest/special-strings.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test M testdata/workloads/tpcds/queries/tpcds-decimal_v2-q98.test M testdata/workloads/tpcds/queries/tpcds-q98.test M tests/common/test_result_verifier.py M tests/query_test/test_exprs.py 10 files changed, 157 insertions(+), 112 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved Tim Armstrong: Verified -- To view, visit http://gerrit.cloudera.org:8080/11800 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I18ddcb0440490ddf8184be66d3681038a1615dd9 Gerrit-Change-Number: 11800 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Brown <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
