IMPALA-4450: qgen: use string concatenation operator for postgres queries The random query generator writes a logical query Python object into Impala or PostgreSQL dialects. When the CONCAT() function is chosen, Impala's and PostgreSQL's CONCAT() implementations behave differently. However, PostgreSQL has a || operator that functions like Impala's CONCAT().
The method added here overrides the default behavior for the PostgresqlSqlWriter. It prevents CONCAT(arg1, arg2, ..., argN) from being written and instead causes the SQL to be written as 'arg1 || arg2 || ... || argN'. Testing: I made sure that we generate syntactically valid queries still on the PostgreSQL side. This includes queries that made use of string concatenation. I also re-ran some failed queries that previously produced different results. They now produce the same results. This is a very straightforward change, so unit or functional tests for this seem overkill. The full effects of using || instead of CONCAT() are hard to test. It's not clear if in my manual testing of || vs. CONCAT() that I missed some edge behavior, especially in some complicated query, nested expressions, GROUPing BY, and so on. Change-Id: I149b695889addfd7df4ca5f40dc991456da51687 Reviewed-on: http://gerrit.cloudera.org:8080/5034 Reviewed-by: Michael Brown <[email protected]> Reviewed-by: Tim Armstrong <[email protected]> Reviewed-by: Taras Bobrovytsky <[email protected]> Tested-by: Sailesh Mukil <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/585ed5aa Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/585ed5aa Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/585ed5aa Branch: refs/heads/master Commit: 585ed5aaae32607218e8056e34b40de0e24c1724 Parents: 0f62bf3 Author: Michael Brown <[email protected]> Authored: Wed Nov 9 16:07:37 2016 -0800 Committer: Sailesh Mukil <[email protected]> Committed: Wed Nov 30 16:22:56 2016 +0000 ---------------------------------------------------------------------- tests/comparison/model_translator.py | 6 ++++++ 1 file changed, 6 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/585ed5aa/tests/comparison/model_translator.py ---------------------------------------------------------------------- diff --git a/tests/comparison/model_translator.py b/tests/comparison/model_translator.py index 202b5ea..b3d4a97 100644 --- a/tests/comparison/model_translator.py +++ b/tests/comparison/model_translator.py @@ -648,6 +648,12 @@ class PostgresqlSqlWriter(SqlWriter): return "'%s' || ''" % data_type.val return SqlWriter._write_data_type(self, data_type) + def _write_concat(self, func): + # PostgreSQL CONCAT() doesn't behave like Impala CONCAT(). PostgreSQL || does. + return '({concat_list})'.format( + concat_list=' || '.join(['({written_item})'.format(written_item=self._write(item)) + for item in func.args])) + class MySQLSqlWriter(SqlWriter):
