Repository: incubator-impala Updated Branches: refs/heads/master 0aeb68050 -> c68734ad6
IMPALA-4461: Make sure data gets loaded for wide hbase tables. Ths patch reverts a change that broke the exhaustive suite of Impala tests. The change was introduced here: https://github.com/apache/incubator-impala/commit/ce4c5f67433ebe050261710920972621d625c81c The orginal problem was that data load was failing when run against a remote cluster, due to a 4000 byte max for SERDEPROPERTIES.PARAM_VALUE, a limitation that is well described in HIVE-1364. Locally, when we load data, we work around the issue here: https://github.com/apache/incubator-impala/blob/master/bin/create-test-configuration.sh#L99 When testing on CDH remote cluster however, this "fix" never gets applied. (It also assumes the database will always by postgres.) I made this change without realizing its full effect, or appreciating exactly how exhaustive our exhaustive test suite really is. Another solution will need to be found for the case of remote cluster testing, but this should unblock the local build for now. As far as testing, I ran the full suite of tests in query_test/ test_scanners.py, and they all pass after removing these lines. Change-Id: If2148d6546789c6c53c8e045717081b24ce76689 Reviewed-on: http://gerrit.cloudera.org:8080/5033 Reviewed-by: Dan Hecht <[email protected]> Tested-by: Internal Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/b14f3197 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/b14f3197 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/b14f3197 Branch: refs/heads/master Commit: b14f319708943b14680c77b2afbc502a056d06a1 Parents: 0aeb680 Author: David Knupp <[email protected]> Authored: Wed Nov 9 17:55:46 2016 -0800 Committer: Internal Jenkins <[email protected]> Committed: Fri Nov 11 00:37:59 2016 +0000 ---------------------------------------------------------------------- testdata/datasets/functional/schema_constraints.csv | 5 ----- 1 file changed, 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/b14f3197/testdata/datasets/functional/schema_constraints.csv ---------------------------------------------------------------------- diff --git a/testdata/datasets/functional/schema_constraints.csv b/testdata/datasets/functional/schema_constraints.csv index 790dba4..d6d1111 100644 --- a/testdata/datasets/functional/schema_constraints.csv +++ b/testdata/datasets/functional/schema_constraints.csv @@ -118,11 +118,6 @@ table_name:overflow, constraint:restrict_to, table_format:text/none/none # seem to like this. table_name:widerow, constraint:exclude, table_format:hbase/none/none -# Wide tables fail due to the SERDEPROPERTIES limits. See HIVE-1364. -table_name:widetable_250_cols, constraints:exclude, table_format:hbase/none/none -table_name:widetable_500_cols, constraints:exclude, table_format:hbase/none/none -table_name:widetable_1000_cols, constraints:exclude, table_format:hbase/none/none - # nullformat_custom is used in null-insert tests, which user insert overwrite, # which is not supported in hbase. The schema is also specified in HIVE_CREATE # with no corresponding LOAD statement.
