Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11000 )
Change subject: IMPALA-5542: Impala cannot scan Parquet decimal stored as int64_t/int32_t ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/11000/1/tests/query_test/test_scanners.py File tests/query_test/test_scanners.py: http://gerrit.cloudera.org:8080/#/c/11000/1/tests/query_test/test_scanners.py@293 PS1, Line 293: def _create_table_from_file(self, table_name, unique_database): > It would be nice to use this for other tests (no requirement, just an obser I have created a related ticket in the past: https://issues.apache.org/jira/browse/IMPALA-6709 I agree that it would be nice to change other tests to use this function. I would prefer to do this in a different commit. http://gerrit.cloudera.org:8080/#/c/11000/1/tests/query_test/test_scanners.py@298 PS1, Line 298: hdfs_file = '/test-warehouse/%s' % filename : check_call(['hdfs', 'dfs', '-copyFromLocal', '-f', local_file, hdfs_file]) I am a bit worried about using /test-warehouse/. Your file names are unlikely to be used elsewhere, but someone may create a test with a more common file name, which could lead to side-effects. My suggestion is to put these files to the directory of the unique database: "/test-warehouse/{0}.db/{1}".format(unique_database, filename) Another option would be to load these files to hdfs during data load (see create-load-data.sh: load-custom-data). This could make the tests faster, but would make developing new tests less convenient. http://gerrit.cloudera.org:8080/#/c/11000/1/tests/query_test/test_scanners.py@302 PS1, Line 302: self.client.execute('create table %s like parquet "%s" stored as parquet' % +1 for doing this with CREATE TABLE LIKE PARQUET, because it is easy to forget adding tests for it -- To view, visit http://gerrit.cloudera.org:8080/11000 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8c41bfc7c1664bdba5099d3893dc8dbe4304794 Gerrit-Change-Number: 11000 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 24 Jul 2018 13:31:41 +0000 Gerrit-HasComments: Yes
