Yes, will add to test On Tue, Mar 29, 2016 at 3:55 PM, Matthew Jacobs <[email protected]> wrote:
> Can you make sure to parse the integer enum values as well as the string > names? > > On Tue, Mar 29, 2016 at 2:36 PM Skye Wanderman-Milne (Code Review) < > [email protected]> wrote: > >> Skye Wanderman-Milne has uploaded a new patch set (#6). >> >> Change subject: IMPALA-2835: introduce PARQUET_FALLBACK_SCHEMA_RESOLUTION >> query option >> ...................................................................... >> >> IMPALA-2835: introduce PARQUET_FALLBACK_SCHEMA_RESOLUTION query option >> >> This patch introduces a new query option, >> PARQUET_FALLBACK_SCHEMA_RESOLUTION which allows Parquet files' schemas >> to be resolved by either name or position. It's "fallback" because >> eventually field IDs will be the primary schema resolution scheme, and >> we don't want to create an option that we will have to change the name >> of later. The default is still by position. I chose to do a query >> option because it will make testing easier and also be easier to >> diagnose resolution problems quickly in the field. If users want to >> switch the default behavior to be by name (like Hive), they can use >> the --default_query_options flag. >> >> This patch also introduces a new test section, SHELL, which can be >> used to execute shell commands in a .test file. This is useful for >> copying files into test tables. >> >> Change-Id: Id0c715ea23792b2a6872610839a40532aabbb5a6 >> --- >> M be/src/exec/hdfs-parquet-scanner.cc >> M be/src/exec/hdfs-parquet-scanner.h >> M be/src/service/query-options.cc >> M be/src/service/query-options.h >> M common/thrift/ImpalaInternalService.thrift >> M common/thrift/ImpalaService.thrift >> A testdata/parquet_schema_resolution/README >> A testdata/parquet_schema_resolution/switched_map.avsc >> A testdata/parquet_schema_resolution/switched_map.json >> A testdata/parquet_schema_resolution/switched_map.parq >> A >> testdata/workloads/functional-query/queries/QueryTest/parquet-resolution-by-name.test >> M tests/common/impala_test_suite.py >> M tests/conftest.py >> M tests/query_test/test_scanners.py >> M tests/util/test_file_parser.py >> 15 files changed, 368 insertions(+), 18 deletions(-) >> >> >> git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/84/2384/6 >> -- >> To view, visit http://gerrit.cloudera.org:8080/2384 >> To unsubscribe, visit http://gerrit.cloudera.org:8080/settings >> >> Gerrit-MessageType: newpatchset >> Gerrit-Change-Id: Id0c715ea23792b2a6872610839a40532aabbb5a6 >> Gerrit-PatchSet: 6 >> Gerrit-Project: Impala >> Gerrit-Branch: cdh5-trunk >> Gerrit-Owner: Skye Wanderman-Milne <[email protected]> >> Gerrit-Reviewer: Dan Hecht <[email protected]> >> Gerrit-Reviewer: Juan Yu <[email protected]> >> Gerrit-Reviewer: Matthew Jacobs <[email protected]> >> Gerrit-Reviewer: Michael Ho <[email protected]> >> Gerrit-Reviewer: Silvius Rus <[email protected]> >> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> >> >
