Can you make sure to parse the integer enum values as well as the string names? On Tue, Mar 29, 2016 at 2:36 PM Skye Wanderman-Milne (Code Review) < [email protected]> wrote:
> Skye Wanderman-Milne has uploaded a new patch set (#6). > > Change subject: IMPALA-2835: introduce PARQUET_FALLBACK_SCHEMA_RESOLUTION > query option > ...................................................................... > > IMPALA-2835: introduce PARQUET_FALLBACK_SCHEMA_RESOLUTION query option > > This patch introduces a new query option, > PARQUET_FALLBACK_SCHEMA_RESOLUTION which allows Parquet files' schemas > to be resolved by either name or position. It's "fallback" because > eventually field IDs will be the primary schema resolution scheme, and > we don't want to create an option that we will have to change the name > of later. The default is still by position. I chose to do a query > option because it will make testing easier and also be easier to > diagnose resolution problems quickly in the field. If users want to > switch the default behavior to be by name (like Hive), they can use > the --default_query_options flag. > > This patch also introduces a new test section, SHELL, which can be > used to execute shell commands in a .test file. This is useful for > copying files into test tables. > > Change-Id: Id0c715ea23792b2a6872610839a40532aabbb5a6 > --- > M be/src/exec/hdfs-parquet-scanner.cc > M be/src/exec/hdfs-parquet-scanner.h > M be/src/service/query-options.cc > M be/src/service/query-options.h > M common/thrift/ImpalaInternalService.thrift > M common/thrift/ImpalaService.thrift > A testdata/parquet_schema_resolution/README > A testdata/parquet_schema_resolution/switched_map.avsc > A testdata/parquet_schema_resolution/switched_map.json > A testdata/parquet_schema_resolution/switched_map.parq > A > testdata/workloads/functional-query/queries/QueryTest/parquet-resolution-by-name.test > M tests/common/impala_test_suite.py > M tests/conftest.py > M tests/query_test/test_scanners.py > M tests/util/test_file_parser.py > 15 files changed, 368 insertions(+), 18 deletions(-) > > > git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/84/2384/6 > -- > To view, visit http://gerrit.cloudera.org:8080/2384 > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings > > Gerrit-MessageType: newpatchset > Gerrit-Change-Id: Id0c715ea23792b2a6872610839a40532aabbb5a6 > Gerrit-PatchSet: 6 > Gerrit-Project: Impala > Gerrit-Branch: cdh5-trunk > Gerrit-Owner: Skye Wanderman-Milne <[email protected]> > Gerrit-Reviewer: Dan Hecht <[email protected]> > Gerrit-Reviewer: Juan Yu <[email protected]> > Gerrit-Reviewer: Matthew Jacobs <[email protected]> > Gerrit-Reviewer: Michael Ho <[email protected]> > Gerrit-Reviewer: Silvius Rus <[email protected]> > Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> >
