Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15998
to look at the new patch set (#2).
Change subject: IMPALA-9777: Set hive.optimize.sort.dynamic.partition to true
for dynamic inserts
......................................................................
IMPALA-9777: Set hive.optimize.sort.dynamic.partition to true for dynamic
inserts
This sets hive.optimize.sort.dynamic.partition to true by default during
data load. This option takes effect during Hive dynamic partitioning
inserts. It introduces a sort into the insert query so that all data is
sorted on the partition key. This allows the reducers to only open a single
file at a time when writing out files. When this config is set to false,
dynamic partitioning inserts will be run as a map-only job that
potentially opens hundreds of files per partition, resulting in lots of
small files. Creating all these small files potentially impacts the
health of the Namenode, and can cause data-load to fail altogether.
Testing:
* Ran core tests for Impala-EC
Change-Id: Ic2b7c0ec40a02da2640fae20cf640517fd1f4fef
---
M testdata/bin/generate-schema-statements.py
M testdata/datasets/tpcds/tpcds_schema_template.sql
2 files changed, 5 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15998/2
--
To view, visit http://gerrit.cloudera.org:8080/15998
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic2b7c0ec40a02da2640fae20cf640517fd1f4fef
Gerrit-Change-Number: 15998
Gerrit-PatchSet: 2
Gerrit-Owner: Sahil Takiar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
