Tim Armstrong created IMPALA-9477:
-------------------------------------
Summary: Only add required partitions to TDescriptorTable for hdfs
table sinks
Key: IMPALA-9477
URL: https://issues.apache.org/jira/browse/IMPALA-9477
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Tim Armstrong
In
https://github.com/apache/impala/blob/b068c2c/fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java#L196
we fall back to adding all partition metadata for the target table for DMLs.
I don't think this is necessary if query specifies a target partition. Or if it
has multiple partition keys and it specified constants for some of them.
If a table has a large number of partitions, the memory overhead can be very
high because of things like IMPALA-8835
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
