Hello Bharath Vissapragada, Zoltan Borok-Nagy, Sailesh Mukil, Todd Lipcon, Tim
Armstrong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10654
to look at the new patch set (#3).
Change subject: IMPALA-7121: Clean up partitionIds_ from HdfsTable
......................................................................
IMPALA-7121: Clean up partitionIds_ from HdfsTable
The purpose of introducing partitionIds_ member to HdfsTable was to be
able to return the IDs of all the current partitions in constant time.
Apparently, partitionMap_ also contains these IDs as the key of the
map and this is accessible via keySet() also in constant time. It
seems reasonable then to remove partitionIds_ and use
partitionMap_.keySet() in getPartitionIds() to save some memory.
One thing needs extra attention here is that modifying the result of
keySet() would also modify partitionMap_ and we should avoid doing
this. On every callsites of getPartitionIds() the first step the
caller does is to copy the received items to a separate set. So as a
solution getPartitionIds() internally creates a copy of the keySet(),
removes the default partition and returns this copy to be sure that
partitionMap_ can't be altered. The caller sites are also changed not
to copy the items but to simpy use the set they received. This will
guarantee that we don't regress the computing complexity of getting
the partition IDs.
Change-Id: I8b5a480e570aeae565fafd4f3e2b279e7a98c7da
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/planner/HdfsPartitionPruner.java
3 files changed, 14 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/10654/3
--
To view, visit http://gerrit.cloudera.org:8080/10654
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8b5a480e570aeae565fafd4f3e2b279e7a98c7da
Gerrit-Change-Number: 10654
Gerrit-PatchSet: 3
Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
