Laszlo Gaal has uploaded a new change for review. http://gerrit.cloudera.org:8080/4914
Change subject: IMPALA-4172: Switch to BlockLocation methods for disk IDs ...................................................................... IMPALA-4172: Switch to BlockLocation methods for disk IDs This change enables Impala to use BlockLocation#getStorageIds, the new call in Hadoop 3 for getting HDFS data block location info from the NameNode. This call supercedes the old getFileBlockLocations call which will be deprecated in Hadoop-3. The presence of BlockLocation#getStorageIds is determined dynamically. This is necessary because the implementation was backported to various Hadoop-2 releases after it appeared in Hadoop-3, so simple version checking is not suitable to decide whether the call can be used. In cases where both getFileBlockLocations and BlockLocation#getStorageIds are supported, the code prefers the latter for performance reasons: getStorageIds does not have to query the DataNodes for this information. BlockLocation#getStorageIds returns disk IDs as UUID-based strings, which would be too expensive to ship around and store. This patch maps these strings to small integers, preserving compatibility with the existing representation of diskIDs. Change-Id: Ibbff94cef9a9db7b3945f8e7b0286866d2cc3b61 --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java 1 file changed, 155 insertions(+), 42 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/4914/1 -- To view, visit http://gerrit.cloudera.org:8080/4914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibbff94cef9a9db7b3945f8e7b0286866d2cc3b61 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Laszlo Gaal <laszlo.g...@cloudera.com>
