Hello Aman Sinha, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17878
to look at the new patch set (#3).
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
IMPALA-10934: Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be
considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file system)
which provides a view of a single file in a directory.' The '+' indicates that
this wrapper can be added on top of multiple underlying file systems/object
storage such as HDFS, S3 etc.
The directory which contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version which also
had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/3
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <[email protected]>
Gerrit-Reviewer: Aman Sinha <[email protected]>
Gerrit-Reviewer: Anonymous Coward <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
