Hive's symlink text input format should be able to work with
ComineHiveInputFormat
----------------------------------------------------------------------------------
Key: HIVE-2126
URL: https://issues.apache.org/jira/browse/HIVE-2126
Project: Hive
Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang
at compile time, if a partition's file format is SymlinkTextInputFormat, will
replace the symlink path with paths in the symlink file. This way, it will work
with Hive's HiveCombineFileInputFormat.
The reason we are doing it at compile time is because:
1) At run time, the input path is not only used to get record reader, but also
used for hive to get aliases and thus operator tree. But the
CombineHiveInputFormat can have multiple paths for each split, and when
switching paths, it also set the job with new input file name. So it always
require a real input path name. Can not fake it.
2) if write a new input format, it will require a lot of duplication work with
existing CombineHiveInputFormat.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
