[
https://issues.apache.org/jira/browse/HIVE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024957#comment-13024957
]
[email protected] commented on HIVE-2126:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/653/
-----------------------------------------------------------
(Updated 2011-04-25 20:48:09.176419)
Review request for hive.
Changes
-------
added a new class SymbolicInputFormat, and move the reworkMapred to this new
class.
move the new code from SemanticAnalyzer to a Utilities method
Summary
-------
Hive's symlink text input format should be able to work with
ComineHiveInputFormat
This addresses bug hive-2126.
https://issues.apache.org/jira/browse/hive-2126
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096548
trunk/conf/hive-default.xml 1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/ReworkMapredInputFormat.java
PRE-CREATION
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymbolicInputFormat.java
PRE-CREATION
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java
1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java
1096548
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
1096548
trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java
1096548
Diff: https://reviews.apache.org/r/653/diff
Testing
-------
Thanks,
Yongqiang
> Hive's symlink text input format should be able to work with
> ComineHiveInputFormat
> ----------------------------------------------------------------------------------
>
> Key: HIVE-2126
> URL: https://issues.apache.org/jira/browse/HIVE-2126
> Project: Hive
> Issue Type: Improvement
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-2126.1.patch, HIVE-2126.2.patch
>
>
> at compile time, if a partition's file format is SymlinkTextInputFormat, will
> replace the symlink path with paths in the symlink file. This way, it will
> work with Hive's HiveCombineFileInputFormat.
> The reason we are doing it at compile time is because:
> 1) At run time, the input path is not only used to get record reader, but
> also used for hive to get aliases and thus operator tree. But the
> CombineHiveInputFormat can have multiple paths for each split, and when
> switching paths, it also set the job with new input file name. So it always
> require a real input path name. Can not fake it.
> 2) if write a new input format, it will require a lot of duplication work
> with existing CombineHiveInputFormat.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
