[
https://issues.apache.org/jira/browse/TEZ-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918616#comment-13918616
]
Siddharth Seth commented on TEZ-873:
------------------------------------
[~kamrul], File name fetching is something that Hive also used to rely upon.
They'd end up polling for the file name on each row (I think they still do
this). Is that something you're OK with, or is it better to have a mechanism
where the user can be notified when a file changes. Similarly for other
information.
Also, I may be missing some context, but can't find where the changes to
TezGroupedSplits are being used.
> Allow MRInputLegacy to expose the individual input split
> --------------------------------------------------------
>
> Key: TEZ-873
> URL: https://issues.apache.org/jira/browse/TEZ-873
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: TEZ-873.1.patch, TEZ-873.2.patch
>
>
> Currently there is no way of getting InputSplit from TezProcessor. In current
> MR framework, there is a way to find out the filename through FileSplit.
> For example, one common uses is to get the filename in map
> String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();
> There are other meta-data in Inputsplit that could be used by existing MR
> user.
> This JIRA is to add APIs to expose the InputSplit by adding these
> TezGroupedSplit.getWrapperSplit() and MRInput.getInputSplit().
> Although MRInputLegacy provide an API to get the InputSplit, it has few
> issues:
> * Without TezGroupedSplit.getWrapperSplit() it is unusable.
> * Since it is used in various use cases, I propose to move it from
> MRInputLegacy to MRInput.
> * Currently the APIs are named as getNewInputSplit() and getOldInputSplit().
> These should be merged into one : getInputSplit(). The new/old API should be
> handled internally.
> Please give your feedback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
