[
https://issues.apache.org/jira/browse/PIG-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13547519#comment-13547519
]
Daniel Dai commented on PIG-3117:
---------------------------------
Pig intermediate file is not snappy. By default it is InterStorage. For your
request:
1. It should be fairly easy to retain temp files, just don't call
FileLocalizer.deleteTempFiles() in Main
2. To retain plain text, you may need to change Utils.getTmpFileCompressorName,
not sure if that's enough. Another approach is to write a decoder which invoke
InterStorage to decode the tmp files
> A debug mode in which pig does not delete temporary files
> ---------------------------------------------------------
>
> Key: PIG-3117
> URL: https://issues.apache.org/jira/browse/PIG-3117
> Project: Pig
> Issue Type: Wish
> Affects Versions: 0.10.0
> Reporter: Ido Hadanny
>
> when we debug our pig jobs on pre-production data, we usually find bugs we
> couldn't detect in our UT, as env and data are not quite the same.
> when the final output of a script is not quite what we expect, we start
> divide-and-conquer, running it line by line and inspecting the intermediate
> output of each stage.
> It would be great if we could simply configure pig not to delete the
> intermediate MR outputs, and store them as plaintext instead of snappy format.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
