[
https://issues.apache.org/jira/browse/PIG-836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529300#comment-14529300
]
Rohini Palaniswamy commented on PIG-836:
----------------------------------------
cat delimiter.properties
{code}
textinputformat.record.delimiter=\n
{code}
You can run as pig -P delimiter.properties test.pig. Problem in passing in
command line or set command is it is interpreted as string and so \n is
actually treated as two characters - \ followed by n. Beware that if the files
are bzip compressed, then this does not work as Pig uses a custom
Bzip2TextInputFormat to read bzip files and that does not honor the
textinputformat.record.delimiter setting.
> Allow setting of end-of-record delimiter in PigStorage
> ------------------------------------------------------
>
> Key: PIG-836
> URL: https://issues.apache.org/jira/browse/PIG-836
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: George Mavromatis
> Assignee: Benjamin Reed
>
> PigStorage allows overriding the default field delimiter ('\t'), but does not
> allow overriding the record delimiter ('\n').
> It is a valid use case that fields contain new lines, e.g. because they are
> contents of a document/web page. It is possible for the user to create a
> custom load/store UDF to achieve that, but that is extra work on the user,
> many users will have to do it , and that udf would be the exact code
> duplicate of the PigStorage except for the delimiter.
> Thus, PigStorage() should allow to configure both field and record separators.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
