[
https://issues.apache.org/jira/browse/KAFKA-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jun Rao reassigned KAFKA-1519:
------------------------------
Assignee: Gwen Shapira (was: Neha Narkhede)
> Console consumer: expose configuration option to enable/disable writing the
> line separator
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-1519
> URL: https://issues.apache.org/jira/browse/KAFKA-1519
> Project: Kafka
> Issue Type: Improvement
> Components: consumer
> Affects Versions: 0.8.1.1
> Reporter: Michael Noll
> Assignee: Gwen Shapira
> Priority: Minor
> Attachments: KAFKA-1519.patch
>
>
> The current console consumer includes a {{DefaultMessageFormatter}}, which
> exposes a few user-configurable options which can be set on the command line
> via --property, e.g. "--property line.separator=XYZ".
> Unfortunately, the current implementation does not allow the user to
> completely disable writing any such line separator. However, this
> functionality would be helpful to enable users to capture data "as is" from a
> Kafka topic to snapshot file. Capturing data "as is" -- without an
> artificial line separator -- is particularly nice for data in a binary format
> (including Avro).
> *No workaround*
> A potential workaround would be to pass an empty string as the property value
> of "line.separator", but this doesn't work in the current implementation.
> The following variants throw an "Invalid parser arguments" exception:
> {code}
> --property line.separator= # "nothing"
> --property line.separator="" # double quotes
> --property line.separator='' # single quotes
> {code}
> Escape tricks via a backslash don't work either.
> If there actually is a workaround please let me know.
> *How to fix*
> We can introduce a "print.line" option to enable/disable writing
> "line.separator" similar to how the code already uses "print.key" to
> enable/disable writing "key.separator".
> This change is trivial. To preserve backwards compatibility, the
> "print.line" option would be set to true by default (unlike the "print.key"
> option, which defaults to false).
> *Alternatives*
> Apart from modifying the built-in {{DefaultMessageFormatter}}, users could of
> course implement their own custom {{MessageFormatter}}. But given that it's
> a) a trivial change to the {{DefaultMessageFormatter}} and b) a nice user
> feature I'd say changing the built-in {{DefaultMessageFormatter}} would be
> the better approach. This way, Kafka would support writing data as-is to a
> file out of the box.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
