[
https://issues.apache.org/jira/browse/STORM-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201568#comment-15201568
]
ASF GitHub Bot commented on STORM-1464:
---------------------------------------
Github user dossett commented on the pull request:
https://github.com/apache/storm/pull/1044#issuecomment-198393099
@arunmahadevan Writers should never be writing to the same file.
`getBasePathForNextFile` will increment the rotation value before opening the
file. File name also includes `System.currentTimeMillis()` which should
further prevent conflict.
There's also a unit test that attempts to validate this by writing two
different schemas and confirming that two files are created:
https://github.com/dossett/storm/blob/STORM-1494/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/AvroGenericRecordBoltTest.java#L156-L168
> storm-hdfs should support writing to multiple files
> ---------------------------------------------------
>
> Key: STORM-1464
> URL: https://issues.apache.org/jira/browse/STORM-1464
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-hdfs
> Reporter: Aaron Dossett
> Assignee: Aaron Dossett
> Labels: avro
>
> Examples of when this is needed include:
> - One avro bolt writing multiple schemas, each of which require a different
> file. Schema evolution is a common use of avro and the avro bolt should
> support that seamlessly.
> - Partitioning output to different directories based on the tuple contents.
> For example, if the tuple contains a "USER" field, it should be possible to
> partition based on that value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
