[
https://issues.apache.org/jira/browse/FLUME-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632752#comment-13632752
]
Thiruvalluvan M. G. commented on FLUME-2003:
--------------------------------------------
Yes it would. But such a configuration will affect _all_ the files created by
Flume. One does not have control over individual files. It is observed that the
throughput of HDFS write operations is a function of replication factor and
block size. They have huge impact on flush() and hence on small "transactions"
because HDFS-sink flushes at the end of each transaction. With this patch one
can configure smaller block size and lower replication for certain types of
events (without affecting others) and then convert to larger configuration as
the data gets processed.
> It'll be nice if we can control the HDFS block-size and replication for
> specific HDFS-sink instances
> ----------------------------------------------------------------------------------------------------
>
> Key: FLUME-2003
> URL: https://issues.apache.org/jira/browse/FLUME-2003
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Reporter: Thiruvalluvan M. G.
> Fix For: v1.4.0
>
> Attachments: FLUME-2003.patch
>
>
> The forthcoming patch provides that functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
