[
https://issues.apache.org/jira/browse/METRON-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Casey Stella updated METRON-955:
--------------------------------
Fix Version/s: 0.4.0
> Make the default sync policy for HDFS Writer be based on the batch size
> -----------------------------------------------------------------------
>
> Key: METRON-955
> URL: https://issues.apache.org/jira/browse/METRON-955
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Assignee: Casey Stella
> Fix For: 0.4.0
>
>
> Right now, we do a sync per record written in HDFS. This has performance
> penalties as it requires a Namenode contact. We get around this by letting
> the user define the sync policy in Flux, but that's less convenient and more
> sensible defaults could be created. We should use the batch size as a
> sensible default for the sync policy (i.e. sync every batch).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
