[
https://issues.apache.org/jira/browse/METRON-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018153#comment-16018153
]
ASF GitHub Bot commented on METRON-965:
---------------------------------------
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/596
@justinleet absolutely. I extracted the lambda function into a class and
tested it individually. Specifically, I ensured that the cloned copies
function like the base copy.
> In the case where we specify the syncpolicy in the HDFS Writer, we do not
> properly clone and end up syncing for every record
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: METRON-965
> URL: https://issues.apache.org/jira/browse/METRON-965
> Project: Metron
> Issue Type: Bug
> Reporter: Casey Stella
>
> Right now the SyncPolicy works as follows:
> * If you specify the sync policy in the flux file, it will use that policy to
> determine when to sync the HDFS Writer.
> * If you do not, then it will create a count sync policy based on the batch
> size.
> In the first case, we are not creating the sync policy properly and end up
> sync'ing per record, which has performance impact and is incorrect.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
