[
https://issues.apache.org/jira/browse/METRON-329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Foley updated METRON-329:
------------------------------
Summary: Add new functionality to BulkMessageWriter and related classes for
batchTimeout flushing (was: Add new functionality to BulkMessageWriter and its
related classes for batchTimeout flushing)
> Add new functionality to BulkMessageWriter and related classes for
> batchTimeout flushing
> ----------------------------------------------------------------------------------------
>
> Key: METRON-329
> URL: https://issues.apache.org/jira/browse/METRON-329
> Project: Metron
> Issue Type: Sub-task
> Reporter: Ajay Yadav
> Assignee: Matt Foley
>
> Add new functionality to BulkMessageWriter and its related classes for
> batchTimeout flushing.
> This primarily affects:
> * BulkMessageWriterBolt
> * BulkWriterComponent
> * IndexingConfigurations
> * and newly added utilities in BatchTimeoutHelper
> All of the other changes (from METRON-516) are basically boilerplate, just
> laying "getBatchTimeout()" facade methods alongside existing "getBatchSize()"
> facade methods. (There's a lot of layers!)
> The flush-on-timeout logic is fairly straightforward. It was implemented by a
> refactoring of BulkWriterComponent and basic Tick Tuple reception in
> BulkMessageWriterBolt.
> The tricky part was figuring out the appropriate setting for
> 'topology.tick.tuple.freq.secs' if the administrator configures non-default
> batchTimeouts. It is necessary to enumerate the batchTimeout settings for all
> configured sensorNames, which is implemented in
> IndexingConfigurations$getAllConfiguredTimeouts(). Then multiple other
> factors must be taken into account to determine the allowed and recommended
> settings, which is implemented in BatchTimeoutHelper. If there are better
> ways to accomplish these things, please share your ideas.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
