[
https://issues.apache.org/jira/browse/METRON-750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Foley updated METRON-750:
------------------------------
Description:
Add new functionality to ParserWriter and its related classes for batchTimeout
flushing.
This parallels the work for the Indexing Writer components in METRON-329.
was:
Add new functionality to BulkMessageWriter and its related classes for
batchTimeout flushing.
This primarily affects:
* BulkMessageWriterBolt
* BulkWriterComponent
* IndexingConfigurations
* and newly added utilities in BatchTimeoutHelper
All of the other changes (from METRON-516) are basically boilerplate, just
laying "getBatchTimeout()" facade methods alongside existing "getBatchSize()"
facade methods. (There's a lot of layers!)
The flush-on-timeout logic is fairly straightforward. It was implemented by a
refactoring of BulkWriterComponent and basic Tick Tuple reception in
BulkMessageWriterBolt.
The tricky part was figuring out the appropriate setting for
'topology.tick.tuple.freq.secs' if the administrator configures non-default
batchTimeouts. It is necessary to enumerate the batchTimeout settings for all
configured sensorNames, which is implemented in
IndexingConfigurations$getAllConfiguredTimeouts(). Then multiple other factors
must be taken into account to determine the allowed and recommended settings,
which is implemented in BatchTimeoutHelper. If there are better ways to
accomplish these things, please share your ideas.
> Add new functionality to ParserWriter related classes for batchTimeout
> flushing
> -------------------------------------------------------------------------------
>
> Key: METRON-750
> URL: https://issues.apache.org/jira/browse/METRON-750
> Project: Metron
> Issue Type: Sub-task
> Reporter: Ajay Yadav
> Assignee: Matt Foley
>
> Add new functionality to ParserWriter and its related classes for
> batchTimeout flushing.
> This parallels the work for the Indexing Writer components in METRON-329.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
