[
https://issues.apache.org/jira/browse/STREAMS-183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Blackmon updated STREAMS-183:
-----------------------------------
Component/s: Runtimes
> streams-local closes persist-writers before datums in-flight are written
> ------------------------------------------------------------------------
>
> Key: STREAMS-183
> URL: https://issues.apache.org/jira/browse/STREAMS-183
> Project: Streams
> Issue Type: Bug
> Components: Runtimes
> Reporter: Steve Blackmon
>
> I have observed this behavior when writing datums from a readCurrent provider
> through to ElasticsearchPersistWriter and ElasticsearchPersistDeleter.
> The log looks like:
> 421777 [pool-2-thread-7] INFO o.a.s.e.ElasticsearchPersistReader -
> PersistReader done
> 469271 [main] INFO o.a.s.l.builders.LocalStreamBuilder - Attempting to
> shutdown tasks
> 476319 [pool-2-thread-3] ERROR o.a.s.e.ElasticsearchPersistWriter - There was
> an error sending the batch: No node available
> 485275 [main] WARN o.a.s.l.builders.LocalStreamBuilder - Task
> org.apache.streams.local.tasks.StreamsPersistWriterTask@6add646d failed to
> terminate in allotted timeframe
> 495276 [pool-2-thread-3] WARN o.a.s.e.ElasticsearchPersistWriter - Catchup
> was interrupted. Data may be lost
> followed by additional No Node Available exceptions.
> When the same stream is executed as a perpetual stream, this problem does not
> occur.
> Possible solutions:
> a) localstreambuilder waits a while between detecting providers have
> completed and stopping downstream components
> b) localstreambuilder does not stop downstream components until all datums
> have been processed/written
> c) writers refuse to close until all datums have been written
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
