The reason I say spout is the naive thought that it is better to leave the ‘backup’/caching to kafka than to add it in somewhere else
On November 4, 2016 at 10:28:53, Otto Fowler ([email protected]) wrote: So spout orchestration/gating? Spout checks for external state flag if CURRENT - process if UPDATING - wait With the ingesting agent sets flag to updating when running? On November 4, 2016 at 09:29:16, [email protected] ([email protected]) wrote: Is there a good method (i.e. something using Stellar/ZK) to implement an intentional processing delay to all tuples in a specific topology? I plan to do some custom enrichments, but the data used to do the enrichment *may* be ingested at roughly the same time the data to be enriched is (it also may not ever be sent). So I'd like to add a delay in my cluster that applies to certain parser topologies. I took a look around in the documentation and in JIRA and didn't find anything available or being worked on, but I did see that this may conflict with METRON-322. Essentially what I'm considering is a {sleep,delay,wait} stellar function, but it could also be a delay in a parser's kafka spout (much less of a fan of the second option). I'm looking for feedback on the best way to approach this, and I'd be happy to do the work myself (if necessary) when it gets to that point. I did consider implementing this delay upstream (in the sensor itself), but after looking in more detail it doesn't seem as feasible. Jon -- Jon
