All, I've been thinking about how we handle the concept of penalizing FlowFiles. We've had a lot of questions lately about how penalization works & the concept in general. Seems the following problems exist:
- Confusion about difference between penalization & yielding - DFM sees option to configure penalization period on all processors, even if they don't penalize FlowFiles. - DFM cannot set penalty duration in 1 case and set a different value for a different case (different relationship, for example). - Developers often forget to call penalize() - Developer has to determine whether or not to penalize when building a processor. It is based on what the developer will think may make sense, but in reality DFM's sometimes want to penalize things when the processor doesn't behave that way. I'm wondering if it doesn't make sense to remove the concept of penalization all together from Processors and instead move the Penalty Duration so that it's a setting on the Connection. I think this would clear up the confusion and give the DFM more control over when/how long to penalize. Could set to the default to 30 seconds for self-looping connections and no penalization for other connections. Any thoughts? Thanks -Mark
