Sorry any thoughts; “ I think it would be better/simpler to have one “out of service” concept to replace penalizing and yielding and when a plugin throws an exception then the plugin is deemed out of service, for a duration and so the connection to that plugin is disabled for the out of service duration.
When a plugin is out of service and the connection disabled - then resources that it uses will be freed(yielded). The question then is what the behaviour of the plugin before the disabled connection - should be. My thought is to tend towards stability and make sure resources are freed, so there may need to be a “domino effect”/cascade affect where all plugins before are gradually put out of service.” Excuse my lack og understanding but does penalizing a processor need to be an action - will it not always be derived from an error condition? With Kind Regards Michael de Courci [email protected] > On 28 Jan 2016, at 17:09, Mark Payne <[email protected]> wrote: > > Dan, > > Certainly a valid concern. I like the idea of an indicator on the connection > that > it is penalizing. I know there has been some thought already going into some > UI redesigns, so that's a good thing to keep in mind there. > > I can also understand the concern about a Connection performing an action > on the FlowFile, but this concerns me less. I say this because the job of the > Connection is to sort/order/prioritize FlowFiles and provide the appropriate > FlowFiles to the 'destination component'. Penalization can be thought of as > simply determining whether or not it is appropriate to provide a given > FlowFile. > I.e., it doesn't really change the FlowFile itself so much as it makes a > decision > about when/how to distribute that FlowFile. > > >> On Jan 28, 2016, at 12:00 PM, dan bress <[email protected]> wrote: >> >> Mark, >> I agree with all the points you mention about penalization being >> confusing, and I think the ability to apply a penalty to Flowfile's outside >> of a processor is a clearer way to express what is happening. >> >> I worry that having the penalty be a property of the connection would >> also be confusing. To me, penalizing a FlowFile is an action you do to a >> FlowFile. In my head, connections don't do actions on FlowFile, they just >> sort them and move them along. I might find it confusing that the >> connection is "doing things" to the flow files, unless there was some kind >> of visual cue as to what was going on. Kind of like how people have >> brought up that the "expire" concept is a little confusing, because of the >> lack of visual cue. >> >> So when I started typing this email I was thinking we should have a new >> concept of a "penalizer" that's kind of like a processor but just puts a >> penalty on a flow file. After typing it, that might be a new construct >> that isn't really needed, and I'm OK with this being put on a connection, >> but I would like there to be a visual cue on the connection indicating that >> it is penalizing flow files. >> >> On Thu, Jan 28, 2016 at 8:34 AM Mark Payne <[email protected]> wrote: >> >>> All, >>> >>> I've been thinking about how we handle the concept of penalizing >>> FlowFiles. We've had a lot of questions >>> lately about how penalization works & the concept in general. Seems the >>> following problems exist: >>> >>> - Confusion about difference between penalization & yielding >>> - DFM sees option to configure penalization period on all processors, even >>> if they don't penalize FlowFiles. >>> - DFM cannot set penalty duration in 1 case and set a different value for >>> a different case (different relationship, for example). >>> - Developers often forget to call penalize() >>> - Developer has to determine whether or not to penalize when building a >>> processor. It is based on what the developer will >>> think may make sense, but in reality DFM's sometimes want to penalize >>> things when the processor doesn't behave that way. >>> >>> I'm wondering if it doesn't make sense to remove the concept of >>> penalization all together from Processors and instead >>> move the Penalty Duration so that it's a setting on the Connection. I >>> think this would clear up the confusion and give the DFM >>> more control over when/how long to penalize. Could set to the default to >>> 30 seconds for self-looping connections and no penalization >>> for other connections. >>> >>> Any thoughts? >>> >>> Thanks >>> -Mark >
