Mark, I agree with all the points you mention about penalization being confusing, and I think the ability to apply a penalty to Flowfile's outside of a processor is a clearer way to express what is happening.
I worry that having the penalty be a property of the connection would also be confusing. To me, penalizing a FlowFile is an action you do to a FlowFile. In my head, connections don't do actions on FlowFile, they just sort them and move them along. I might find it confusing that the connection is "doing things" to the flow files, unless there was some kind of visual cue as to what was going on. Kind of like how people have brought up that the "expire" concept is a little confusing, because of the lack of visual cue. So when I started typing this email I was thinking we should have a new concept of a "penalizer" that's kind of like a processor but just puts a penalty on a flow file. After typing it, that might be a new construct that isn't really needed, and I'm OK with this being put on a connection, but I would like there to be a visual cue on the connection indicating that it is penalizing flow files. On Thu, Jan 28, 2016 at 8:34 AM Mark Payne <[email protected]> wrote: > All, > > I've been thinking about how we handle the concept of penalizing > FlowFiles. We've had a lot of questions > lately about how penalization works & the concept in general. Seems the > following problems exist: > > - Confusion about difference between penalization & yielding > - DFM sees option to configure penalization period on all processors, even > if they don't penalize FlowFiles. > - DFM cannot set penalty duration in 1 case and set a different value for > a different case (different relationship, for example). > - Developers often forget to call penalize() > - Developer has to determine whether or not to penalize when building a > processor. It is based on what the developer will > think may make sense, but in reality DFM's sometimes want to penalize > things when the processor doesn't behave that way. > > I'm wondering if it doesn't make sense to remove the concept of > penalization all together from Processors and instead > move the Penalty Duration so that it's a setting on the Connection. I > think this would clear up the confusion and give the DFM > more control over when/how long to penalize. Could set to the default to > 30 seconds for self-looping connections and no penalization > for other connections. > > Any thoughts? > > Thanks > -Mark
