Dan, Certainly a valid concern. I like the idea of an indicator on the connection that it is penalizing. I know there has been some thought already going into some UI redesigns, so that's a good thing to keep in mind there.
I can also understand the concern about a Connection performing an action on the FlowFile, but this concerns me less. I say this because the job of the Connection is to sort/order/prioritize FlowFiles and provide the appropriate FlowFiles to the 'destination component'. Penalization can be thought of as simply determining whether or not it is appropriate to provide a given FlowFile. I.e., it doesn't really change the FlowFile itself so much as it makes a decision about when/how to distribute that FlowFile. > On Jan 28, 2016, at 12:00 PM, dan bress <[email protected]> wrote: > > Mark, > I agree with all the points you mention about penalization being > confusing, and I think the ability to apply a penalty to Flowfile's outside > of a processor is a clearer way to express what is happening. > > I worry that having the penalty be a property of the connection would > also be confusing. To me, penalizing a FlowFile is an action you do to a > FlowFile. In my head, connections don't do actions on FlowFile, they just > sort them and move them along. I might find it confusing that the > connection is "doing things" to the flow files, unless there was some kind > of visual cue as to what was going on. Kind of like how people have > brought up that the "expire" concept is a little confusing, because of the > lack of visual cue. > > So when I started typing this email I was thinking we should have a new > concept of a "penalizer" that's kind of like a processor but just puts a > penalty on a flow file. After typing it, that might be a new construct > that isn't really needed, and I'm OK with this being put on a connection, > but I would like there to be a visual cue on the connection indicating that > it is penalizing flow files. > > On Thu, Jan 28, 2016 at 8:34 AM Mark Payne <[email protected]> wrote: > >> All, >> >> I've been thinking about how we handle the concept of penalizing >> FlowFiles. We've had a lot of questions >> lately about how penalization works & the concept in general. Seems the >> following problems exist: >> >> - Confusion about difference between penalization & yielding >> - DFM sees option to configure penalization period on all processors, even >> if they don't penalize FlowFiles. >> - DFM cannot set penalty duration in 1 case and set a different value for >> a different case (different relationship, for example). >> - Developers often forget to call penalize() >> - Developer has to determine whether or not to penalize when building a >> processor. It is based on what the developer will >> think may make sense, but in reality DFM's sometimes want to penalize >> things when the processor doesn't behave that way. >> >> I'm wondering if it doesn't make sense to remove the concept of >> penalization all together from Processors and instead >> move the Penalty Duration so that it's a setting on the Connection. I >> think this would clear up the confusion and give the DFM >> more control over when/how long to penalize. Could set to the default to >> 30 seconds for self-looping connections and no penalization >> for other connections. >> >> Any thoughts? >> >> Thanks >> -Mark
