Edgardo It's not really a clear winner in either place. Your intuition here is totally fair and indeed it used to be a nifi properties config item. Then we had stronger cases for it being a flow specific configuration item. I think the reality is that you need a bit of both. The total number of available threads can impact the way someone designing a flow thinks. This is seen in how they allocate a number of 'concurrent tasks' to a given processor. This means it is in a sense more of a 'flow configuration' item. But you're absolutely right that the other way of looking at this is that the total number of threads will often be a function of the capabilities of a given system.
So, I'd propose us supporting both if this turns out to be worth tackling. The current method works well. But we could add back a configuration property item that is more tied to the 'system' than the flow and that property would be 'max threads' or something. Then at runtime we'd set the actual 'max threads' to the lesser of the flow configured value or the nifi properties value. Thanks Joe On Wed, May 6, 2015 at 7:07 PM, Matt Gilman <[email protected]> wrote: > Ah, yes Corey you're totally right. Sorry about that. Also wanted to point > out that starting with this upcoming release (0.1.0-incubating) the > controller services and reporting task will configurable in the UI and become > part of the flow configuration. > > Sent from my iPhone > >> On May 6, 2015, at 6:23 PM, Corey Flowers <[email protected]> wrote: >> >> Hey guys, >> >> The flow.xml.gz should not be updated by puppet. Puppet is great for >> config files, controller services, reporting tasks, really anything >> "variable/value" or "tag/value" like that but not the flow.xml.gz. It is >> constantly changed and dynamic in its nature, which doesn't lend well to >> the overall concept of puppet manifest control. Just a heads up. >> >> I will let you guys decide about the idea to make these values config >> variables. >> >> Later! >> >> >>> On Wednesday, May 6, 2015, Edgardo Vega <[email protected]> wrote: >>> >>> That's an interesting place to store that information wouldn't it make more >>> sense to be in nifi.properties? If your sharing the flow.xml.gz on multiple >>> machines with different specs you would have to manually tweak the flow on >>> each machine. >>> >>> I would gladly make a ticket if it seems to make more sense for these >>> properties to be moved. What's the consensus? >>> >>> On Wednesday, May 6, 2015, Matt Gilman <[email protected] >>> <javascript:;>> wrote: >>> >>>> It's actually saved as part of your flow. So assuming your flow is part >>> of >>>> your puppet configuration, you should be good. In a standalone instance >>> it >>>> would be in <NIFI_HOME>/conf/flow.xml.gz. >>>> >>>> Matt >>>> >>>> On Wed, May 6, 2015 at 3:47 PM, Edgardo Vega <[email protected] >>> <javascript:;> >>>> <javascript:;>> wrote: >>>> >>>>> Thanks Matt. Is this configurable in a file or through a java argument? >>>>> That way we can set that via puppet. >>>>> >>>>> Cheers, >>>>> >>>>> Edgardo >>>>> >>>>> On Wed, May 6, 2015 at 3:31 PM, Matt Gilman <[email protected] >>> <javascript:;> >>>> <javascript:;>> >>>>> wrote: >>>>> >>>>>> The existing documentation is a little short but it is mentioned in >>> the >>>>>> Other Management Features section [1]. In the UI can you click the >>>> wrench >>>>>> and screwdriver icon in the upper right. There you can set the name >>> of >>>>> your >>>>>> dataflow, it's description, and the size of the timer/event driven >>>> thread >>>>>> pools. >>>>>> >>>>>> Matt >>>>>> >>>>>> [1] >>> https://nifi.incubator.apache.org/docs/nifi-docs/user-guide.html#other_management_features >>>>>> >>>>>> On Wed, May 6, 2015 at 3:22 PM, Edgardo Vega <[email protected] >>> <javascript:;> >>>> <javascript:;>> >>>>>> wrote: >>>>>> >>>>>>> The admin guide says the Maximum Forked Processes section says >>> "NiFi >>>>> may >>>>>>> be configured to generate a significant number of threads." How is >>>> this >>>>>>> done? I read the rest of the documentation and didn't see a way to >>>>> define >>>>>>> the number of threads to use. Also any recommendation on number of >>>>>> threads >>>>>>> per code? >>>>>>> >>>>>>> I have been seeing the number of threads in the gui pegged at 10 >>> that >>>>> is >>>>>>> why I went looking for how to change that number. >>>>>>> >>>>>>> Thanks in advance, >>>>>>> >>>>>>> Edgardo >>>>> >>>>> >>>>> >>>>> -- >>>>> Cheers, >>>>> >>>>> Edgardo >>> >>> >>> -- >>> Cheers, >>> >>> Edgardo >>> >>> Sent from Gmail Mobile >> >> >> -- >> Corey Flowers >> Vice President, Onyx Point, Inc >> (410) 541-6699 >> [email protected] >> >> -- This account not approved for unencrypted proprietary information --
