I am working on NIFI-10243 [1]. The goal is to allow ControlRate to throttle based on both data rate and FlowFile count. If either rate is exceeded, throttling occurs.
Currently, throttling occurs in only one mode. Therefore, a single property, Maximum Rate, is overloaded to accept either a size limit or a count limit. Changing how this property is used could become a breaking change. However, keeping backward compatibility makes the property usage less logical. It makes good sense to have two properties, one for data rate, e.g. "1 MB", and one for count rate, e.g. "10". Unfortunately, this implementation would break current instantiations of the processor since an integer value in the existing attribute would not be accepted any longer. It is possible to keep the functionality the same and introduce a new "Maximum Count Rate" property which is to be used _only_ when the Rate Control Criteria is the new 'data rate and flowfile count'. I don't feel this makes the processor property usage easy to understand though. The flowfile count rate would be provided in the existing Maximum Rate property if the criteria is 'flowfile count', but it would have to be specified in a new property if the criteria is 'data rate and flowfile count'. This is inconsistent and confusing. Perhaps, "Maximum Rate" property could be overloaded to accept a byte value, an integer value or a comma-separated list of both, depending on the selected rate control criteria. It's clunky, but could work. Another alternative to avoid a breaking change is to introduce a new processor, ExtendedControlRate, which allows for the new Rate Control Criteria and redefined rate properties. So, the choices are: 1) Breaking change aligning properties with their purpose in an easy to understand manner 2) Backward compatible where flowfile count is specified in different properties depending on rate control criteria 3) Backward compatible where Maximum Rate accepts a string, an integer or both (comma-separated) 4) Introduce a new processor I think option 1 makes the most sense, but I'm concerned about the breaking nature of the approach. Looking for suggestions on how to proceed. [1] https://issues.apache.org/jira/browse/NIFI-10243
