Those sound like really good candidates for consolidation. It might also be worth looking at the dependency graph to find a lot of co-occurence. If every module that depends on A also depends on B, there's less of an argument to keep them separate.
-Joey On Thu, Dec 18, 2014 at 7:37 PM, Michael Moser <[email protected]> wrote: > You didn't fall too far, Joe. core-flowfile-attributes, > naive-search-ring-buffer, nifi-properties, nifi-stream-utils, and > processor-utilities have no dependencies. And nifi-logging-utils only > depends on the provided slf4j-api. Just sayin' ;) > > -- Mike > > On Thu, Dec 18, 2014 at 8:06 PM, Joe Witt <[email protected]> wrote: >> i'm sure mark payne was laughing at my response on this thread pretty >> hard. I've given him grief before for the many split utilities jars we >> have and he'd each time quickly remind me that it was to avoid pulling >> needless deps into spaces we dont want them. So fell into the trap again >> today... >> >> Thanks >> Joe >> >> On Thu, Dec 18, 2014 at 5:47 PM, Michael Moser <[email protected]> wrote: >>> >>> Joey makes a good point. Without nifi-socket-utils, the transitive >>> dependencies will be commons-codec, commons-compress and >>> commons-lang3. slf4j-api is also in there but that's marked provided >>> by all of NiFi. >>> >>> When you add nifi-socket-utils to the equation, that adds commons-io >>> and commons-net. Interestingly, nifi-socket-utils also depends on >>> three of the other nifi-*-utils. So if you need nifi-socket-utils, >>> you also get nifi-properties, nifi-logging-utils, and nifi-utils >>> anyway. >>> >>> It's probably worth leaving nifi-socket-utils separate for now. It >>> had the biggest footprint of all of the utils to begin with. >>> >>> flowfile-packager is the only one that pulls in commons-compress. >>> nifi-file-utils is the only one that pulls in commons-codec (though >>> that dependency could be removed with a clever refactor of >>> computeMd5Digest(File file) using just the JDK). >>> >>> -- Mike >>> >>> >>> On Thu, Dec 18, 2014 at 2:38 PM, Joey Echeverria <[email protected]> >>> wrote: >>> > Do we know how many transitive dependencies this ends up mixing together? >>> > >>> > I bring it up because that's often a reason for splitting a small >>> > number of classes into their own module. For example, if I care about >>> > socket-based data flow maybe I don't need the dependancies utilities >>> > related to file-based data flow. I'll try to take a look at the actual >>> > modules, but I thought I would throw that out there for others to >>> > think on. >>> > >>> > One thing I've seen work well is creating a dependency aggregator >>> > module for users that don't care about the extra dependencies. >>> > >>> > -Joey >>> > >>> > On Thu, Dec 18, 2014 at 12:13 PM, Joe Witt <[email protected]> wrote: >>> >> Mike, >>> >> >>> >> I think this is a great point and a great analysis. >>> >> >>> >> +1 and unless anyone specifically objects I'll go ahead and do this >>> >> tonight. If i run into any curveballs I'll throw it on this thread. >>> >> >>> >> Thanks >>> >> Joe >>> >> >>> >> On Thu, Dec 18, 2014 at 11:04 AM, Michael Moser <[email protected]> >>> wrote: >>> >>> >>> >>> I'm not sure if this is the most appropriate forum or I should have >>> >>> just written a Jira ticket, but here goes. >>> >>> >>> >>> I believe we should consolidate the number of artifacts we have in the >>> >>> nifi/commons module. We create three jars that contain just 1 class >>> >>> each and there are three more jars with 3 or fewer classes in them. >>> >>> This makes it annoying (especially for beginners) to find the location >>> >>> of classes that you need and slightly bloats our footprint for number >>> >>> of artifacts that nifi create. I believe we can improve this. >>> >>> >>> >>> I analyzed all of the nar-bundles to find where each common library >>> >>> was used. Several are used by many framework, services, and >>> >>> processors bundles already, so consolidating these common jars is a >>> >>> no-brainer. Other jars that are used more sparingly contain just 1 or >>> >>> 2 classes, so it really will have minimal impact to consolidate them >>> >>> even if the classes aren't needed by a nar. >>> >>> >>> >>> So, I propose we consolidate these artifacts into the nifi-utils >>> >>> artifact. The number in (parentheses) is the number of classes in >>> >>> them. >>> >>> >>> >>> core-flowfile-attributes (2) >>> >>> flowfile-packager (9) >>> >>> naive-search-ring-buffer (1) >>> >>> nifi-file-utils (1) >>> >>> nifi-logging-utils (1) >>> >>> nifi-properties (2) >>> >>> nifi-security-utils (5) >>> >>> nifi-socket-utils (24) >>> >>> nifi-stream-utils (17) >>> >>> processor-utilities (3) (this would also resolve why the name doesn't >>> >>> start with "nifi") >>> >>> >>> >>> nifi-utils would go from 24 classes to 89 classes. >>> >>> >>> >>> nifi-web-utils (3), remote-communications-utils (13), and search-utils >>> >>> (5) I did not include because their use is limited to just one or two >>> >>> places. >>> >>> >>> >>> Thanks, >>> >>> -- Mike >>> >>> >>> > >>> > >>> > >>> > -- >>> > Joey Echeverria >>> -- Joey Echeverria
