The profiles are excellent when you are developing malhar-contrib. Profiles do not work when you are using malhar-contrib. The problem Andy is trying to solve is the later. If there is an elegant solution which I am missing using profiles, please correct me.
The way Andy suggested is the way many successful projects do it. Look at Netty as an example. +1 for that. -- Chetan On Thu, Sep 17, 2015 at 11:22 AM, Timothy Farkas <[email protected]> wrote: > I think restructuring the project in that way would be the technically > correct thing to do, but if people are unwilling to accept the change in > project structure you could achieve something similar by using maven > profiles. With profiles the project structure would remain as is. Profiles > could be added to the malhar pom, and a profile would define the > dependencies needed for different types of operators. For example the hbase > profile would define the dependencies for the hbase operator. Then any > project using a malhar library would just activate the correct profile in > it's pom, and the correct dependencies would be pulled in. > > http://maven.apache.org/guides/introduction/introduction-to-profiles.html > > On Thu, Sep 17, 2015 at 10:01 AM, Andy Perlitch <[email protected]> > wrote: > > > Hi everyone, > > > > I am currently assigned to MLHR-1843 > > <https://malhar.atlassian.net/browse/MLHR-1843>, which essentially aims > to > > expose smaller, more consumable maven artifacts that would do away with > the > > need to manually include necessary dependencies based on the operators in > > use. > > > > As an example, say I am building an app package that needs Kafka input > and > > output operators, but I don't want all the other transitive dependencies > > that come via malhar-contrib. Currently I would need to specify > > malhar-contrib as a dependency, and add an exclusions block in my app > > package pom: > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > <artifactId>malhar-contrib</artifactId> <version>3.0.0</version> <!-- > so > > none of malhar-contrib's deps are included -->* > > > > > > > > > > > > > > * <exclusions> <exclusion> <groupId>*</groupId> > > <artifactId>*</artifactId> </exclusion> </exclusions></dependency>* > > > > Then, I would have to include the kafka library explicitly as a > dependency: > > > > > > > > > > > > *<dependency> <groupId>org.apache.kafka</groupId> > > <artifactId>kafka_2.10</artifactId> > > <version>0.8.1.1</version></dependency>* > > > > Wouldn't it be nice if I could just put this in my pom?: > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > <artifactId>malhar-contrib-kafka</artifactId> > > <version>3.0.0</version></dependency>* > > > > > > In order to make this possible, we will need to organize the malhar > project > > into more granular modules (artifacts). Specifically, the malhar-contrib > > artifact would essentially just be a pom that specifies each smaller > module > > as a dependency: > > > > *<!-- in malhar-contrib's pom.xml: -->* > > > > *<modules> <module>kafka</module>* > > * <module>twitter</module>* > > * <module>redis</module>* > > > > * <!-- other smaller modules --></modules>* > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > <artifactId>malhar-contrib-kafka</artifactId> > > <version>3.0.0</version></dependency>* > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > <artifactId>malhar-contrib-twitter</artifactId> > > <version>3.0.0</version></dependency>* > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > <artifactId>malhar-contrib-redis</artifactId> > > <version>3.0.0</version></dependency>* > > > > With these changes, there may be a risk of breaking backwards > > compatibility, however I think the gain in usability of malhar merits the > > effort to make this work. > > > > I am still relatively new to maven, so I would love to get some feedback > > from other devs about this! > > > > -- > > Regards, > > Andy Perlitch > > Software Engineer > > DataTorrent Inc > > (408)829-9319 > > >
