Currently all the dependencies in Malhar-Contrib are marked as optional. So users have to already modify the existing POM to use it in their project. So restructuring should be fine.
On Thu, Sep 17, 2015 at 11:29 AM Chetan Narsude <[email protected]> wrote: > The profiles are excellent when you are developing malhar-contrib. Profiles > do not work when you are using malhar-contrib. The problem Andy is trying > to solve is the later. If there is an elegant solution which I am missing > using profiles, please correct me. > > The way Andy suggested is the way many successful projects do it. Look at > Netty as an example. > > +1 for that. > > > -- > Chetan > > > > On Thu, Sep 17, 2015 at 11:22 AM, Timothy Farkas <[email protected]> > wrote: > > > I think restructuring the project in that way would be the technically > > correct thing to do, but if people are unwilling to accept the change in > > project structure you could achieve something similar by using maven > > profiles. With profiles the project structure would remain as is. > Profiles > > could be added to the malhar pom, and a profile would define the > > dependencies needed for different types of operators. For example the > hbase > > profile would define the dependencies for the hbase operator. Then any > > project using a malhar library would just activate the correct profile in > > it's pom, and the correct dependencies would be pulled in. > > > > > http://maven.apache.org/guides/introduction/introduction-to-profiles.html > > > > On Thu, Sep 17, 2015 at 10:01 AM, Andy Perlitch <[email protected]> > > wrote: > > > > > Hi everyone, > > > > > > I am currently assigned to MLHR-1843 > > > <https://malhar.atlassian.net/browse/MLHR-1843>, which essentially > aims > > to > > > expose smaller, more consumable maven artifacts that would do away with > > the > > > need to manually include necessary dependencies based on the operators > in > > > use. > > > > > > As an example, say I am building an app package that needs Kafka input > > and > > > output operators, but I don't want all the other transitive > dependencies > > > that come via malhar-contrib. Currently I would need to specify > > > malhar-contrib as a dependency, and add an exclusions block in my app > > > package pom: > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > <artifactId>malhar-contrib</artifactId> <version>3.0.0</version> <!-- > > so > > > none of malhar-contrib's deps are included -->* > > > > > > > > > > > > > > > > > > > > > * <exclusions> <exclusion> <groupId>*</groupId> > > > <artifactId>*</artifactId> </exclusion> </exclusions></dependency>* > > > > > > Then, I would have to include the kafka library explicitly as a > > dependency: > > > > > > > > > > > > > > > > > > *<dependency> <groupId>org.apache.kafka</groupId> > > > <artifactId>kafka_2.10</artifactId> > > > <version>0.8.1.1</version></dependency>* > > > > > > Wouldn't it be nice if I could just put this in my pom?: > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > <artifactId>malhar-contrib-kafka</artifactId> > > > <version>3.0.0</version></dependency>* > > > > > > > > > In order to make this possible, we will need to organize the malhar > > project > > > into more granular modules (artifacts). Specifically, the > malhar-contrib > > > artifact would essentially just be a pom that specifies each smaller > > module > > > as a dependency: > > > > > > *<!-- in malhar-contrib's pom.xml: -->* > > > > > > *<modules> <module>kafka</module>* > > > * <module>twitter</module>* > > > * <module>redis</module>* > > > > > > * <!-- other smaller modules --></modules>* > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > <artifactId>malhar-contrib-kafka</artifactId> > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > <artifactId>malhar-contrib-twitter</artifactId> > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > <artifactId>malhar-contrib-redis</artifactId> > > > <version>3.0.0</version></dependency>* > > > > > > With these changes, there may be a risk of breaking backwards > > > compatibility, however I think the gain in usability of malhar merits > the > > > effort to make this work. > > > > > > I am still relatively new to maven, so I would love to get some > feedback > > > from other devs about this! > > > > > > -- > > > Regards, > > > Andy Perlitch > > > Software Engineer > > > DataTorrent Inc > > > (408)829-9319 > > > > > >
