+1 On Tue, Sep 22, 2015 at 11:08 AM, Gaurav Gupta <[email protected]> wrote:
> I agree with David.. Each artifact should have it's own version > > Thanks > -Gaurav > > On Tue, Sep 22, 2015 at 11:07 AM, David Yan <[email protected]> wrote: > > > I actually think that each baby artifact should have its own version, > > because each artifact has its own interface and its own life cycle, > > especially after we break up the giant library, applications will depend > on > > the baby artifacts instead of the giant library. For example if there is > > no change in malhar-contrib-kafka (I think the name should actually be > > apex-malhar-kafka), we should not confuse users by bumping the version. > > > > David > > > > On Tue, Sep 22, 2015 at 9:03 AM, Andy Perlitch <[email protected]> > > wrote: > > > > > Tushar, > > > > > > I agree that all modules should inherit the version from the "parent > pom" > > > of the malhar repo. I think the benefits outweigh the cost of bumping > > > versions of components that haven't actually changed. I'd love to get > > > others feedback on this as well. > > > > > > On another note, I plan on starting a spreadsheet/googledoc with the > > > possible groupings of operators into these modules. Stay tuned... > > > > > > -Andy > > > > > > On Mon, Sep 21, 2015 at 11:51 PM, Tushar Gosavi < > [email protected]> > > > wrote: > > > > > > > +1 for the general idea > > > > > > > > Does these independent modules going to have independent versions? > For > > > > example, if there is no change in kafka operator between malhar 3.0 > and > > > > malhar 4.0, will we increment version of malhar-contrib-kafka to > 4.0. I > > > > have learned from my previous project that, It is easier to manage > > > versions > > > > if we make all modules at same version level for a release, even if > > there > > > > is no change in a particular module. > > > > > > > > - Tushar. > > > > > > > > > > > > > > > > On Fri, Sep 18, 2015 at 12:18 AM, Timothy Farkas < > [email protected]> > > > > wrote: > > > > > > > > > I agree Andy's solution is better, but just for the sake of > argument > > > > > profiles can be inherited from a parent pom, so if the maven > > archetype > > > > > defines a new project with a parent pom with the correct profiles > > > > defined, > > > > > then the desired profiles can be activated in the pom of the new > > > project. > > > > > It is no more complicated than adding additional dependencies to > your > > > > > project. > > > > > > > > > > On Thu, Sep 17, 2015 at 10:32 AM, Sandesh Hegde < > > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > Currently all the dependencies in Malhar-Contrib are marked as > > > > optional. > > > > > So > > > > > > users have to already modify the existing POM to use it in their > > > > project. > > > > > > So restructuring should be fine. > > > > > > > > > > > > On Thu, Sep 17, 2015 at 11:29 AM Chetan Narsude < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > The profiles are excellent when you are developing > > malhar-contrib. > > > > > > Profiles > > > > > > > do not work when you are using malhar-contrib. The problem Andy > > is > > > > > > trying > > > > > > > to solve is the later. If there is an elegant solution which I > am > > > > > missing > > > > > > > using profiles, please correct me. > > > > > > > > > > > > > > The way Andy suggested is the way many successful projects do > it. > > > > Look > > > > > at > > > > > > > Netty as an example. > > > > > > > > > > > > > > +1 for that. > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Chetan > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 17, 2015 at 11:22 AM, Timothy Farkas < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > I think restructuring the project in that way would be the > > > > > technically > > > > > > > > correct thing to do, but if people are unwilling to accept > the > > > > change > > > > > > in > > > > > > > > project structure you could achieve something similar by > using > > > > maven > > > > > > > > profiles. With profiles the project structure would remain as > > is. > > > > > > > Profiles > > > > > > > > could be added to the malhar pom, and a profile would define > > the > > > > > > > > dependencies needed for different types of operators. For > > example > > > > the > > > > > > > hbase > > > > > > > > profile would define the dependencies for the hbase operator. > > > Then > > > > > any > > > > > > > > project using a malhar library would just activate the > correct > > > > > profile > > > > > > in > > > > > > > > it's pom, and the correct dependencies would be pulled in. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://maven.apache.org/guides/introduction/introduction-to-profiles.html > > > > > > > > > > > > > > > > On Thu, Sep 17, 2015 at 10:01 AM, Andy Perlitch < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > I am currently assigned to MLHR-1843 > > > > > > > > > <https://malhar.atlassian.net/browse/MLHR-1843>, which > > > > essentially > > > > > > > aims > > > > > > > > to > > > > > > > > > expose smaller, more consumable maven artifacts that would > do > > > > away > > > > > > with > > > > > > > > the > > > > > > > > > need to manually include necessary dependencies based on > the > > > > > > operators > > > > > > > in > > > > > > > > > use. > > > > > > > > > > > > > > > > > > As an example, say I am building an app package that needs > > > Kafka > > > > > > input > > > > > > > > and > > > > > > > > > output operators, but I don't want all the other transitive > > > > > > > dependencies > > > > > > > > > that come via malhar-contrib. Currently I would need to > > specify > > > > > > > > > malhar-contrib as a dependency, and add an exclusions block > > in > > > > my > > > > > > app > > > > > > > > > package pom: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > > > > > > > <artifactId>malhar-contrib</artifactId> > > > <version>3.0.0</version> > > > > > > <!-- > > > > > > > > so > > > > > > > > > none of malhar-contrib's deps are included -->* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > * <exclusions> <exclusion> <groupId>*</groupId> > > > > > > > > > <artifactId>*</artifactId> </exclusion> > > > > > > </exclusions></dependency>* > > > > > > > > > > > > > > > > > > Then, I would have to include the kafka library explicitly > > as a > > > > > > > > dependency: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>org.apache.kafka</groupId> > > > > > > > > > <artifactId>kafka_2.10</artifactId> > > > > > > > > > <version>0.8.1.1</version></dependency>* > > > > > > > > > > > > > > > > > > Wouldn't it be nice if I could just put this in my pom?: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > > > > > > > <artifactId>malhar-contrib-kafka</artifactId> > > > > > > > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > > > > > > > > > > > > > In order to make this possible, we will need to organize > the > > > > malhar > > > > > > > > project > > > > > > > > > into more granular modules (artifacts). Specifically, the > > > > > > > malhar-contrib > > > > > > > > > artifact would essentially just be a pom that specifies > each > > > > > smaller > > > > > > > > module > > > > > > > > > as a dependency: > > > > > > > > > > > > > > > > > > *<!-- in malhar-contrib's pom.xml: -->* > > > > > > > > > > > > > > > > > > *<modules> <module>kafka</module>* > > > > > > > > > * <module>twitter</module>* > > > > > > > > > * <module>redis</module>* > > > > > > > > > > > > > > > > > > * <!-- other smaller modules --></modules>* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > > > > > > > <artifactId>malhar-contrib-kafka</artifactId> > > > > > > > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > > > > > > > <artifactId>malhar-contrib-twitter</artifactId> > > > > > > > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *<dependency> <groupId>com.datatorrent</groupId> > > > > > > > > > <artifactId>malhar-contrib-redis</artifactId> > > > > > > > > > <version>3.0.0</version></dependency>* > > > > > > > > > > > > > > > > > > With these changes, there may be a risk of breaking > backwards > > > > > > > > > compatibility, however I think the gain in usability of > > malhar > > > > > merits > > > > > > > the > > > > > > > > > effort to make this work. > > > > > > > > > > > > > > > > > > I am still relatively new to maven, so I would love to get > > some > > > > > > > feedback > > > > > > > > > from other devs about this! > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Regards, > > > > > > > > > Andy Perlitch > > > > > > > > > Software Engineer > > > > > > > > > DataTorrent Inc > > > > > > > > > (408)829-9319 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Regards, > > > Andy Perlitch > > > Software Engineer > > > DataTorrent Inc > > > (408)829-9319 > > > > > >
