As there’s really only one community for AsterixDB and Hyracks (inside the ASF and before that outside of the ASF), I think that it’s better to refer to them as 2 different products, not 2 different projects. They previously have been in 2 separate repositories and they are in 2 repositories right now. They also have been released separately before (e.g. when a specific change in Hyacks was needed for another dependent like VXQuery and AsterixDB was not ready for a release). But often the releases were in-sync, as AsterixDB clearly is the biggest consumer of Hyracks and the biggest driver for change. The challenge of having them in 2 separate repositories is, that it is sometimes inconvenient for development/reviews in AsterixDB if a new development/bug fix requires changes in both products. I think that putting both into the same repository would 1) ease some parts of AsterixDB development and 2) potentially muddy the waters between AsterixDB and Hyracks.
I also think that the separation of concerns between Hyracks and AsterixDB is a great feature. And the fact that Pregelix and VXQuery work well on top Hyracks is an indicator that the separation is well designed. I agree with Chris (especially with my VXQuery hat on :) ) that it would be a better direction to move the development model in a direction where we improve for all dependent projects instead of giving one dependent special treatment. On the mechanical side, if we had both products in one repository, what would the relationship between the release tag and the release artifact look like? Would we tag the whole repository and the release artifact would agree with a subtree of the repository only? Or is there a way to only tag part of a repository? How does Mahout handle this? Cheers, Till > On May 10, 2015, at 4:47 AM, Ted Dunning <[email protected]> wrote: > > Putting the projects into the same repository says nothing about linking > releases. > > A single Apache project can have multiple released artifacts. For > instance, Mahout has mahout-math, mahout-collections, mahout-core, > mahout-samsara. These releases only include their own source code. > > Yes, the commit stream on master would have both kinds of commits, but that > is pretty non-fatal. If you want to isolate the projects you can have two > threads, each with only a single set of source code, but that seems strange > and obscure in this case. > > In Apache parlance, a sub-project usually refers to having a disjoint set > of committers. That is discouraged, even though it often happens at least > temporarily when new code bases are imported. Having multiple released > artifacts is entirely a separate matter and is a good idea in many cases. > > > > > > On Sat, May 9, 2015 at 12:49 AM, Chris Hillery <[email protected]> > wrote: > >> Apache-specific issues aside, I must admit it would be a bit disappointing >> to have to join Hyracks and Asterix into a single project base. It would be >> convenient, but convenience breeds apathy. We solve the cross-product >> releasing issues for Asterix, which makes us less likely to buckle down and >> solve them for other Hyracks consumers like VXQuery and hopefully others in >> the future. >> >> Ceej >> aka Chris Hillery >> >> On Fri, May 8, 2015 at 2:47 PM, Ian Maxon <[email protected]> wrote: >> >>> I see your point, that is true. In this case a release of just Hyracks >>> would also be visible in the AsterixDB commit log and vice-versa. I'm not >>> certain what this means (or if it matters) on the Apache front. Is >> having a >>> sub-project, that keeps its own version an unprecedented thing? >>> >>> Agreed about not rushing through with this though. I think we should >>> certainly wait until after the upcoming 0.8.7 release to actually commit >> to >>> any of this. >>> >>> -Ian >>> >>> >>> >>> On Fri, May 8, 2015 at 2:29 PM, Till Westmann <[email protected]> wrote: >>> >>>> I'm not sure about that. An Apache release will be a source code >> release >>>> and not a binary release. We can have binary "convenience artifacts", >> but >>>> the official release is the source release. >>>> Usually source releases are tagged in revision control such that the >>>> content of the source archive agrees with the tag. Now if we have all >> the >>>> code in a same repository, I am not sure how that will work. I'm not >>> saying >>>> that it doesn't work, but I'm not sure how to do that. >>>> I think that it would be good to make a full Apache release of both >>>> projects first, such that we have a clear understanding how to do that >>>> before we change the project layout. >>>> >>>> Cheers, >>>> Till >>>> >>>> >>>> On 8 May 2015, at 13:58, Ian Maxon wrote: >>>> >>>> Releasing would be the same, probably simpler actually. I suppose I >>>>> haven't >>>>> tried it so I can't be totally certain, but performing 'mvn release' >> in >>> a >>>>> module directly doesn't do anything different than when it is run >> from a >>>>> higher-up pom as a submodule. Nothing would change if a user is >>> dependent >>>>> on a stable version of Hyracks, because they only ever see binary >>>>> artifacts >>>>> from Maven. 'hyracks' will still be called 'hyracks' and have the same >>>>> coordinates in Maven. >>>>> >>>>> - Ian >>>>> >>>>> On Fri, May 8, 2015 at 1:47 PM, Till Westmann <[email protected]> >> wrote: >>>>> >>>>> Hmm, and what do we do about the other dependents of Hyracks (e.g. >>>>>> VXQuery)? >>>>>> We had separate releases of Hyracks for those in the past. >>>>>> How would releases (branching, tagging ...) work in that case? >>>>>> >>>>>> Cheers, >>>>>> Till >>>>>> >>>>>> On 8 May 2015, at 13:17, Ian Maxon wrote: >>>>>> >>>>>> Hi all, >>>>>>> An idea was brought up today in the meeting (I believe by Yingyi) >> for >>>>>>> solving the issues we have right now with maven project >>>>>>> >>>>>> interdependencies. >>>>>> >>>>>>> The idea is to just merge AsterixDB and Hyracks into one git >>> repository, >>>>>>> and to have them as separate maven projects with a top level pom >>> joining >>>>>>> them. We actually have part of this implemented already (in the tlp/ >>>>>>> >>>>>> folder >>>>>> >>>>>>> a pom.xml exists for this). Doing this change would eliminate the >>>>>>> >>>>>> necessity >>>>>> >>>>>>> of the topic field hack in Gerrit, as well as ensure changes in >>> Hyracks >>>>>>> don't break AsterixDB. >>>>>>> >>>>>>> I went ahead and made a branch that has this change implemented, >>> please >>>>>>> take a look at >>>>>>> >>>>>>> >>>>>> >>> >> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hyracks-merge >>>>>> >>>>>>> to get an idea of what's proposed. I merged the Hyracks repository >>> into >>>>>>> a >>>>>>> subtree of the asterix repository- so all of the commit history is >>>>>>> merged >>>>>>> properly. I think we would want to not commit this change through >>>>>>> Gerrit, >>>>>>> because if we did all of the Hyracks commit history would not be >>>>>>> >>>>>> included, >>>>>> >>>>>>> which would be unfortunate. >>>>>>> >>>>>>> - Ian >>>>>>> >>>>>> >>>>>> >>> >>
