Hmm, good point. It doesn’t have to. One question might be if we can push it to some maven repository, if it’s not an official release. But I think that should also be fine as long as we don’t push it to a repository that claims to contain official releases.
Some mentor input might be helpful on this as well :) Cheers, Till > On Jun 7, 2015, at 6:53 PM, Ildar Absalyamov <[email protected]> > wrote: > > Does version bump always mean full-fledged Apache release? We need the former > just to resolve compile time dependencies. > >> On Jun 7, 2015, at 18:49, Till Westmann <[email protected]> wrote: >> >> In principle I agree with this, but creating a new release will be a little >> more involved that just running maven, when we do this at the ASF. >> To publish a new release we will have to vet and vote on the release. This >> takes at least 72 hours in the best case if we’re a TLP, the first release >> candidate is great, and have enough people to vote. While we’re still in the >> incubator, releasing will take a little longer as we also have to get enough >> votes for the release in the incubator. >> As I proposed earlier, it would be really good to go through the full >> release process once, before we decide how to structure our processes and >> infrastructure. >> >> Cheers, >> Till >> >>> On Jun 4, 2015, at 6:37 PM, Ildar Absalyamov <[email protected]> >>> wrote: >>> >>> I am with Chris on repository separation and I think that the solution to >>> the issue of Hyracks commits breaking Asterix build is using release >>> Hyracks versions instead of snapshot ones. Yes, that will create a frequent >>> Hyracks releases (we will have to release it each time there is a change >>> which spans both Hyracks & Asterix) and we have abandoned this practice a >>> while ago, but it seems that’s the only way to separate projects logically. >>> >>> Here are few examples to clear the picture. In all examples Hyracks version >>> is 4.5.6-Snapshot, Asterix version is 1.2.3-Snapshot (but it depends on >>> previous release version Hyracks 4.5.5): >>> 1) The changes span both Asterix & Hyracks. >>> First make sure that Asterix could depend on Hyracks 4.5.6-Snapshot without >>> API conflicts & switch Asterix dependency to 4.5.6-Snapshot. >>> Submit Gerrit review, once it is done as a part of git-asf script commit >>> changes, bump Hyracks version to 4.5.6, make Asterix depend on 4.5.6 and >>> bump Hyracks to 4.5.7-Snapshot right after. >>> 2) The changes are located only in Hyracks. Regular review and commit (with >>> snapshot version) without any version bump. >>> 3) The changes are located only in Asterix. Regular review and commit (with >>> snapshot version) without any version bump. >>> >>> In this scenario Hyracks commit can never make Asterix build fail (since it >>> depends on a stable release) and it’s the responsibility of the first >>> person, whose commits spans both repos to make sure that the changes in >>> snapshot Hyracks version are properly merged. >>> >>> Regarding the Yingyi’s issue with Gerrit topics: could we modify git-gerrit >>> script so it would submit both Asterix & Hyracks reviews (granted that the >>> latter is needed), and link them together, setting the proper topic? Gerrit >>> seems to have API for changing that, right? >>> >>>> On Jun 4, 2015, at 15:45, Mike Carey <[email protected]> wrote: >>>> >>>> Just a quick high-level note from our nearest equivalent of the >>>> pointy-haired Dilbert guy (aka me): What would be nice is to have Hyracks >>>> changes kick off tests of all "supported client projects" - AsterixDB, >>>> VXQuery, maybe also Pregelix, IMRU, and possibly others in the future. I >>>> don't think we'll ever prevent such downstream things from being broken >>>> unless we run their tests - so I would suggest that we need a mechanism to >>>> keep Hyracks changes from being permitted to happen without verifying the >>>> ongoing integrity of all "blessed" (priority 1) affected projects.... We >>>> could have an agreed upon list of such projects and tests for each.... It >>>> would be nice to have a "quick check" (hello world still works, basics are >>>> working) that was synchronously blocking of such changes, and at least a >>>> daily verification that all's totally well (AFAWK) for them all. >>>> >>>> Not sure how this affects the still two-sided discussion... :-) >>>> >>>> Cheers, >>>> Mike >>>> >>>> >>>> On 6/2/15 10:00 AM, Chris Hillery wrote: >>>>> On Mon, Jun 1, 2015 at 9:46 PM, Yingyi Bu <[email protected]> wrote: >>>>> >>>>>> In my opinion, merging the repository doesn't break the separation of >>>>>> hyracks and asterixdb, because the dependencies are controlled by mvn pom >>>>>> files. >>>>>> >>>>> That wasn't the separation I was talking about. I meant API separation. As >>>>> it is now, when we make a change to both Asterix and Hyracks, we are >>>>> forced >>>>> to consider the API implications, or at least they are put out there in a >>>>> very clear way that we need to look at. If we merge them, people will >>>>> (rightly) treat the whole thing as one product, and there will be no >>>>> brakes >>>>> on making wide-ranging API changes. >>>>> >>>>> (As an aside: I don't trust Maven's pom files to do a good job of keeping >>>>> the dependency management clean. In fact I trust it to do precisely the >>>>> opposite, by making it both easier to screw up the dependencies and harder >>>>> to update them in future.) >>>>> >>>>> Again, my point is this: If we truly believe that Hyracks is a re-usable >>>>> component, it should be treated as such from source to build to delivery. >>>>> By merging in Asterix, we are saying that Asterix is "more equal" than >>>>> others Hyracks clients, to the point that we're tacitly willing to break >>>>> those other clients in favor of simplifying Asterix development. If that >>>>> is >>>>> a fair and true statement, well, then, sure, let's merge them. >>>>> >>>>> 1) It forces those hyracks-only changes to pass asterixdb regression >>>>>> tests. Currently hyracks-only change are not verified by asterixdb >>>>>> tests. >>>>>> >>>>> This is a good point, I will admit. However, I think this same goal can be >>>>> met in other ways. My strong preference would be to create a set of true >>>>> API tests inside of Hyracks, which both document and test the external >>>>> Hyracks API. That will make API-breaking changes in future much easier to >>>>> spot, and also make it clear when Asterix is using internal APIs that it >>>>> should not. >>>>> >>>>> >>>>>> 2) On my local machine, I don't need to always install hyracks and then >>>>>> verify asterixdb from time to time. Especially, switching branches seems >>>>>> painful because the installed hyracks snapshot is overwritten from time >>>>>> to >>>>>> time. >>>>>> >>>>> I haven't tried working on multiple Hyracks branches at the same time, so >>>>> I >>>>> haven't experienced this. This seems like a working method error, though. >>>>> If you're working with two things that are "the same version" (even if >>>>> that's a snapshot version), you'll need to use separate Maven repositories >>>>> to install them. In fact, merging the two git repositories would do >>>>> nothing >>>>> to fix this problem, will it? If the proposal is to put the two source >>>>> repositories in the same git repo but otherwise leave them untouched, then >>>>> nothing would change in the build process. It's possible I'm missing >>>>> something there, though. >>>>> >>>>> >>>>>> 3) I only need to make one code review request and one jenkins job. >>>>>> Currently I need to manually change the topic of my asterixdb gerrit CL >>>>>> every time before I update my hyracks CL, and then manually schedule >>>>>> jenkins to run a new asterixdb job. If I forget to schedule the jenkins >>>>>> job, the asterixdb CL is still shown to be "verified by jenkins". >>>>>> >>>>> This is a problem, but it's a problem in commit validation, not in the >>>>> source. Modifying the source to work around these issues is still a bad >>>>> idea IMHO. >>>>> >>>>> The "change-topic" issue could be fixed with a bit of development work >>>>> (have the topic point to a change, rather than a specific patchset on the >>>>> change, so you only need to set it once, for instance). >>>>> >>>>> As for manually scheduling Asterix Jenkins jobs, that sounds like it's >>>>> only >>>>> a problem where your Hyracks change breaks an existing public API. That >>>>> would be obviated by having true API testing inside of Hyracks, which is >>>>> something that we should have regardless of any decisions about source >>>>> locations. >>>>> >>>>> In summary / repeating myself again: yes, we have some problems because >>>>> Hyracks and Asterix are in seperate repositories. But those problems are >>>>> pointing out true issues with our development and processes. Merging the >>>>> repositories isn't fixing those problems, it's sweeping them under the >>>>> rug. >>>>> Long term we would be much better off to identify, isolate, and fix the >>>>> problems themselves. >>>>> >>>>> Ceej >>>>> aka Chris Hillery >>>>> >>>> >>> >>> Best regards, >>> Ildar >>> >> > > Best regards, > Ildar >
