I've fixed the problems Aled observed with the `all.sh` script and the problem with previous branches.
I've also added PR's which: * set up submodules: https://github.com/ahgittin/brooklyn/pull/5 * add docs for using multi-repos and submodules Key things: * the submodules are there, and work well (provided you read about it) -- but they don't get in the way if you don't want to use them * the docs also describe an easy way to move changes made against incubator-brooklyn (you lose the history but `git diff` then `patch -p1` does what I'd hoped; that's how I made these PR's! dogfooding FTW!) Obviously these are test PR's as we'll be migrating from incubator again on Saturday (assuming all goes well). I wanted to test the processes and the submodules, and have docs ready for after the merge. I think it's going to be relatively smooth from here ... famous last words. Please let me know how they work for you -- or if you object to a Saturday migration for any reason! Best Alex On 28 January 2016 at 12:23, Alex Heneveld <[email protected]> wrote: > > Thanks Aled. > > > branches="0.4.0 0.5.0 0.6.0 0.7.0-incubating 0.8.0-incubating" > > I suggest adding 0.8.x. > > 0.8.x has the old structure so it will be hard to work with here, and > unless we release an 0.8.1 it will be noise. I'm expecting instead we can > get an 0.9.0 out soon. The other repos also have the same problem of the > old structure, split among the repos. Plus another problem I've noticed > that the branches didn't populate correctly. (I'll fix that!) > > We can add new branches if we decide we do want 0.8.x or 0.8.1. > > > I'm also interested in whether two runs of all.sh will give identical > git sha1 commit ids for each new repo. > > It doesn't. We considered amending the git commit *messages* to refer to > the original ID but decided against it as it is easy enough to correlate > them based on date, sequence, content, and author+committer, all of which > are preserved. I don't think preserving the commit ID will help any > further with provenance. > > > The all.sh is running in the background - I'll report on how that does. > > The scripts are throwaway -- sorry for the glitches though -- so no need > for others to go through the pain. I'm more interested in whether the new > repos look as expected. > > Best > Alex > > > > On 28/01/2016 11:53, Aled Sage wrote: > >> Thanks Alex, sounds great. >> >> I'm running through the brooklyn-repo-split release script locally now, >> to reproduce. >> >> A few initial comments... >> >> --- >> https://github.com/ahgittin/brooklyn-repo-split/blob/master/README.md >> needs updated. >> >> It says that "Steps 1 and 2 are done and checked in as reorg* branches at >> https://github.com/ahgittin/incubator-brooklyn";. There is no "reorg*" >> branch in ahgittin/incubator-brooklyn. I believe the results of >> 1-rearrange-incubator is now in github.com/apache/incubator-brooklyn, >> and that step 2 needs to be run (which matches all.sh). >> >> --- >> all.sh fails because reorg branch does not exist. >> >> Suggest changing: >> >> ( cd incubator-brooklyn && git pull && git branch -D reorg && git >> checkout -b reorg ) >> >> to: >> >> ( cd incubator-brooklyn && git pull && ( git branch -D reorg || true >> ) && git checkout -b reorg ) >> >> --- >> It would be good to document how to run it a bit more (in README.md). For >> example: >> >> In env.sh, set REPO_ORG to your own github id. >> >> In the root directory of brooklyn-repo-split, run: >> git clone https://github.com/apache/incubator-brooklyn.git >> >> And then run `nohup all.sh > all.out 2>&1 &` >> >> --- >> Looking at https://github.com/apache/incubator-brooklyn/branches/all, >> there are branches not mentioned in env.sh branches: >> >> branches="0.4.0 0.5.0 0.6.0 0.7.0-incubating 0.8.0-incubating" >> >> I suggest adding 0.8.x. >> >> Any objections from the community of deleting the others (I'm in >> agreement)? We'll assume lazy consensus on that one. >> >> --- >> The all.sh is running in the background - I'll report on how that does. >> >> --- >> I'm also interested in whether two runs of all.sh will give identical git >> sha1 commit ids for each new repo. >> >> Assuming it does, that is a great way to prove the providence of the new >> repos if we are every legally asked in the future (i.e. it is entirely >> deterministically reproducible, so cannot have included any manual >> modifications where additional code was inserted/removed). >> >> Aled >> >> >> On 28/01/2016 10:09, Alex Heneveld wrote: >> >>> >>> Brooklyners- >>> >>> TL;DR: *switch to new repos at the weekend, incubator commits cut-off >>> proposed for Sat 10am UK* >>> >>> The new project structure seems to be working well and I think it's time >>> to move to the new repos apache/brooklyn and apache/brooklyn-*, and then >>> retire the incubator project. >>> >>> I've built on Richard's separation scripts, with the current version at >>> https://github.com/ahgittin/brooklyn-repo-split, and it also seems to >>> be working very well. You can inspect the resulting projects at these >>> URL's: >>> >>> https://github.com/ahgittin/brooklyn >>> https://github.com/ahgittin/brooklyn-dist >>> https://github.com/ahgittin/brooklyn-docs >>> https://github.com/ahgittin/brooklyn-library >>> https://github.com/ahgittin/brooklyn-server >>> https://github.com/ahgittin/brooklyn-ui >>> >>> You can try them for yourself e.g. using: >>> >>> for x in brooklyn{,-{dist,docs,library,server,ui}} ; do >>> git clone [email protected]:ahgittin/$x.git >>> done >>> cd brooklyn/ >>> mvn clean install >>> >>> I have done a lot of checking that these are all healthy -- see below -- >>> but I'd value some others also casting their eyes over the projects. If >>> there are other checks I should do when I run them again please let me know. >>> >>> If this looks good I propose we wait until the weekend to cut over. If >>> there is no objection we would STOP committing to the incubator project at >>> Saturday 10am UK, and I will re-run the scripts, test, and push to apache/ >>> repos instead of ahgittin. >>> >>> Meanwhile I have some notes on migrating incubator PR's and branches to >>> the new repos and on using subprojects which I will complete and circulate. >>> >>> Best >>> Alex >>> >>> >>> CHECKS I'VE DONE >>> >>> A) The command: >>> >>> git log --oneline --follow `find . -name AbstractEntity.java` >>> >>> gives the same output modulo commit ID's and excluding the additional >>> directory promotion commit in the new repo, 415 commits total >>> >>> B) `find .` gives the same output, modulo items in the root and the >>> .git/ dirs (actual command: `find -E . \! -regex '.*/\.git(/.*)?' \! -regex >>> '\./[[:alnum:]\.]+'`) >>> >>> C) Both builds work and the built artifacts are identical except for >>> MANIFEST.MF (Implementation-SHA-1 and Bnd-LastModified) and pom.properties >>> (timestamp) >>> >>> D) Size of project is the same and history is substantially smaller: >>> >>> incubator: >>> >>> 262M ./.git >>> 44K ./brooklyn >>> 652K ./brooklyn-dist >>> 16M ./brooklyn-docs >>> 9.6M ./brooklyn-library >>> 19M ./brooklyn-server >>> 5.1M ./brooklyn-ui >>> 312M . >>> of which 262M is */.git, 50M current >>> >>> new: >>> >>> 180K ./brooklyn/.git >>> 224K ./brooklyn >>> 812K ./brooklyn-dist/.git >>> 1.4M ./brooklyn-dist >>> 20M ./brooklyn-docs/.git >>> 36M ./brooklyn-docs >>> 15M ./brooklyn-library/.git >>> 24M ./brooklyn-library >>> 36M ./brooklyn-server/.git >>> 55M ./brooklyn-server >>> 7.6M ./brooklyn-ui/.git >>> 13M ./brooklyn-ui >>> 129M . >>> of which 79M is */.git, 50M current >>> >>> The size improvement comes of course from big WAR artifacts in ancient >>> history which aren't being copied across. >>> >>> The remaining size is mostly accounted for by: >>> * screenshots in docs >>> * some big JS in library/sandbox history and in ui (a bit of a shame as >>> when we move to bower/grunt these deps won't be part of the repo, but a few >>> megs isn't really that much) >>> * lots of files in server and its history (none esp big, nearly all >>> needed) >>> >>> END >>> >>> >>> >> >> >
