The job push-to-releases [1] is responsible for propagation of new changesets from the main-silver default branch to the default branch in the releases repo.
Regards, PetrG [1] http://deadlock.netbeans.org/view/Push%20To%20Team%20Repository%20builds/job/push-to-releases/ On Dec 21, 2016, at 12:06 PM, Emilian Bold <[email protected]> wrote: > This is a good point. > > I somehow believed that releases/ only has the releases (in the _fcs > branches,etc) while the main development is still in mail-silver (and the > team repositories). > > Only yesterday I noticed that releases/ seems to have in the default brach > recent commits. > > Except it didn't occur to me I could just pull from releases/ into a > main-silver clone! I will try it out. > > > --emi > > On Wed, Dec 21, 2016 at 1:01 PM, Jaroslav Tulach <[email protected] >> wrote: > >> On úterý 20. prosince 2016 15:47:08 CET Emilian Bold wrote: >>> You guys weren't very clear which repository are reviewing for the >> donation. >>> >>> Initially I converted releases/ but then I released I have some commits >> of >>> mine in there so I couldn't push it to github. >>> >>> So I did a fresh conversion for main-silver/ which I assumed is the >>> repository Oracle will donate. >>> >>> I will re-do releases/ but it will take a bit. >> >> Hello Emilian, >> I want to point out that main-silver and releases aren't that different. In >> fact every commit inside of main-silver (like http://hg.netbeans.org/main- >> silver/rev/f966be3cb73a) is also available in releases (at http:// >> hg.netbeans.org/releases/rev/f966be3cb73a). >> >> However releases is more important, as it contains more. Especially the >> actual >> sources used for individual NetBeans releases - like >> http://hg.netbeans.org/ >> releases/rev/release82_fcs - those aren't in main-silver. >> >> As these repos are 90% the same, I thought it is enough to keep the >> repository >> you already have (https://github.com/emilianbold/main-silver) and just >> migrate >> rest of releases on top of it. >> -jt >> >>> On Tue, Dec 20, 2016 at 3:31 PM, Jaroslav Tulach < >> [email protected] >>>> wrote: >>>> >>>> On pátek 9. prosince 2016 19:05:48 CET Emilian Bold wrote: >>>>> Martin, I have just pushed https://github.com/ >> emilianbold/main-silver >>>> >>>> You >>>> >>>>> may experiment with that. >>>> >>>> Hello Emilian, >>>> I managed to fork & use your repository and everything seems great. I >> have >>>> a >>>> functional job that executes >>>> >>>> $ ant build-platform >>>> >>>> for each pull request. I plan to add a call to "ant test-platform" >> once it >>>> is >>>> stable enough[1]. Great work! I believe we shall use your Git >> repository >>>> as a >>>> base (somehow) when donating code to Apache once my Oracle peers finish >>>> review >>>> of the code to donate. >>>> >>>>> To https://github.com/emilianbold/main-silver.git >>>>> >>>>> * [new branch] master -> master >>>>> >>>>> Branch master set up to track remote branch master from origin. >>>> >>>> Could you synchronize https://hg.netbeans.org/releases/ instead? It >>>> contains >>>> history of all the NetBeans releases (in branches like release82, etc.) >>>> and it >>>> is the repository that is currently under the review. Btw. The releases >>>> repository contains everything that is available in the main-silver - >> just >>>> more. >>>> >>>> It would be fantastic, if you could create a complete mirror of the >>>> releases >>>> repository. Thanks again for your great work! >>>> >>>> -jt >>>> >>>> [1] http://deadlock.netbeans.org/job/prototypes-MavenDownload269264/ - >>>> still >>>> 12 test failures remaining >>>> >>>>> git push -u origin master >>>>> Counting objects: 3951610, done. >>>>> Delta compression using up to 8 threads. >>>>> Compressing objects: 100% (732965/732965), done. >>>>> Writing objects: 100% (3951610/3951610), 674.94 MiB | 717.00 KiB/s, >>>>> done. >>>>> Total 3951610 (delta 2068729), reused 3951610 (delta 2068729) >>>>> remote: Resolving deltas: 100% (2068729/2068729), done. >>>>> remote: Checking connectivity: 3951610, done. >>>>> remote: warning: GH001: Large files detected. You may want to try Git >>>> >>>> Large >>>> >>>>> File Storage - https://git-lfs.github.com. >>>>> remote: warning: See http://git.io/iEPt8g for more information. >>>>> remote: warning: File dlight.util/test/manual/ >> DLight_Simple_Tests/core >>>> >>>> is >>>> >>>>> 51.88 MB; this is larger than GitHub's recommended maximum file size >> of >>>>> 50.00 MB >>>>> To https://github.com/emilianbold/main-silver.git >>>>> >>>>> * [new branch] master -> master >>>>> >>>>> Branch master set up to track remote branch master from origin. >>>>> >>>>> >>>>> --emi >>>>> >>>>> On Wed, Dec 7, 2016 at 4:42 PM, Martin Balin < >> [email protected]> >>>>> >>>>> wrote: >>>>>> Hello Emilian, >>>>>> I'm working at Oracle on NetBeans development and we would like to >>>> >>>> start >>>> >>>>>> fixing build scripts to use Git instead of HG. >>>>>> This could be done earlier on your Git repo if you agree to as it >> will >>>>>> take time. Does not need to wait for final official donation of >>>> >>>> sources. >>>> >>>>>> Can you please send me the URL,... >>>>>> Thank you Martin Balin >>>>>> >>>>>> On 24.11.2016 20:07, Emilian Bold wrote: >>>>>>> At under 1GB the repository size is not an issue anymore. >>>>>>> >>>>>>> It's sad to see we will still have migration problems due to legal >>>>>>> considerations. >>>>>>> >>>>>>> Could you provide an estimate how long it would take to verify and >>>>>>> whitelist the entire codebase Oracle plans on donating? >>>>>>> >>>>>>> It's unclear to me how history would be preserved with an >> incremental >>>>>>> approach. >>>>>>> >>>>>>> I would prefer we migrate the whole thing in one piece with >> history >>>> >>>> and >>>> >>>>>>> all. >>>>>>> >>>>>>> >>>>>>> --emi >>>>>>> >>>>>>> On Thu, Nov 24, 2016 at 5:22 PM, Jaroslav Tulach < >>>>>>> [email protected] >>>>>>> >>>>>>>> wrote: >>>>>>>> Emilian, Jan, Mark, great work. >>>>>>>> >>>>>>>> Smooth migration from Hg to Git is essential for successful >>>> >>>> migration to >>>> >>>>>>>> Apache. Thanks a lot for investigating how to do that. >>>>>>>> >>>>>>>> My plan (as described in another email) is to prepare the code >>>> >>>> donation >>>> >>>>>>>> in >>>>>>>> Hg >>>>>>>> and update it incrementally with code integrated into Hg. >>>>>>>> >>>>>>>> Are your conversions methods ready for incremental updates or do >>>>>>>> they >>>>>>>> only >>>>>>>> work as a one-time batch conversion? >>>>>>>> >>>>>>>> -jt >>>>>>>> >>>>>>>> On čtvrtek 24. listopadu 2016 10:41:50 CET Jan Lahoda wrote: >>>>>>>>> Interesting. I tried "git gc --aggressive" on the Mark's >> converted >>>>>>>>> repository, and the result is: >>>>>>>>> netbeans-import/.git$ du -hs . >>>>>>>>> 792M . >>>>>>>>> >>>>>>>>> The original was: >>>>>>>>> netbeans-import.git $ du -hs . >>>>>>>>> 3,5G . >>>>>>>>> >>>>>>>>> (IIRC Mark was converting http://hg.netbeans.org/main, not >>>> >>>> releases, so >>>> >>>>>>>> the >>>>>>>> >>>>>>>>> repository is a little bit smaller than the releases one.) >>>>>>>>> >>>>>>>>> I tried: >>>>>>>>> $ git log -p | sha1sum >>>>>>>>> >>>>>>>>> on both repositories, and the hashes appear to be the same. I >> also >>>>>>>>> tried >>>>>>>> >>>>>>>> to >>>>>>>> >>>>>>>>> clone the gc-ed repository using git clone --bare --no-local, >> and >>>> >>>> the >>>> >>>>>>>>> resulting repository is still about the same size. So, this >> seems >>>> >>>> good >>>> >>>>>>>>> to >>>>>>>>> me, unless there is some downside I don't know about. >>>>>>>>> >>>>>>>>> Jan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Nov 23, 2016 at 8:26 PM, Emilian Bold < >>>> >>>> [email protected]> >>>> >>>>>>>>> wrote: >>>>>>>>>> Actually I don't believe the data loss is that large. (There >> may >>>> >>>> also >>>> >>>>>>>>> be >>>>>>>>> >>>>>>>>> mercurial commits that are intentionally ignored by the >> conversion >>>>>>>>> >>>>>>>>> script, >>>>>>>>> >>>>>>>>> like commits that only add tags?) >>>>>>>>> >>>>>>>>>> hg log | grep '^changeset:' | wc -l >>>>>>>>>> >>>>>>>>>> 313209 >>>>>>>>>> >>>>>>>>>> git log | grep '^commit ' | wc -l >>>>>>>>>> >>>>>>>>>> 301478 >>>>>>>>>> >>>>>>>>>> So there is a difference of 11731 commits (about 4%) but those >>>>>>>>>> couldn't >>>>>>>>>> have such a large impact on repository size. >>>>>>>>>> >>>>>>>>>> I hope somebody else is willing to work with me on this so we >>>> >>>> document >>>> >>>>>>>>>> everything and do a reproducible repository conversion. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --emi >>>>>>>>>> >>>>>>>>>> On Wed, Nov 23, 2016 at 9:10 PM, Emilian Bold < >>>> >>>> [email protected]> >>>> >>>>>>>>>> wrote: >>>>>>>>>>> Well, I dunno what black magic `gc --aggressive` does but the >>>>>>>>>> >>>>>>>>>> repository >>>>>>>>> >>>>>>>>> is 0.85GB now! >>>>>>>>> >>>>>>>>>>> I also ran `git reflog expire` first but it didn't change the >>>> >>>> size at >>>> >>>>>>>>>> all. >>>>>>>>>> >>>>>>>>>> One thing to keep in mind is that I used --force although I >> had 6 >>>>>>>>>> >>>>>>>>>>> commits >>>>>>>>>>> with the warning "repository has at least one unnamed head". >>>>>>>>>>> Which >>>>>>>>>> >>>>>>>>>> were >>>>>>>>> >>>>>>>>> probably all close branch commits (hg commit --close-branch). >>>>>>>>> >>>>>>>>>>> So I might have have data loss(!) since I believe I read >>>>>>>>>> >>>>>>>>>> hg-fast-export.sh >>>>>>>>>> >>>>>>>>>> picks only one unnamed head as the migration winner. I wonder >> if >>>> >>>> the >>>> >>>>>>>>>> gc >>>>>>>>> >>>>>>>>> command didn't just purge a lot of valid commits from such an >>>> >>>> unnamed >>>> >>>>>>>>>> head >>>>>>>>>> >>>>>>>>>> and that's why the repository became so small. >>>>>>>>>> >>>>>>>>>>> Could somebody else try a test repository conversion and >> validate >>>> >>>> my >>>> >>>>>>>>>>> numbers? >>>>>>>>>>> >>>>>>>>>>> git gc --aggressive --prune=now >>>>>>>>>>> Counting objects: 4085031, done. >>>>>>>>>>> Delta compression using up to 8 threads. >>>>>>>>>>> Compressing objects: 100% (2909203/2909203), done. >>>>>>>>>>> Writing objects: 100% (4085031/4085031), done. >>>>>>>>>>> Total 4085031 (delta 2150468), reused 1585934 (delta 0) >>>>>>>>>>> Checking connectivity: 4085031, done. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> --emi >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 23, 2016 at 7:59 PM, Paul Merlin < >>>> >>>> [email protected]> >>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> Hi Emilian, >>>>>>>>>>>> >>>>>>>>>>>> I see hg-fast-export.sh finished at some point. >>>>>>>>>>>> >>>>>>>>>>>>> As expected though, git does not have any of the disk space >>>> >>>> gains. >>>> >>>>>>>>>>>>> The >>>>>>>>>>>>> converted git releases/ repository is 3.6GB. >>>>>>>>>>>> >>>>>>>>>>>> Just a thought. >>>>>>>>>>>> Did you try some git cleanups after the conversion? >>>>>>>>>>>> >>>>>>>>>>>> git reflog expire --expire=now --all >>>>>>>>>>>> git gc --aggressive --prune=now >>>>>>>>>>>> >>>>>>>>>>>> Cheers >>>>>>>>>>>> >>>>>>>>>>>> In case these statistics mean something: >>>>>>>>>>>>> git-fast-import statistics: >>>>>>>>>>>>> ------------------------------ >> ------------------------------ >>>>>>>>>>>> >>>>>>>>>>>> --------- >>>>>>>>> >>>>>>>>> Alloc'd objects: 4090000 >>>>>>>>> >>>>>>>>>>>>> Total objects: 4085509 ( 40220100 duplicates >>>>>>>>>>>>> >>>>>>>>>>>> ) >>>>>>>>>>>> >>>>>>>>> blobs : 1036365 ( 28386238 duplicates 858087 >>>>>>>>>>>> >>>>>>>>>>>> deltas >>>>>>>>> >>>>>>>>> of >>>>>>>>> >>>>>>>>>> 969684 attempts) >>>>>>>>>> >>>>>>>>>>>>> trees : 2735935 ( 11833862 duplicates >> 1370606 >>>>>>>>>>>> >>>>>>>>>>>> deltas >>>>>>>>> >>>>>>>>> of >>>>>>>>> >>>>>>>>>> 2613480 attempts) >>>>>>>>>> >>>>>>>>>>>>> commits: 313209 ( 0 duplicates >> 0 >>>>>>>>>>>> >>>>>>>>>>>> deltas >>>>>>>>> >>>>>>>>> of >>>>>>>>> >>>>>>>>>> 0 attempts) >>>>>>>>>> >>>>>>>>>>>>> tags : 0 ( 0 duplicates >> 0 >>>>>>>>>>>> >>>>>>>>>>>> deltas >>>>>>>>> >>>>>>>>> of >>>>>>>>> >>>>>>>>>> 0 attempts) >>>>>>>>>>>>> >>>>>>>>>>>>> Total branches: 1283 ( 346 loads ) >>>>>>>>>>>>> >>>>>>>>>>>>> marks: 1048576 ( 313209 unique ) >>>>>>>>>>>>> atoms: 124011 >>>>>>>>>>>>> >>>>>>>>>>>>> Memory total: 218429 KiB >>>>>>>>>>>>> >>>>>>>>>>>>> pools: 26711 KiB >>>>>>>>>>>>> >>>>>>>>>>>>> objects: 191718 KiB >>>>>>>>>>>>> >>>>>>>>>>>>> ------------------------------ >> ------------------------------ >>>>>>>>>>>> >>>>>>>>>>>> --------- >>>>>>>>> >>>>>>>>> pack_report: getpagesize() = 4096 >>>>>>>>> >>>>>>>>>>>>> pack_report: core.packedGitWindowSize = 1073741824 >>>>>>>>>>>>> pack_report: core.packedGitLimit = 8589934592 >>>>>>>>>>>>> pack_report: pack_used_ctr = 39000045 >>>>>>>>>>>>> pack_report: pack_mmap_calls = 733040 >>>>>>>>>>>>> pack_report: pack_open_windows = 4 / >> 7 >>>>>>>>>>>>> pack_report: pack_mapped = 4280730006 / >> 6950823920 >>>>>>>>>>>>> ------------------------------ >> ------------------------------ >>>>>>>>>>>> >>>>>>>>>>>> --------- >>>>>>>>>>>> >>>>>>>>>>>>> --emi >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Nov 18, 2016 at 1:32 PM, Emilian Bold < >>>>>>>>>>>> >>>>>>>>>>>> [email protected] >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>>>> A releases/ clone which on my system takes 3.8GB is >> reduced to >>>>>>>>>>>>> >>>>>>>>>>>>> 1.6GB >>>>>>>>> >>>>>>>>> with >>>>>>>>> >>>>>>>>>>>> the generaldelta and aggressivemergedeltas flags (took about >> 14 >>>>>>>>>>>> >>>>>>>>>>>>> hours). >>>>>>>>>> >>>>>>>>>> Pretty impressive! >>>>>>>>>> >>>>>>>>>>>>>> Converting to git with hg-fast-export.sh complains that >>>>>>>>>>>>> >>>>>>>>>>>>> "repository >>>>>>>>> >>>>>>>>> has at >>>>>>>>> >>>>>>>>>>>> least one unnamed head" for about 6 revisions. With --force >> I'm >>>>>>>>>>>> >>>>>>>>>>>>> able >>>>>>>>> >>>>>>>>> to >>>>>>>>> >>>>>>>>>> start the conversion but it hasn't finished yet. >>>>>>>>>> >>>>>>>>>>>>>> The git conversion is about 35% done and already using >> 1.3GB. >>>>>>>>>>>>>> >>>>>>>>>>>>>> So... I assume it's going to need just like the original >>>>>>>>>>>>> >>>>>>>>>>>>> repository >>>>>>>>> >>>>>>>>> about >>>>>>>>> >>>>>>>>>>>> 3.8GB. >>>>>>>>>>>> >>>>>>>>>>>>>> I wonder if git has similar space-saving tricks? >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> --emi >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Nov 17, 2016 at 8:46 AM, Emilian Bold < >>>>>>>>>>>>> >>>>>>>>>>>>> [email protected]> >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>>>>>> Forgot about this. I've just started the Mercurial >> repository >>>>>>>>>>>>>> >>>>>>>>>>>>>> conversion >>>>>>>>>>>> >>>>>>>>>>>> which will take a few hours. >>>>>>>>>>>> >>>>>>>>>>>>>>> Will report tomorrow or when it's done. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> --emi >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Nov 16, 2016 at 11:18 PM, cowwoc < >>>>>>>>>>>>>> >>>>>>>>>>>>>> [email protected]> >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>>> Hi Emilian, >>>>>>>>>>>>> >>>>>>>>>>>>>>>> Any update on this? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Gili >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2016-11-11 01:33 (-0500), Emilian Bold < >> [email protected]> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>> Thank you for following through with this after we talked on >>>>>>>>> >>>>>>>>>>>>>>>>> IRC.> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will check later the size reduction for the releases/ >>>> >>>> repo.> >> >> >>
