AFAIR the main concern was for developers to have to clone several dozen of repositories I don't think it is a real issue. We can share a script to do it easily
On Sat, Oct 7, 2017 at 10:26 PM, Chas Honton <[email protected]> wrote: > What are the concerns with separate git repository per artifact? Is it > the monolithic build? > > Chas > > > On Oct 7, 2017, at 1:20 PM, Arnaud Héritier <[email protected]> wrote: > > > > +1 > > > >> On Sat, Oct 7, 2017 at 7:11 PM, Hervé BOUTEMY <[email protected]> > wrote: > >> > >> we can start with naming the future git repos: I suppose adding a table > at > >> the > >> end of the Git migration Wiki page can give a good view of the result > >> (with 78 > >> new repos + existing 7 repos + the 6 in migration if nobody objects = 91 > >> repos) > >> > >> once we're all convinced of the target, we'll see how to work with infra > >> and I suppose they already have the right authors list, given the > current > >> mirrors already published > >> > >> Regards, > >> > >> Hervé > >> > >> Le samedi 7 octobre 2017, 16:05:45 CEST Tibor Digana a écrit : > >>> Herve, > >>> > >>> I think, first of all, we should write a list with exact names of git > >>> repositories for which INFRA will crate them in > >>> git1-us-west.apache.org/repos/asf > >>> and mirror in github.com/apache > >>> > >>> And then the authors.txt. > >>> > >>> In my case I did not migrate tags and branches which is my problem > given > >> in > >>> my example with previous e-mail. > >>> > >>> On Sat, Oct 7, 2017 at 3:48 PM, Tibor Digana <[email protected]> > >> wrote: > >>>> Herve, > >>>> I think making the tag as I wanted would not be possible in git. > >>>> > >>>> Let's make it with all 78 projects. > >>>> In 2016 I migrated my svn project "audit" located in sub-folder behind > >>>> trunk (trunk/libs/audit) to git which is exactly what we need to do in > >>>> ASF. > >>>> > >>>> In our case the commands would be 99% the same. The 1% is the source > >> SVN > >>>> URL and target GitHub URL. That's all. Then if we provide INFRA with > >> bunch > >>>> of commands like these then they will only execute them with their > >>>> permissions. > >>>> I do not think they will give me admin permissions to make it. > >>>> > >>>> git svn clone --no-metadata --authors-file=E:\*authors.txt* http:// > >>>> ***/svn/***/*trunk/libs/audit audit* > >>>> git config --global user.name "*** ***" > >>>> git config --global user.email "***@***.***" > >>>> git remote add origin https://***@github.com/apache/ > >> *maven-clean-plugin* > >>>> .git > >>>> git push -u origin --all > >>>> > >>>> I observed the *authors.txt* from command > >>>> *svn log -q* > >>>> Each person from between first two pipes, e.g.: > >>>> r1234567 | tibordigana | 2015-10-19 02:20:00 > >>>> > >>>> Every developer added e-mail and full name, i.e.: > >>>> tibordigana = Tibor Digana <[email protected]> > >>>> etc. > >>>> > >>>> This list of users which should be created in prior to contact INFRA. > >>>> Not all users are active after years but it would be nice to include > >> them > >>>> too. > >>>> This list of users is one thing which cannot be automated! > >>>> > >>>> > >>>> > >>>> > >>>> On Sat, Oct 7, 2017 at 2:48 PM, Tibor Digana <[email protected]> > >>>> > >>>> wrote: > >>>>> In my company I have such situation in SVN. Making a tag from > >> subfolder. > >>>>> Git always makes the tag from the root, right? > >>>>> Let's google something... > >>>>> > >>>>> On Sat, Oct 7, 2017 at 2:47 PM, Tibor Digana < > >> [email protected] > >>>>> > >>>>>> wrote: > >>>>>> In my company I have such situation. > >>>>>> Making a tag from subfolder. > >>>>>> Git always makes the tag from the root, right? > >>>>>> Let's google something... > >>>>>> > >>>>>> On Sat, Oct 7, 2017 at 2:05 PM, Hervé BOUTEMY < > [email protected] > >>> > >>>>>> > >>>>>> wrote: > >>>>>>> thanks for your interest and feedback > >>>>>>> > >>>>>>> Le samedi 7 octobre 2017, 12:00:32 CEST Tibor Digana a écrit : > >>>>>>>> 78 is too much. > >>>>>>> > >>>>>>> notice that there would also be a question on the git repos naming > >>>>>>> convention, > >>>>>>> to avoid flat 78 names but keep at least 3 meaningful groups > >> (plugins, > >>>>>>> shared, > >>>>>>> resources: I think this is not absolutely necessary for > doxia-tools) > >>>>>>> > >>>>>>>> There is no problem to trigger release over sub-folders and tag it > >>>>>>> > >>>>>>> with > >>>>>>> > >>>>>>>> prefix which is already done in SVN. > >>>>>>> > >>>>>>> is it supported by maven-release plugin with git? > >>>>>>> > >>>>>>> notice I did not know that git was able to tag only a subtree: but > I > >>>>>>> knew I > >>>>>>> don't yet understand every aspect of git magic... :) > >>>>>>> > >>>>>>>> The CI build can always trigger the root and Jenkinsfile would > >> have > >>>>>>>> 41 > >>>>>>>> stages for plugins, 26 stages for Shared, etc. > >>>>>>>> It can be done in Jenkinsfile and so the shell would not throw > >>>>>>> > >>>>>>> exception > >>>>>>> > >>>>>>>> but status would be set instead and goes to the next stage. > >>>>>>>> I do not know how to detect in GitSCM which sub-folder(s) received > >>>>>>> > >>>>>>> but I > >>>>>>> > >>>>>>>> guess this can be investigated. > >>>>>>> > >>>>>>> +1 > >>>>>>> but I don't know where to look for myself: on that point, I hope to > >>>>>>> have some > >>>>>>> help from Jenkinsfile experts > >>>>>>> and eventually start with svn, to have something before the git > >>>>>>> migration > >>>>>>> > >>>>>>>> Then it can be a kind of switch-case over committed folders in > >>>>>>> > >>>>>>> Jenkinsfile. > >>>>>>> > >>>>>>>> This means that every time another stage/sub-folder is shown in > >>>>>>> > >>>>>>> Pipeline > >>>>>>> > >>>>>>>> which will be messy in the UI. :( > >>>>>>> > >>>>>>> notice that we can perhaps split the repos in less parts than we > >> have > >>>>>>> components: on plugins, for example, we already have 4 categories > >> [1] > >>>>>>> which > >>>>>>> would avoid 1 repo with 41 plugins, but more something like > >> 6+10+10+15 > >>>>>>> I suppose we could do the same for shared (reporting folder comes > >> to my > >>>>>>> mind) > >>>>>>> > >>>>>>> The key feasibility issue is the capacity to release a > sub-component > >>>>>>> from git > >>>>>>> with maven-release-plugin, IMHO > >>>>>>> (taking apart the git purists idea of 1 lifecycle = 1 git repo) > >>>>>>> > >>>>>>> Regards, > >>>>>>> > >>>>>>> Hervé > >>>>>>> > >>>>>>> > >>>>>>> [1] http://maven.apache.org/plugins/ > >>>>>>> > >>>>>>>> On Sat, Oct 7, 2017 at 4:17 AM, Hervé BOUTEMY < > >> [email protected]> > >>>>>>> > >>>>>>> wrote: > >>>>>>>>> There are 6 svn locations without any special complexity that > >> are > >>>>>>> > >>>>>>> waiting > >>>>>>> > >>>>>>>>> for > >>>>>>>>> a volunteer for git migration for a few years but nobody does > >>>>>>> > >>>>>>> anything > >>>>>>> > >>>>>>>>> about: > >>>>>>>>> I started 3 days ago to ask for help about it and got pretty no > >>>>>>> > >>>>>>> feedback > >>>>>>> > >>>>>>>>> [1] > >>>>>>>>> > >>>>>>>>> then there are the 4 complex svn locations (plugins, shared, > >>>>>>> > >>>>>>> resources, > >>>>>>> > >>>>>>>>> doxia- > >>>>>>>>> tools) that require much more work: I suppose we can't tell git > >>>>>>> > >>>>>>> migration > >>>>>>> > >>>>>>>>> is > >>>>>>>>> abandoned, but it will require someone to work on it seriously > >>>>>>>>> Remember that the key question [2] is: do we transform these 4 > >> svn > >>>>>>>>> locations > >>>>>>>>> into 41 +26 + 6 + 5 = 78 independent git repos? > >>>>>>>>> Yes, I told that Jenkins configuration is one key aspect we > >> need a > >>>>>>>>> strategy > >>>>>>>>> about, in parallel with git strategy. > >>>>>>> > >>>>>>>>> then there will be the remaining cases where we need to decide: > >>>>>>> lower > >>>>>>> > >>>>>>>>> impact, > >>>>>>>>> lower priority. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Summary: nothing is abandoned, but: > >>>>>>>>> - simple cases: no volunteer to do the job > >>>>>>>>> - hard cases: is there a volunteer either to define a strategy > >> or > >>>>>>> > >>>>>>> do some > >>>>>>> > >>>>>>>>> work > >>>>>>>>> on it? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> > >>>>>>>>> Hervé > >>>>>>>>> > >>>>>>>>> [1] https://lists.apache.org/thread.html/ > >>>>>>>>> edf3642a7bdd515f0cad421c25589741819446463614bf0515e56dbe@ > >>>>>>>>> %3Cdev.maven.apache.org%3E > >>>>>>>>> > >>>>>>>>> [2] https://cwiki.apache.org/confluence/display/MAVEN/Git > >>>>>>>>> +Migration#GitMigration-Migratinganaggregatortreeintoacollec > >>>>>>> > >>>>>>> tionofgitrepos > >>>>>>> > >>>>>>>>> Le vendredi 6 octobre 2017, 20:35:45 CEST Arnaud Héritier a > >> écrit : > >>>>>>>>>> But did we completely abandoned the idea of moving everything > >> to > >>>>>>> > >>>>>>> git ? > >>>>>>> > >>>>>>>>>> The CI setup was the main stop for it ? > >>>>>>>>>> > >>>>>>>>>> On Fri, Oct 6, 2017 at 8:05 PM, Hervé BOUTEMY < > >>>>>>> > >>>>>>> [email protected]> > >>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>> I was expecting the usual litany > >>>>>>>>>>> > >>>>>>>>>>> what I'm not confident currently with pipeline on Maven > >> core is > >>>>>>> > >>>>>>> that > >>>>>>> > >>>>>>>>>>> we > >>>>>>>>>>> have > >>>>>>>>>>> for example the "maven-3.x-jenkinsfile/MNG-6242 - build #1 > >> - > >>>>>>> > >>>>>>> null" > >>>>>>> > >>>>>>>>>>> message, > >>>>>>>>>>> with this "null" part that makes me wonder if we are using > >> it > >>>>>>>>>>> as > >>>>>>>>> > >>>>>>>>> expected. > >>>>>>>>> > >>>>>>>>>>> And for large multi-module svn trunks (the ones we don't > >>>>>>> > >>>>>>> migrate to > >>>>>>> > >>>>>>>>> git: > >>>>>>>>>>> mainly plugins and shared), is there a solution to rebuild > >> just > >>>>>>>>>>> changed > >>>>>>>>>>> modules? > >>>>>>>>>>> > >>>>>>>>>>> ideally, the rebuild when a SNAPSHOT dependency is published > >>>>>>> > >>>>>>> would > >>>>>>> > >>>>>>>>>>> have > >>>>>>>>>>> been a > >>>>>>>>>>> plus, but this is sufficiently a rare use that I won't be > >>>>>>> > >>>>>>> extremist > >>>>>>> > >>>>>>>>>>> Then IIUC, this migration job is one additional TODO for > >> me... > >>>>>>>>>>> > >>>>>>>>>>> Regards, > >>>>>>>>>>> > >>>>>>>>>>> Hervé > >>>>>>>>>>> > >>>>>>>>>>> Le vendredi 6 octobre 2017, 17:46:53 CEST Arnaud Héritier a > >>>>>>> > >>>>>>> écrit : > >>>>>>>>>>>> I agree that we should use pipeline nowdays > >>>>>>>>>>>> perhaps a shared lib if we want to standardize some stuffs > >>>>>>> > >>>>>>> and a set > >>>>>>> > >>>>>>>>> of > >>>>>>>>> > >>>>>>>>>>>> multi-branches jobs (or org folder but it requires GitHub > >> :( > >>>>>>>>>>>> ) > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Oct 6, 2017 at 9:16 AM, Stephen Connolly < > >>>>>>>>>>>> > >>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>> On Fri 6 Oct 2017 at 06:32, Hervé BOUTEMY < > >>>>>>> > >>>>>>> [email protected]> > >>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>> I just fixed a few failing jobs [1] to have again a > >>>>>>>>>>>>>> usable > >>>>>>>>> > >>>>>>>>> Jenkins. > >>>>>>>>> > >>>>>>>>>>>>>> Now I'm facing some issues, I suppose caused by newer > >>>>>>> > >>>>>>> Jenkins > >>>>>>> > >>>>>>>>>>> versions: > >>>>>>>>>>>>>> - Maven 3.0.5 causes NoSuchMethodError: > >>>>>>>>>>>>>> o.c.plexus.util.xml.pull. > >>>>>>>>>>>>> > >>>>>>>>>>>>> MXParser > >>>>>>>>>>>>> > >>>>>>>>>>>>>> [2] > >>>>>>>>>>>>>> - I had to switch to JDK 8 for maven-plugin-tools job, > >>>>>>> > >>>>>>> since JDK > >>>>>>> > >>>>>>>>>>> causes > >>>>>>>>>>> > >>>>>>>>>>>>>> failures (looks like Jenkins uses a hack to inject > >> JDK 7 > >>>>>>> > >>>>>>> as a > >>>>>>> > >>>>>>>>> tool > >>>>>>>>> > >>>>>>>>>>> while > >>>>>>>>>>> > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>> build JVM is Java 8) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Should we drop Maven 3.0.5 builds and JDK 7? > >>>>>>>>>>>>>> Notice I didn't check which is the minimum Maven > >> version > >>>>>>>>> > >>>>>>>>> required... > >>>>>>>>> > >>>>>>>>>>>>>> Or perhaps simply don't use the Jenkins Maven plugin > >> with > >>>>>>> > >>>>>>> this > >>>>>>> > >>>>>>>>> Maven > >>>>>>>>> > >>>>>>>>>>>>> 3.0.5 > >>>>>>>>>>>>> > >>>>>>>>>>>>>> or > >>>>>>>>>>>>>> JDK 7 configuration: default build as Jenkins Maven > >>>>>>> > >>>>>>> plugin with > >>>>>>> > >>>>>>>>> JDK > >>>>>>>>> > >>>>>>>>>>> 8 + > >>>>>>>>>>> > >>>>>>>>>>>>>> latest > >>>>>>>>>>>>>> Maven, and other configurations as scripted jobs? > >>>>>>>>>>>>> > >>>>>>>>>>>>> http://javaadventure.blogspot. > >> ie/2013/11/jenkins-maven-job-> >>>> > > > > > > > > >>>>>>>>>>> > >>>>>>>>>>> type-considered-evil.html > >>>>>>>>>>> > >>>>>>>>>>>>> <http://javaadventure.blogspot.ie/2013/11/jenkins-> > > >>>>>>>>>>> > >>>>>>>>>>> maven-job-type-considered-evil.html?m=1> > >>>>>>>>>>> > >>>>>>>>>>>>> We should stop using the evil job type as it’s minimum > >> Java > >>>>>>>>> > >>>>>>>>> version is > >>>>>>>>> > >>>>>>>>>>>>> dictated by Jenkins’ Java minimum. > >>>>>>>>>>>>> > >>>>>>>>>>>>>> We need to define our common strategy and have a > >>>>>>> > >>>>>>> consistent > >>>>>>> > >>>>>>>>>>>>>> configuration > >>>>>>>>>>>>>> for > >>>>>>>>>>>>>> every job understood by everybody > >>>>>>>>>>>>> > >>>>>>>>>>>>> I recommend Jenkinsfile and the `withMaven` wrapper > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hervé > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [1] https://builds.apache.org/view/M-R/view/Maven/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [2] > >>>>>>>>>>>>>> https://builds.apache.org/ > >> view/M-R/view/Maven/job/maven-> > >>>>>>>>>>>>> > >>>>>>>>>>>>> plugin-tools-jdk-1.7/162/console > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ------------------------------ > >>>>>>> > >>>>>>> ------------------------------ > >>>>>>> > >>>>>>>>>>> --------- > >>>>>>>>>>> > >>>>>>>>>>>>>> To unsubscribe, e-mail: [email protected]. > >> org > >>>>>>> > >>>>>>>>>>>>>> For additional commands, e-mail: > >>>>>>> [email protected] > >>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>> > >>>>>>>>>>>>> Sent from my phone > >>>>>>>>>>> > >>>>>>>>>>> ------------------------------ > >> ------------------------------ > >>>>>>> > >>>>>>> --------- > >>>>>>> > >>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>> > >>>>>>>>> ------------------------------------------------------------ > >>>>>>> > >>>>>>> --------- > >>>>>>> > >>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>> > >>>>>>> ------------------------------------------------------------ > >> --------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: [email protected] > >>>>>> > >>>>>> -- > >>>>>> Cheers > >>>>>> Tibor > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > > > -- > > ----- > > Arnaud Héritier > > http://aheritier.net > > Mail/GTalk: aheritier AT gmail DOT com > > Twitter/Skype : aheritier > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- ----- Arnaud Héritier http://aheritier.net Mail/GTalk: aheritier AT gmail DOT com Twitter/Skype : aheritier
