> On Oct 22, 2015, at 10:24 PM, Satish Balay <[email protected]> wrote: > > As I said - depending on the precondition for how tightly petsc & > petsc4py should be synced - we can choose the appropriate model. > > [for the metioned precondition - I was suggestion the model]. > > As you say - the initilal phrasing is wrong - so the model is not > suitable. > > A single repo does make it easy for 'all feature branches' to be in > sync.. > > In that model with-petsc4py=1 would be the default. However anyone > changing the petsc UI would also have to update petsc4py with the > change.. [and Lisandro says - that change could be buggy - and he > would like to review it -before its applied]
Just like Jed was going to review everything anybody put into PETSc before it got into master. That is just unrealistic and won't last long. Better to just have a good test suite. > > For 2 repos model - perhaps sync at 'master' might work. > > Satish > > On Thu, 22 Oct 2015, Barry Smith wrote: > >> >>> On Oct 22, 2015, at 9:53 PM, Satish Balay <[email protected]> wrote: >>> >>> If dmplex with petsc4py is the primary use case [and assuming the >>> usage is primarily --download-petsc4py] - we chould use the first >>> model I mentioned - and that should not be too difficult. >>> >>> The workflow would be something like: >>> >>> - Matt would do the updates in petsc/dmplex feature branch & petsc4py >>> dmplex branches simultaneosusly. >> >> That's nuts; Matt is a college professor he doesn't have time to keep two >> branches in sync for no good reason when they could be in the same >> repository. >> >> Plus other people will be making other changes in PETSc that break petsc4py >> >> This discussion has shown to me that keeping petsc and petsc4py in >> different repositories is just silly and unproductive. >>> >>> - And would always use --download-petsc4py for all his build tests >>> [we'll have to fix auto-update for --download-pkg for git version of >>> pkg] >>> >>> - Any isues that come up - Matt will fix in petsc4py/dmplex and update >>> petsc4py [before this branch is ever merged to next] >>> >>> So after next testing - a merge to master - Matt could issue a pull >>> requst to Lisandro [or leave it for Lisandro to figureout when >>> petsc4py-master gets updated with the fix]. >>> >>> The update would involve merging/updating petsc4py-master [and and >>> update to petsc4py.py in petsc-master] >>> >>> [even if this petsc4py-master update doesn't happen immediately] it >>> won't affect petsc4py/dmplex users as they would be using >>> --download-petsc4py [which will use the working petsc4py.commitid set >>> my Matt anyway] >>> >>> If the usage is not via --download-petsc4py - then there could be a >>> more appropriate workflow for that.. [and a different testsuite to >>> match that usage..] >>> >>> Satish >>> >>> On Thu, 22 Oct 2015, Barry Smith wrote: >>> >>>> >>>> The model we adopt also depends on how cutting edge petsc4py users want to >>>> be with petsc. For example if people want to use dmplex with petsc4py then >>>> the two packages have to move very closely in sync. >>>> >>>>> On Oct 22, 2015, at 9:11 PM, Satish Balay <[email protected]> wrote: >>>>> >>>>> On Thu, 22 Oct 2015, Barry Smith wrote: >>>>> >>>>>> >>>>>>> On Oct 22, 2015, at 7:28 PM, Satish Balay <[email protected]> wrote: >>>>>>> >>>>>>> The current infrastruture tests --download-petsc4py in petsc 'next' >>>>>>> and 'master' branches. >>>>>>> >>>>>>> Wrt next - usually a change in a feature branch (when it gets merged >>>>>>> to next) triggers this breakage. And usual thing to do is to fix it >>>>>>> in the feature branch (and merge to next again) >>>>>>> >>>>>>> Wrt to petsc4py - this would mean updating petsc4py.py [in petsc] with >>>>>>> the petsc4py repo gitcommit [optionally also tarball ] that >>>>>>> corresponds to the fix. [one way to handle this on petsc4py repo is to >>>>>>> create a feature branch for this fix] >>>>>> >>>>>> Yeah this is nuts, having to maintaining matching branches between petsc >>>>>> and petsc4py and merge them at the same time into next and then master. >>>>>> Clearly no one will ever do this and hence petsc4py will always be out >>>>>> of step from PETSc. I do really believe that incorporating petsc4py into >>>>>> PETSc would just make all these difficulties go away. For example >>>>>> maintaining Tao is a breeze now since it is updated in the same branches >>>>>> as PETSc. >>>>>> >>>>> >>>>> Perhaps I should rephrase the issue in the following way: >>>>> >>>>> We should decide on what level petsc4py and petsc should be synced. >>>>> [and the testsuite should be modified appropriately for that] >>>>> >>>>> Currently its done at next, master branchs. For this to work - the >>>>> above - previously mentioned workflow is required. [and for this >>>>> workflow - a single repo does makes thing easier] >>>>> >>>>> We could say sync should happen only between petsc4py-master & >>>>> petsc-master. [so issues - if any in feature branches/next wont't be >>>>> dealt with - and its ok for the master branches to be out of sync for >>>>> a brief time for the fix to propergate]. >>>>> >>>>> For this - we would have a different workflow [wrt syncing petsc4py >>>>> wrt petsc - and a slightly different test suite - testing only >>>>> master. [We could continue having the curent test suite - but ignore >>>>> errors in next]. And e-mails to Lisandro will trigger only on >>>>> petsc4py errors on master. Lisandro would then fix petsc4py (master) >>>>> - and then update petsc4py.py in petsc-master. >>>>> >>>>> Or perhaps some other mode is preferable.. >>>>> >>>>> Satish >>>>> >>>>>> Barry >>>>>> >>>>>>> >>>>>>> When this (petsc) feature branch is merged into master - the >>>>>>> corresponding feature branch on the petsc4py side can be merged into >>>>>>> petsc4py master [if petsc4py-master is to be in sync with >>>>>>> petsc-master]. >>>>>>> >>>>>>> [if there are multiple feature branches on the petsc side requiring >>>>>>> corresponding petsc4py changes - somehow that has to be handled] >>>>>>> >>>>>>> Perhaps testing petsc4py via petsc infrastructure [using >>>>>>> --download-petsc4py] is not the appropriate approach? >>>>>>> >>>>>>> Or we should do this only on master - not next? [or just have a >>>>>>> single --download-petsc4py test - thats not mixed with other tests? - >>>>>>> so its breakage is not critical for other things] >>>>>>> >>>>>>> Also we don't currently run any petsc4py tests. [we just test if it >>>>>>> compiles or not] >>>>>>> >>>>>>> Satish >>>>>>> >>>>>>> On Thu, 22 Oct 2015, Barry Smith wrote: >>>>>>> >>>>>>>> >>>>>>>> Lisandro, >>>>>>>> >>>>>>>> It is tested nightly but there is apparently currently no email sent >>>>>>>> on failure. See the bottom of, for example, >>>>>>>> http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/10/22/make_next_arch-linux-pkgs-opt_crank.log >>>>>>>> >>>>>>>> I think what should happen is the petsc4py log file should be >>>>>>>> published to the web (like the other log files) and a script look >>>>>>>> through it and decide to send email if it detects a problem. Like we >>>>>>>> currently have for the other log files. Someone has to dig into the >>>>>>>> nightly test infrastructure to figure out where the new logic needs to >>>>>>>> go. Currently Satish and Karl understand that infrastructure the best. >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Oct 22, 2015, at 6:44 PM, Lisandro Dalcin <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> On 22 October 2015 at 23:20, Barry Smith <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> Lisandro, >>>>>>>>>> >>>>>>>>>> You never responded to this. I assume it was because you did not >>>>>>>>>> like the idea? This is not an attempt to take control of petsc4py or >>>>>>>>>> to take credit away from petsc4py from you. Petsc4py is your package >>>>>>>>>> and you will always be the one who deserves the credit. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Oh, sorry! I missed this email. >>>>>>>>> >>>>>>>>> >>>>>>>>>> The problem is that mutual efficient development of very intertwined >>>>>>>>>> packages is difficult if they are in different respositories. For >>>>>>>>>> example petsc4py has been out of date with petsc master for many >>>>>>>>>> weeks now because it is too much effort to update the petsc4py >>>>>>>>>> repository each time we make some change in PETSc. Thus it becomes a >>>>>>>>>> burden on you to go in every once in a while and fix up petsc4py. >>>>>>>>>> With one repository changes would happen in both PETSc source and >>>>>>>>>> petsc4py source at the same time and tests would detect problems >>>>>>>>>> immediately. >>>>>>>>>> >>>>>>>>>> Thoughts? >>>>>>>>>> >>>>>>>>> >>>>>>>>> I general terms, I prefer to keep things separate as much a possible. >>>>>>>>> IMHO, a much important step would be to incorporate petsc4py to the >>>>>>>>> nightly tests, clearly separating the petsc and petsc4py tests runs. I >>>>>>>>> would love to get an email every time petsc4py nightly breaks, this >>>>>>>>> way I would be able to push fixes within a day or two, say. >>>>>>>>> Maintaining petsc4py requires knowledge of PETSc+Python+Cython, so >>>>>>>>> ultimately I'm the one in best position to do such work, if you guys >>>>>>>>> start adding commits to petsc4py withing the main petsc repo, I'm >>>>>>>>> likely going to miss these changes. For example, today Michael Lange >>>>>>>>> made some changes, I got an email from Bitbucket, then did a review >>>>>>>>> and spotted a few issues. >>>>>>>>> >>>>>>>>> My hesitation to incorporate petsc4py within petsc repo is, in >>>>>>>>> Python's [and linear algebra :-)] Zen : "Sparse is better than dense". >>>>>>>>> If we continue adding stuff to core PETSc, at same point we will endup >>>>>>>>> like Trilinos ( DON'T MISS the sencond entry in the FAQ: >>>>>>>>> https://trilinos.org/download/public-git-repository/). >>>>>>>>> >>>>>>>>> All that being said, I don't have any problem at all adding petsc4py >>>>>>>>> to the main PETSc repository. But I still think this is not a good >>>>>>>>> idea, it will not solve all the problems. Nightly tests with automatic >>>>>>>>> notifications to me about breakages is perhaps is all what we need to >>>>>>>>> keep petsc4py in close sync with petsc/master. What about trying this >>>>>>>>> first to see how it goes? >>>>>>>>> >>>>>>>>> >>>>>>>>> PS: Please understand my position has nothing to do with credits about >>>>>>>>> the project or ego-related crap. Actually, your proposal would mean >>>>>>>>> less responsibility and maintenance work for me. But I don't think it >>>>>>>>> is a move in the right direction. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Lisandro Dalcin >>>>>>>>> ============ >>>>>>>>> Research Scientist >>>>>>>>> Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) >>>>>>>>> Numerical Porous Media Center (NumPor) >>>>>>>>> King Abdullah University of Science and Technology (KAUST) >>>>>>>>> http://numpor.kaust.edu.sa/ >>>>>>>>> >>>>>>>>> 4700 King Abdullah University of Science and Technology >>>>>>>>> al-Khawarizmi Bldg (Bldg 1), Office # 4332 >>>>>>>>> Thuwal 23955-6900, Kingdom of Saudi Arabia >>>>>>>>> http://www.kaust.edu.sa >>>>>>>>> >>>>>>>>> Office Phone: +966 12 808-0459 >>>> >>>> >>> >> >> >
