Re: [OMPI devel] Bitbucket vs. GitHub (was: Conversion to GitHub: POSTPONED)

2014-09-25 Thread Paul Hargrove 
On Thu, Sep 25, 2014 at 2:28 PM, Jed Brown  wrote:

> Paul Hargrove  writes:
> > The GUIs for things like browsing commits, viewing diffs, etc are pretty
> > similar in capability and each is sufficiently intuitive (after a brief
> > learning curve) that I don't find I need any conscious effort to "mode
> > switch" between their use.  The ability to comment on commits in lieu of
> > the ticket system is a good feature, for instance, that "just works" in
> > both systems.
>
> One concern is that those commits appearing in the release repository
> won't have the comments from the development repository.  So if you use
> the comments and pull requests for debugging and discussion, you would
> be better off with one repository so that you don't have to keep
> switching repositories (in your browser) to access that metadata.
>

Jed as a good point there (about comments and pull-request discussion
threads not appearing in the repo itself and therefore not moving with them
across repos).  On that point I agree there is a reason to be concerned.
So, I only consider use of those comments as a good alternative to use of
an email thread (which is not connected to either/any repo).  And, of
course, at some point neither commit/pull-request comments nor email are a
proper alternative to use of an issue tracker.

I think it is worth noting that while the comment threads are not part of
the git repo, the commits (and thus their comments) do have persistent URLs
that can be useful in commit messages (especially CMR/pull-request merge
commits), the issue tracker and even in devel-list emails.

Those points might be worthy of mentioning in the git-usage docs for OMPI.

-Paul


-- 
Paul H. Hargrove  phhargr...@lbl.gov
Future Technologies Group
Computer and Data Sciences Department Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900

Re: [OMPI devel] Bitbucket vs. GitHub (was: Conversion to GitHub: POSTPONED)

2014-09-25 Thread Jed Brown 
Paul Hargrove  writes:
> The GUIs for things like browsing commits, viewing diffs, etc are pretty
> similar in capability and each is sufficiently intuitive (after a brief
> learning curve) that I don't find I need any conscious effort to "mode
> switch" between their use.  The ability to comment on commits in lieu of
> the ticket system is a good feature, for instance, that "just works" in
> both systems.  

One concern is that those commits appearing in the release repository
won't have the comments from the development repository.  So if you use
the comments and pull requests for debugging and discussion, you would
be better off with one repository so that you don't have to keep
switching repositories (in your browser) to access that metadata.


PETSc uses bitbucket mostly because it lets us unify our ACLs.  All our
source code is open, but we use private repositories for proposals and
(sometimes) papers.  We have *.edu addresses, so there is no charge.

I'm a happy user of both github and bitbucket and think either will work
for your purposes.


pgp7HMEm270Ph.pgp
Description: PGP signature

Re: [OMPI devel] Bitbucket vs. GitHub (was: Conversion to GitHub: POSTPONED)

2014-09-25 Thread Paul Hargrove 
First off: this is not an attempt at a "strong objection" to delay or
change Jeff's stated plan.

Just my view as somebody who already contributes to projects hosted on both
GH and BB:

Since the bulk of the DVCS "work" as a developer is (in my experience) done
at the git command line, the choice of hosting provider is not something
that matters "day to day".

For me the per-branch ACLs at BB is (as has been proposed here) used as a
simple mechanism to ensure a release branch can only be modified by the
RM/GK.  However, I suspect that a (public) fork (perhaps owned by the RM?)
would be a perfectly usable alternative at GH (sounds like that, or the
inverse in which the trunk is the fork, is Jeff's plan).

The GUIs for things like browsing commits, viewing diffs, etc are pretty
similar in capability and each is sufficiently intuitive (after a brief
learning curve) that I don't find I need any conscious effort to "mode
switch" between their use.  The ability to comment on commits in lieu of
the ticket system is a good feature, for instance, that "just works" in
both systems.  The same goes for pull-requests, though the two sites treat
them a bit differently (they are integrated with the issue tracker at GH
and a distinct object type at BB).
Caveat: I don't do things via GUI when I know of a command line equivalent
(for instance, I don't create, destroy or merge branches using the GUI),
and therefore I am probably not pushing the limits of either.

Site navigation is essentially the same, but with icons on the left vs
right.  Once you learn the navigation icons, I again find no conscious
effort to switch between them.  Each has minor quirks, and both sites make
changes occasionally (and without advance warning that I am aware of).

Personally, I don't like the issue trackers at either site as compared to
Trac or Bugzilla.  The available feature set in each is small enough that
in my experience if you can't immediately figure out how to do something,
it is probably because you can't do it at all.

In short, I find that switching between BB and GH is far easier than
switching among CVS, SVN and Git (which I also have to do because of the
variety of projects I work on or follow trunk/head/tip development of).  So
if, as Jeff suggests might happen, the OMPI community eventually finds work
split between BB and GH, then I don't personally think it is going to be a
productivity-killer.

-Paul

Disclaimer:  My employer pays for an institutional Unlimited account at BB,
which then owns all of our repos (project leads just get Admin status).
So, I host my own projects at BB without any objective evaluation of the
relative costs or merits of the hosting providers.

On Thu, Sep 25, 2014 at 11:42 AM, Jeff Squyres (jsquyres) <
jsquy...@cisco.com> wrote:

> I had a look at bitbucket.
>
> SHORT VERSION
>
> I'm inclined to stick with Github.  If no one strongly disagrees, I'll
> proceed with the GitHub migration next Wednesday, 1 Oct, 2014, starting at
> 8am US Eastern.
>
> MORE DETAIL
>
> All in all, there is massive overlap of features between Github and
> Bitbucket.  There's a few on each that aren't in the other, obviously
> (e.g., per-branch push ACLs at Bitbucket), but all in all, they're very
> similar.
>
> The cost model is a major difference.  At first look, OMPI had 42 unique
> committers over the last 12 months, which means we'd be on the 50-user plan
> at $50/mo ($600/yr).  This is twice as expensive at Github ($300/yr so we
> can have 10 private repos).  The cost difference is not an automatic
> deal-breaker, but it is something to note.  Plus, since we used 42
> committers last year, it's not inconceivable that we'll have to play tricks
> sometimes to stay under 50 committers (i.e., pruning accounts more than
> once a year).  Since I'm the guy that typically has to handle that kind of
> stuff, I'm not very excited about that prospect.
>
> I asked GitHub if they were going to support per-branch push ACLs.  They
> gave an unsurprising "Thanks for your comments!  We'll add it to the list
> of suggestions" kind of answer.  The exact text of their reply actually
> gave me a little hope that they're at least seriously thinking about it,
> but it is definitely not a promise of future functionality.
>
> As I mentioned in the prior email, one possibility is that we could take
> the main OMPI repo to BB and leave everything else as GH.
>
> In reality, ORCM would likely also follow (since it's closely related to
> OMPI -- it uses OPAL and ORTE).  And Dave/Ralph/I are discussing the
> possibility of using git subtrees to split OPAL and ORTE into their own
> repos (this is all talk at the moment -- don't worry).  If that happens,
> they'll likely be at BH with the main OMPI repo.
>
> As such, we'd end up with a bit of split-brain -- some repos at BB and
> some at GH.  Keep in mind that this is two different web UIs, two different
> ticket systems, two different wiki formats, etc.  For those of us who work
> in multiple differ

[OMPI devel] Bitbucket vs. GitHub (was: Conversion to GitHub: POSTPONED)

2014-09-25 Thread Jeff Squyres (jsquyres) 
I had a look at bitbucket.

SHORT VERSION

I'm inclined to stick with Github.  If no one strongly disagrees, I'll proceed 
with the GitHub migration next Wednesday, 1 Oct, 2014, starting at 8am US 
Eastern.

MORE DETAIL

All in all, there is massive overlap of features between Github and Bitbucket.  
There's a few on each that aren't in the other, obviously (e.g., per-branch 
push ACLs at Bitbucket), but all in all, they're very similar.

The cost model is a major difference.  At first look, OMPI had 42 unique 
committers over the last 12 months, which means we'd be on the 50-user plan at 
$50/mo ($600/yr).  This is twice as expensive at Github ($300/yr so we can have 
10 private repos).  The cost difference is not an automatic deal-breaker, but 
it is something to note.  Plus, since we used 42 committers last year, it's not 
inconceivable that we'll have to play tricks sometimes to stay under 50 
committers (i.e., pruning accounts more than once a year).  Since I'm the guy 
that typically has to handle that kind of stuff, I'm not very excited about 
that prospect.

I asked GitHub if they were going to support per-branch push ACLs.  They gave 
an unsurprising "Thanks for your comments!  We'll add it to the list of 
suggestions" kind of answer.  The exact text of their reply actually gave me a 
little hope that they're at least seriously thinking about it, but it is 
definitely not a promise of future functionality.

As I mentioned in the prior email, one possibility is that we could take the 
main OMPI repo to BB and leave everything else as GH.  

In reality, ORCM would likely also follow (since it's closely related to OMPI 
-- it uses OPAL and ORTE).  And Dave/Ralph/I are discussing the possibility of 
using git subtrees to split OPAL and ORTE into their own repos (this is all 
talk at the moment -- don't worry).  If that happens, they'll likely be at BH 
with the main OMPI repo.  

As such, we'd end up with a bit of split-brain -- some repos at BB and some at 
GH.  Keep in mind that this is two different web UIs, two different ticket 
systems, two different wiki formats, etc.  For those of us who work in multiple 
different projects in OMPI, it could be annoying to have to mentally switch 
between the two.

Don't get me wrong: using two different systems is definitely do-able, but... 
meh.

All in all, I think it distills down to:

1. There's one feature we hope GitHub will implement (per-branch push ACLs; we 
can easily switch from a two-repo system to a single-repo system if they ever 
do); Bitbucket has this feature today.  Otherwise, BB vs. GH = pretty 
feature-comparable.

2. Bitbucket is a bit more expensive / Cisco already paid for GitHub.  As a 
side-effect, using Bitbucket *may* result in committer-counting games (to stay 
on a given plan).

3. All the rest of OMPI projects are at GitHub

Because of inertia, monetary cost, an logistics/mental cost, I'm inclined to 
stick with the existing migration plan and move the main Open MPI repo to 
GitHub next Wednesday, 1 Oct 2014, starting at 8am US Eastern.

Comments?




On Sep 24, 2014, at 6:37 AM, Jeff Squyres (jsquyres)  wrote:

> If someone with a .edu account gets us a free Bitbucket for Open MPI, and 
> then we use it for both research and industry stuff... at best, I think that 
> falls into a grey area as to whether this is within Bitbucket's TOS 
> (disclaimer: I haven't read their TOS).  It still sounds like a murky 
> prospect; I'm not sure it's within the intent of a free account.
> 
> Paying a reasonable amount for a private account isn't out of the question.  
> Indeed, Cisco has already paid $300 for the first year of a Github account so 
> that OMPI can have a private repo.  :-\  That can be written off, if 
> necessary, but it would be nice not to.  However, paying per developer may 
> become prohibitive -- infrequent bulk payments (e.g., $300/year) are do-able 
> from those of us at corporations.  Managing a monthly fee that is dependent 
> upon the number of active committers (and that number changes over time) 
> could get a bit... complex, in terms of corporate payments / reimbursements.
> 
> That being said, there's quite a bit of OMPI infrastructure that is actively 
> in use at GitHub.  It would be a bit of a pain to migrate all of that *again* 
> (from SVN/Trac -> Git/Github -> Git/Bitbucket).  Remember, it's not just 
> moving the repos (which, since most repos are now Git, is easy to move to 
> another hosting provider); it's also moving the wiki and the tickets, too.  
> That will take more effort.
> 
> All the above being said:
> 
> 1. I'll still have a look at Bitbucket today.  It may be a workable model 
> that the main OMPI repo (and wiki and tickets) is at Bitbucket, and most 
> other repos (and wikis and tickets) are at Github.
> 2. I just sent a mail to Github support asking them if they plan to support 
> per-branch push ACLs.  I don't know if they'll be able to give a direct 
> answer, but it's worth asking.
> 
>

Re: [OMPI devel] Neighbor collectives with periodic Cartesian topologies of size one

2014-09-25 Thread Nathan Hjelm 
On Tue, Aug 26, 2014 at 07:03:24PM +0300, Lisandro Dalcin wrote:
> I finally managed to track down some issues in mpi4py's test suite
> using Open MPI 1.8+. The code below should be enough to reproduce the
> problem. Run it under valgrind to make sense of my following
> diagnostics.
> 
> In this code I'm creating a 2D, periodic Cartesian topology out of
> COMM_SELF. In this case, the process in COMM_SELF has 4 logical in/out
> links to itself. So we have size=1 but indegree=outdegree=4. However,
> in ompi/mca/coll/basic/coll_basic_module.c, "size * 2" request are
> being allocated to manage communication:
> 
> if (OMPI_COMM_IS_INTER(comm)) {
> size = ompi_comm_remote_size(comm);
> } else {
> size = ompi_comm_size(comm);
> }
> basic_module->mccb_num_reqs = size * 2;
> basic_module->mccb_reqs = (ompi_request_t**)
> malloc(sizeof(ompi_request_t *) * basic_module->mccb_num_reqs);
> 
> I guess you have to also special-case for topologies and allocate
> indegree+outdegree requests (not sure about this number, just
> guessing).
>

I wish this was possible but the topology information is not available
at that point. We may be able to change that but I don't see the work
completing anytime soon. I committed an alternative fix as r32796 and
CMR'd it to 1.8.3. I can confirm that the attached reproducer no longer
produces a SEGV. Let me know if you run into any more issues.


-Nathan


pgpiboDbxhbSj.pgp
Description: PGP signature

Re: [OMPI devel] [OMPI bugs] [Open MPI] #4919: Fix the application abort routine so we actually abort

2014-09-25 Thread Joshua Ladd 
@iivanov I am looking into a fix.

On Thu, Sep 25, 2014 at 11:42 AM, Open MPI  wrote:

> #4919: Fix the application abort routine so we actually abort
> ---+-
> Reporter:  rhc |   Owner:  ompi-rm1.8
> Type:  changeset move request  |  Status:  closed
> Priority:  blocker |   Milestone:  Open MPI 1.8.3
>  Version:  trunk   |  Resolution:  fixed
> Keywords:  |
> ---+-
>
> Comment (by iivanov):
>
>  I would like to avoid misunderstanding in this issue.
>  Here is my observation points:
>  1. trunk is fine for slurm and osrun launches w/ and w/o corresponding
>  commit in master branch;
>  2. this commit (In [32793]) fixes issue in v1.8 branch for osrun only;
>  3. srun launch is not fixed by this change;
>
>  Unfortunately I do not have a fix for srun launch at the moment;
>
> --
> Ticket URL: 
> Open MPI 
>
> ___
> bugs mailing list
> b...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/bugs
>