Re: [OMPI devel] Git submodules are coming

2020-11-13 Thread Brice Goglin via devel 
FYI, this was a git bug that will be fixed soon (the range of commits
being rebased was wrong).

https://lore.kernel.org/git/pull.789.git.1605314085.gitgitgad...@gmail.com/T/#t

https://lore.kernel.org/git/20d6104d-ca02-4ce4-a1c0-2f9386ded...@gmail.com/T/#t

Brice



Le 07/02/2020 à 10:27, Brice Goglin a écrit :
>
> Hello
>
> I have a git submodule issue that I don't understand.
>
> PR#7367 was initially on top of PR #7366. When Jeff merged PR#7366, I
> rebased my #7367 with git prrs and got this error:
>
> $ git prrs origin master
> From https://github.com/open-mpi/ompi
>  * branch  master -> FETCH_HEAD
> Fetching submodule opal/mca/hwloc/hwloc2/hwloc
> fatal: cannot rebase with locally recorded submodule modifications
>
> I didn't touch the hwloc submodule as far as I can see. The hwloc
> submodule also didn't change in origin/master between before and after
> the rebasing.
>
> $ git submodule status
>  38433c0f5fae0b761bd20e7b928c77f3ff2e76dc opal/mca/hwloc/hwloc2/hwloc 
> (hwloc-2.1.0rc2-33-g38433c0f)
> opal/mca/hwloc/hwloc2/hwloc $ git status
> HEAD detached from f1a2e22a
> nothing to commit, working tree clean
>
> I am not sure what's this "HEAD detached ..." is doing here.
>
> I seem to be able to reproduce the issue in my master branch by doing
> "git reset --hard HEAD^". git prrs will then fail the same.
>
> I worked around the issue by manually reapplying all commits from my
> PR on top of master with git cherry-pick, but I'd like to understand
> what's going on. It looks like my submodule is clean but not clean
> enough for a rebase?
>
> Thanks
>
> Brice
>
>
>
> Le 07/01/2020 à 18:02, Jeff Squyres (jsquyres) via devel a écrit :
>> We now have two PRs pending that will introduce the use of Git submodules 
>> (and there are probably more such PRs on the way).  At last one of these 
>> first two PRs will likely be merged "Real Soon Now".
>>
>> We've been talking about using Git submodules forever.  Now we're just about 
>> ready.
>>
>> **
>> *** DEVELOPERS: THIS AFFECTS YOU!! ***
>> **
>>
>> You cannot just "clone and build" any more:
>>
>> -
>> git clone g...@github.com:open-mpi/ompi.git
>> cd ompi && ./autogen.pl && ./configure ...
>> -
>>
>> You will *have* to initialize the Git submodule(s) -- either during or after 
>> the clone.  *THEN* you can build Open MPI.
>>
>> Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules
>>
>> May the force be with us!
>>

Re: [OMPI devel] Git submodules are coming

2020-02-07 Thread Ralph Castain via devel 
FWIW: I have major problems when rebasing if that rebase runs across the point 
where a submodule is added. Every file that was removed and replaced by the 
submodule generates a conflict. Only solution I could find was to whack the 
subdirectory containing the files-to-be-replaced and work thru it (and it isn't 
an easy process). Rather painful, which is why rebasing the "remove ORTE" PR 
has been a nightmare.


> On Feb 7, 2020, at 7:31 AM, Jeff Squyres (jsquyres) via devel 
>  wrote:
> 
> On Feb 7, 2020, at 4:27 AM, Brice Goglin via devel  
> wrote:
>> 
>> PR#7367 was initially on top of PR #7366. When Jeff merged PR#7366, I 
>> rebased my #7367 with git prrs and got this error:
>> 
>> $ git prrs origin master
>> From 
>> https://github.com/open-mpi/ompi
>> 
>> * branch  master -> FETCH_HEAD
>> Fetching submodule opal/mca/hwloc/hwloc2/hwloc
>> fatal: cannot rebase with locally recorded submodule modifications
>> 
>> I didn't touch the hwloc submodule as far as I can see. The hwloc submodule 
>> also didn't change in origin/master between before and after the rebasing.
> 
> Huh.  I can't see from this what happened; I have no insight to offer here, 
> sorry...
> 
>> $ git submodule status
>> 38433c0f5fae0b761bd20e7b928c77f3ff2e76dc opal/mca/hwloc/hwloc2/hwloc 
>> (hwloc-2.1.0rc2-33-g38433c0f)
> 
> I see this in my ompi clone as well (i.e., it's where the master/HEAD hwloc 
> submodule is pointing).
> 
>> opal/mca/hwloc/hwloc2/hwloc $ git status
>> HEAD detached from f1a2e22a
>> nothing to commit, working tree clean
>> 
>> I am not sure what's this "HEAD detached ..." is doing here.
> 
> If you look at the graph log in the opal/mca/hwloc/hwloc2/hwloc tree, you'll 
> see:
> 
> * 03d42600 (origin/v2.1) doxy: add a ref to envvar from the XML section
> ...a bunch more commits...
> * 38433c0f (HEAD) .gitignore: add config/ltmain.sh.orig
> ...a bunch more commits...
> * f1a2e22a (tag: hwloc-2.1.0rc2, tag: hwloc-2.1.0) contrib/windows: update 
> README
> 
> Meaning:
> - 03d42600 is the head of the "v2.1" branch in the hwloc repo
> - 38433c0f is where the submodule is pointing (i.e., local HEAD)
> - f1a2e22a is the last tag before that
> 
> So I think the "HEAD detached" means that the HEAD is not pointing to a named 
> commit (i.e., there's no tags or branches pointing to 38433c0f).
> 
>> I seem to be able to reproduce the issue in my master branch by doing "git 
>> reset --hard HEAD^". git prrs will then fail the same.
>> 
>> I worked around the issue by manually reapplying all commits from my PR on 
>> top of master with git cherry-pick, but I'd like to understand what's going 
>> on. It looks like my submodule is clean but not clean enough for a rebase?
> 
> I haven't had problems with rebasing and submodules; I'm not sure what I'm 
> doing different than you.
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
>

Re: [OMPI devel] Git submodules are coming

2020-02-07 Thread Jeff Squyres (jsquyres) via devel 
On Feb 7, 2020, at 4:27 AM, Brice Goglin via devel  
wrote:
> 
> PR#7367 was initially on top of PR #7366. When Jeff merged PR#7366, I rebased 
> my #7367 with git prrs and got this error:
> 
> $ git prrs origin master
> From 
> https://github.com/open-mpi/ompi
> 
>  * branch  master -> FETCH_HEAD
> Fetching submodule opal/mca/hwloc/hwloc2/hwloc
> fatal: cannot rebase with locally recorded submodule modifications
> 
> I didn't touch the hwloc submodule as far as I can see. The hwloc submodule 
> also didn't change in origin/master between before and after the rebasing.

Huh.  I can't see from this what happened; I have no insight to offer here, 
sorry...

> $ git submodule status
>  38433c0f5fae0b761bd20e7b928c77f3ff2e76dc opal/mca/hwloc/hwloc2/hwloc 
> (hwloc-2.1.0rc2-33-g38433c0f)

I see this in my ompi clone as well (i.e., it's where the master/HEAD hwloc 
submodule is pointing).

> opal/mca/hwloc/hwloc2/hwloc $ git status
> HEAD detached from f1a2e22a
> nothing to commit, working tree clean
> 
> I am not sure what's this "HEAD detached ..." is doing here.

If you look at the graph log in the opal/mca/hwloc/hwloc2/hwloc tree, you'll 
see:

* 03d42600 (origin/v2.1) doxy: add a ref to envvar from the XML section
...a bunch more commits...
* 38433c0f (HEAD) .gitignore: add config/ltmain.sh.orig
...a bunch more commits...
* f1a2e22a (tag: hwloc-2.1.0rc2, tag: hwloc-2.1.0) contrib/windows: update 
README

Meaning:
- 03d42600 is the head of the "v2.1" branch in the hwloc repo
- 38433c0f is where the submodule is pointing (i.e., local HEAD)
- f1a2e22a is the last tag before that

So I think the "HEAD detached" means that the HEAD is not pointing to a named 
commit (i.e., there's no tags or branches pointing to 38433c0f).

> I seem to be able to reproduce the issue in my master branch by doing "git 
> reset --hard HEAD^". git prrs will then fail the same.
> 
> I worked around the issue by manually reapplying all commits from my PR on 
> top of master with git cherry-pick, but I'd like to understand what's going 
> on. It looks like my submodule is clean but not clean enough for a rebase?

I haven't had problems with rebasing and submodules; I'm not sure what I'm 
doing different than you.

-- 
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI devel] Git submodules are coming

2020-02-07 Thread Brice Goglin via devel 
Hello

I have a git submodule issue that I don't understand.

PR#7367 was initially on top of PR #7366. When Jeff merged PR#7366, I
rebased my #7367 with git prrs and got this error:

$ git prrs origin master
>From https://github.com/open-mpi/ompi
 * branch  master -> FETCH_HEAD
Fetching submodule opal/mca/hwloc/hwloc2/hwloc
fatal: cannot rebase with locally recorded submodule modifications

I didn't touch the hwloc submodule as far as I can see. The hwloc
submodule also didn't change in origin/master between before and after
the rebasing.

$ git submodule status
 38433c0f5fae0b761bd20e7b928c77f3ff2e76dc opal/mca/hwloc/hwloc2/hwloc 
(hwloc-2.1.0rc2-33-g38433c0f)

opal/mca/hwloc/hwloc2/hwloc $ git status
HEAD detached from f1a2e22a
nothing to commit, working tree clean

I am not sure what's this "HEAD detached ..." is doing here.

I seem to be able to reproduce the issue in my master branch by doing
"git reset --hard HEAD^". git prrs will then fail the same.

I worked around the issue by manually reapplying all commits from my PR
on top of master with git cherry-pick, but I'd like to understand what's
going on. It looks like my submodule is clean but not clean enough for a
rebase?

Thanks

Brice



Le 07/01/2020 à 18:02, Jeff Squyres (jsquyres) via devel a écrit :
> We now have two PRs pending that will introduce the use of Git submodules 
> (and there are probably more such PRs on the way).  At last one of these 
> first two PRs will likely be merged "Real Soon Now".
>
> We've been talking about using Git submodules forever.  Now we're just about 
> ready.
>
> **
> *** DEVELOPERS: THIS AFFECTS YOU!! ***
> **
>
> You cannot just "clone and build" any more:
>
> -
> git clone g...@github.com:open-mpi/ompi.git
> cd ompi && ./autogen.pl && ./configure ...
> -
>
> You will *have* to initialize the Git submodule(s) -- either during or after 
> the clone.  *THEN* you can build Open MPI.
>
> Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules
>
> May the force be with us!
>

Re: [OMPI devel] Git submodules are coming

2020-01-08 Thread Jeff Squyres (jsquyres) via devel 
The hwloc git submodule just got merged 
(https://github.com/open-mpi/ompi/pull/6821).

A new age is upon us!

Be sure you read the wiki before you complain here about your builds being 
broken.  :-)

https://github.com/open-mpi/ompi/wiki/GitSubmodules




On Jan 8, 2020, at 9:04 AM, Ralph Castain via devel 
mailto:devel@lists.open-mpi.org>> wrote:

Actually, I take that back - making a separate PR to change the opal/pmix 
embedded component to a submodule was way too painful. I simply added it to the 
existing #7202.


On Jan 7, 2020, at 1:33 PM, Ralph Castain via devel 
mailto:devel@lists.open-mpi.org>> wrote:

Just an FYI: there will soon be THREE PRs introducing submodules - I am 
breaking #7202 into two pieces. The first will replace opal/pmix with direct 
use of PMIx everywhere and replace the embedded pmix component with a submodule 
pointing to PMIx master, and the second will replace ORTE with PRRTE.


On Jan 7, 2020, at 9:02 AM, Jeff Squyres (jsquyres) via devel 
mailto:devel@lists.open-mpi.org>> wrote:

We now have two PRs pending that will introduce the use of Git submodules (and 
there are probably more such PRs on the way).  At last one of these first two 
PRs will likely be merged "Real Soon Now".

We've been talking about using Git submodules forever.  Now we're just about 
ready.

**
*** DEVELOPERS: THIS AFFECTS YOU!! ***
**

You cannot just "clone and build" any more:

-
git clone g...@github.com:open-mpi/ompi.git
cd ompi && ./autogen.pl && ./configure ...
-

You will *have* to initialize the Git submodule(s) -- either during or after 
the clone.  *THEN* you can build Open MPI.

Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules

May the force be with us!

--
Jeff Squyres
jsquy...@cisco.com







--
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI devel] Git submodules are coming

2020-01-08 Thread Ralph Castain via devel 
Actually, I take that back - making a separate PR to change the opal/pmix 
embedded component to a submodule was way too painful. I simply added it to the 
existing #7202.


> On Jan 7, 2020, at 1:33 PM, Ralph Castain via devel 
>  wrote:
> 
> Just an FYI: there will soon be THREE PRs introducing submodules - I am 
> breaking #7202 into two pieces. The first will replace opal/pmix with direct 
> use of PMIx everywhere and replace the embedded pmix component with a 
> submodule pointing to PMIx master, and the second will replace ORTE with 
> PRRTE.
> 
> 
>> On Jan 7, 2020, at 9:02 AM, Jeff Squyres (jsquyres) via devel 
>>  wrote:
>> 
>> We now have two PRs pending that will introduce the use of Git submodules 
>> (and there are probably more such PRs on the way).  At last one of these 
>> first two PRs will likely be merged "Real Soon Now".
>> 
>> We've been talking about using Git submodules forever.  Now we're just about 
>> ready.
>> 
>> **
>> *** DEVELOPERS: THIS AFFECTS YOU!! ***
>> **
>> 
>> You cannot just "clone and build" any more:
>> 
>> -
>> git clone g...@github.com:open-mpi/ompi.git
>> cd ompi && ./autogen.pl && ./configure ...
>> -
>> 
>> You will *have* to initialize the Git submodule(s) -- either during or after 
>> the clone.  *THEN* you can build Open MPI.
>> 
>> Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules
>> 
>> May the force be with us!
>> 
>> -- 
>> Jeff Squyres
>> jsquy...@cisco.com
>> 
> 
>

Re: [OMPI devel] Git submodules are coming

2020-01-07 Thread Jeff Squyres (jsquyres) via devel 
Good catches -- I updated the hwloc (to "hwloc2") bit, but I'll leave the 
"bar50" edits to Brian/Ralph, since they wrote that section.

Meaning: I *think* you're right, but I'm going to let them do it...  :-0


On Jan 7, 2020, at 4:47 PM, Brice Goglin via devel 
mailto:devel@lists.open-mpi.org>> wrote:


Thanks a lot for writing all this.


At the end 
https://github.com/open-mpi/ompi/wiki/GitSubmodules#adding-a-new-submodule-pointing-to-a-specific-commit
should "bar" be "bar50x" in line "$ git add bar" ?

It seems to me that you are in opal/mca/foo and the new submodule is in 
"bar50x" (according to "cd opal/mca/foo/bar50x" at the beginning).

There's also a "bar-50x" instead of "bar50x" in line "git submodule add --name 
bar-50x ...". Should the submodule name match the directory name?


By the way, in 
https://github.com/open-mpi/ompi/wiki/GitSubmodules#updating-the-commit-that-a-submodule-refers-to
you may want to rename hwloc201 into hwloc2 to avoid confusion and match the 
current PR.

Brice (who cannot edit the wiki :))



Le 07/01/2020 à 18:02, Jeff Squyres (jsquyres) via devel a écrit :

We now have two PRs pending that will introduce the use of Git submodules (and 
there are probably more such PRs on the way).  At last one of these first two 
PRs will likely be merged "Real Soon Now".

We've been talking about using Git submodules forever.  Now we're just about 
ready.

**
*** DEVELOPERS: THIS AFFECTS YOU!! ***
**

You cannot just "clone and build" any more:

-
git clone 
g...@github.com:open-mpi/ompi.git
cd ompi && ./autogen.pl && ./configure ...
-

You will *have* to initialize the Git submodule(s) -- either during or after 
the clone.  *THEN* you can build Open MPI.

Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules

May the force be with us!




--
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI devel] Git submodules are coming

2020-01-07 Thread Brice Goglin via devel 
Thanks a lot for writing all this.


At the end
https://github.com/open-mpi/ompi/wiki/GitSubmodules#adding-a-new-submodule-pointing-to-a-specific-commit
should "bar" be "bar50x" in line "$ git add bar" ?

It seems to me that you are in opal/mca/foo and the new submodule is in
"bar50x" (according to "cd opal/mca/foo/bar50x" at the beginning).

There's also a "bar-50x" instead of "bar50x" in line "git submodule add
--name bar-50x ...". Should the submodule name match the directory name?


By the way, in
https://github.com/open-mpi/ompi/wiki/GitSubmodules#updating-the-commit-that-a-submodule-refers-to
you may want to rename hwloc201 into hwloc2 to avoid confusion and match
the current PR.

Brice (who cannot edit the wiki :))



Le 07/01/2020 à 18:02, Jeff Squyres (jsquyres) via devel a écrit :
> We now have two PRs pending that will introduce the use of Git submodules 
> (and there are probably more such PRs on the way).  At last one of these 
> first two PRs will likely be merged "Real Soon Now".
>
> We've been talking about using Git submodules forever.  Now we're just about 
> ready.
>
> **
> *** DEVELOPERS: THIS AFFECTS YOU!! ***
> **
>
> You cannot just "clone and build" any more:
>
> -
> git clone g...@github.com:open-mpi/ompi.git
> cd ompi && ./autogen.pl && ./configure ...
> -
>
> You will *have* to initialize the Git submodule(s) -- either during or after 
> the clone.  *THEN* you can build Open MPI.
>
> Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules
>
> May the force be with us!
>

Re: [OMPI devel] Git submodules are coming

2020-01-07 Thread Ralph Castain via devel 
Just an FYI: there will soon be THREE PRs introducing submodules - I am 
breaking #7202 into two pieces. The first will replace opal/pmix with direct 
use of PMIx everywhere and replace the embedded pmix component with a submodule 
pointing to PMIx master, and the second will replace ORTE with PRRTE.


> On Jan 7, 2020, at 9:02 AM, Jeff Squyres (jsquyres) via devel 
>  wrote:
> 
> We now have two PRs pending that will introduce the use of Git submodules 
> (and there are probably more such PRs on the way).  At last one of these 
> first two PRs will likely be merged "Real Soon Now".
> 
> We've been talking about using Git submodules forever.  Now we're just about 
> ready.
> 
> **
> *** DEVELOPERS: THIS AFFECTS YOU!! ***
> **
> 
> You cannot just "clone and build" any more:
> 
> -
> git clone g...@github.com:open-mpi/ompi.git
> cd ompi && ./autogen.pl && ./configure ...
> -
> 
> You will *have* to initialize the Git submodule(s) -- either during or after 
> the clone.  *THEN* you can build Open MPI.
> 
> Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules
> 
> May the force be with us!
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
>

[OMPI devel] Git submodules are coming

2020-01-07 Thread Jeff Squyres (jsquyres) via devel 
We now have two PRs pending that will introduce the use of Git submodules (and 
there are probably more such PRs on the way).  At last one of these first two 
PRs will likely be merged "Real Soon Now".

We've been talking about using Git submodules forever.  Now we're just about 
ready.

**
*** DEVELOPERS: THIS AFFECTS YOU!! ***
**

You cannot just "clone and build" any more:

-
git clone g...@github.com:open-mpi/ompi.git
cd ompi && ./autogen.pl && ./configure ...
-

You will *have* to initialize the Git submodule(s) -- either during or after 
the clone.  *THEN* you can build Open MPI.

Go read this wiki: https://github.com/open-mpi/ompi/wiki/GitSubmodules

May the force be with us!

-- 
Jeff Squyres
jsquy...@cisco.com