Re: [PATCH 0/6] git clone: Marry --recursive and --reference
Stefan Beller writes: > The plan for other layouts might be > > git submodule update --reference-dir /var/cache/ That is not a plan for "other layouts", but a plan for "the other layout that was mentioned as a possibility". As I said, both layouts are equally plausible, and I do not know if we need to plan for layouts other than these two, but we should consider if we want to add one new option every time we find a new layout we need to support, or we want a general framework that is more flexible and allows us to make the "borrow from the $GIT_DIR/modules/ of the repository the superproject borrows from" a mere special case of it. Thanks. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] git clone: Marry --recursive and --reference
On Fri, Aug 5, 2016 at 12:47 PM, Junio C Hamano wrote: > * You want the "clone" command above with "--recursive" to do "the >right thing". That is, the clone of the superproject borrows >from /var/cache/super.git local mirror, and the clone of xyzzy >that would be made at .git/modules/xyzzy in the superproject >would borrow from /var/cache/xyzzy.git local mirror. This is not what I intend to solve here. The solution in 6/6 solves the scenario as you outlined in [1]. > > What I am not sure about is how /var/cache/xyzzy.git should be > automatically derived from the information given from the command > line of "clone" and what the clone of the superproject contains. Generally speaking you cannot do that without assumptions. The scenario in [1] can be done without assumptions of the locations of the submodules. The only requirement for [1] is to have submodules checked out, which is a rather strong requirement, as that doesn't help you when you want to reference multiple superrpojects with incomplete submodule checkout. (Given all of them together may or may not produce the full set of references) > > IOW, both layouts are equally sensible; what layout (either one > of the above two, or something entirely different) is your "at > the same paths" assumption meant to serve well, and what is the > plan to serve other layouts? > The plan for other layouts might be git submodule update --reference-dir /var/cache/ ? -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] git clone: Marry --recursive and --reference
Stefan Beller writes:
> Currently when cloning a superproject with --recursive and --reference
> only the superproject learns about its alternates. The submodules are
> cloned independently, which may incur lots of network costs.
>
> Assume that the reference repository has the submodules at the same
> paths as the to-be-cloned submodule and try to setup alternates from
> there.
I am a bit fuzzy about what scenario you are trying to help. Is it
something like this? [*1*]
* The superproject is at git://frotz.xz/super.git but you locally
have a mirror at /var/cache/super.git (bare) repository.
* "git clone --refererence /var/cache/super.git git://frotz.xz/super.git"
would borrow that existing local mirror to cut down the traffic.
* The super.git project uses git://nitfol.xz/xyzzy.git as its
submodule. You locally have a mirror of it at
/var/cache/xyzzy.git (bare) repository.
* The xyzzy submodule is bound at libs/xyzzy in the superproject
tree.
* You want the "clone" command above with "--recursive" to do "the
right thing". That is, the clone of the superproject borrows
from /var/cache/super.git local mirror, and the clone of xyzzy
that would be made at .git/modules/xyzzy in the superproject
would borrow from /var/cache/xyzzy.git local mirror.
What I am not sure about is how /var/cache/xyzzy.git should be
automatically derived from the information given from the command
line of "clone" and what the clone of the superproject contains.
You'd need to make certain assumptions, like
- The parent directory of the --reference for superproject,
i.e. /var/cache/, is the directry that has many other mirrors;
- The .gitmodules file in superproject says a "xyzzy" project from
git://nitfol.xz/xyzzy.git must be checked out at "libs/xyzzy",
and it is a reasonable assumption that we would find a local
mirror at /var/cache/xyzzy.git (the /var/cache prefix comes from
the above assumptino).
There may be some other assumptions you are making. I am wondering
if your specific assumption is applicable generally.
[Footnote]
*1* Another common arrangement could be that you have one full clone
of the superproject at /var/cache/super.git and its submodule
xyzzy in /var/cache/super.git/modules/xyzzy and assuming that
would allow you to guess where to borrow submodules from. But
the result of assuming this layout would already be different
from the scenario I gave the above. If the organization uses
the xyzzy project only in the context of the frotz superproject,
"borrow from submodule repository inside $GIT_DIR/modules/ of
the superproject" may be sensible, but that makes use of "xyzzy"
as an independent project quite awkward (also you may have
another superproject that is unrelated to super.git that uses
the same "xyzzy" project as its submodule). The more "xyzzy"
project is suitable to be used as "submodule", the more it makes
sense to have its mirror independent from the superprojects that
use it, by using /var/cache/{super,xyzzy}.git layout.
IOW, both layouts are equally sensible; what layout (either one
of the above two, or something entirely different) is your "at
the same paths" assumption meant to serve well, and what is the
plan to serve other layouts?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/6] git clone: Marry --recursive and --reference
Currently when cloning a superproject with --recursive and --reference only the superproject learns about its alternates. The submodules are cloned independently, which may incur lots of network costs. Assume that the reference repository has the submodules at the same paths as the to-be-cloned submodule and try to setup alternates from there. Some submodules in the referenced superproject may not be there, (they are just not initialized/cloned/checked out), which yields an error for now. In future work we may want to soften the alternate check and not die in the clone when one of the given alternates doesn't exist. patch 1,2 are modernizing style of t7408, patches 3,4 are not strictly necessary, but I think it is a good thing to not leave the submodule related C code in a crippled state (i.e. allowing only one reference). The shell code would also need this update, but it looked ugly to me, so I postpone it until more of the submodule code is written in C. Thanks, Stefan Stefan Beller (6): t7408: modernize style t7408: merge short tests, factor out testing method submodule--helper module-clone: allow multiple references submodule--helper update-clone: allow multiple references submodule update: add super-reference flag clone: reference flag is used for submodules as well builtin/clone.c| 22 -- builtin/submodule--helper.c| 45 git-submodule.sh | 12 +++- t/t7408-submodule-reference.sh | 153 +++-- 4 files changed, 147 insertions(+), 85 deletions(-) -- 2.9.2.572.g9d9644e.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
