2023. 10. 18. 15:28 keltezéssel, Julien Stephan írta:
Le mer. 18 oct. 2023 à 15:25, Zoltan Boszormenyi <[email protected]> a écrit :
Thanks, I will try it.
2023. 10. 18. 15:16 keltezéssel, Martin Jansa írta:
Just add the name to pytorch repo url and use only that in SRCREV_FORMAT, you
probably
don't care about representing the other repos in SRCPV if you're updating them
only
together with main pytorch update.
On Wed, Oct 18, 2023 at 3:05 PM Zoltan Boszormenyi <[email protected]> wrote:
Hi,
I have a working python3-pytorch recipe which originally used this:
=======================================================
addtask do_git_submodules after do_unpack before do_patch
do_git_submodules[network] = "1"
do_git_submodules () {
cd ${S}
git submodule update --init --recursive
}
=======================================================
I would like to replace the above with a series of gitsm:// SRC_URI lines.
Here's a subset of submodules that shows the problem I am seeing:
SRC_URI = " \
git://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1
<http://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1> \
...
gitsm://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm
<http://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm>
\
gitsm://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit
<http://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit>
\
gitsm://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo
<http://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo>
\
gitsm://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass
<http://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass>
\
gitsm://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest
<http://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest>
\
gitsm://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch
<http://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch>
\
...
"
Hi Zoltan
This is not how you are supposed to use gitsm.. This is much simpler:
SRC_URI = "
gitsm://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1
"
will do all the magic for you :)
When I first created that recipe (Yocto 4.0, PyTorch 1.13.0), it didn't.
The problems were:
* PyTorch source releases are not on https://pypi.org/project/torch
This would have been the most convenient.
* Yocto complains about using the (unstable) release tarball URL from Github.
* Using the git:// SRC_URI did not populate the submodules' directories.
This was noticed because some submodules need patches to build under Yocto.
Without checking out submodules manually, those patches fail.
Just checked the last one on mickledore, still a problem.
Have Yocto master changed this?
You can look at the vulkan-sample recipes in poky for an example of
how to use it:
https://git.openembedded.org/openembedded-core/tree/meta/recipes-graphics/vulkan/vulkan-samples_git.bb?h=master
Cheers
Julien
As you can see, there are recursively placed git submodules
and do_fetch fails with an error message about SRCREV_FORMAT
having to be set.
The above is a limited subset of the complete list and
some of the repositories (like googletest, gloo, glog and pybind11)
occurs multiple times as child submodules of upper ones,
with different SRCREV values in different leaf submodules.
I have not found a conclusive example to use SRCREV_FORMAT
with git:// + gitsm://
Can someone enlighten me how to use SRCREV_FORMAT properly
for this case? Should I stick to the proven working nonstandard way
and just run git submodule update --init --recursive?
As an easy way out, is there a flag to git:// to process its
submodules automatically?
Thanks in advance,
Zoltán Böszörményi
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189393):
https://lists.openembedded.org/g/openembedded-core/message/189393
Mute This Topic: https://lists.openembedded.org/mt/102038382/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
