Le mer. 18 oct. 2023 à 16:12, Böszörményi Zoltán <[email protected]> a écrit : > > 2023. 10. 18. 15:28 keltezéssel, Julien Stephan írta: > > Le mer. 18 oct. 2023 à 15:25, Zoltan Boszormenyi <[email protected]> a > > écrit : > >> Thanks, I will try it. > >> > >> 2023. 10. 18. 15:16 keltezéssel, Martin Jansa írta: > >>> Just add the name to pytorch repo url and use only that in SRCREV_FORMAT, > >>> you probably > >>> don't care about representing the other repos in SRCPV if you're updating > >>> them only > >>> together with main pytorch update. > >>> > >>> On Wed, Oct 18, 2023 at 3:05 PM Zoltan Boszormenyi <[email protected]> > >>> wrote: > >>> > >>> Hi, > >>> > >>> I have a working python3-pytorch recipe which originally used this: > >>> > >>> ======================================================= > >>> addtask do_git_submodules after do_unpack before do_patch > >>> > >>> do_git_submodules[network] = "1" > >>> > >>> do_git_submodules () { > >>> cd ${S} > >>> git submodule update --init --recursive > >>> } > >>> ======================================================= > >>> > >>> I would like to replace the above with a series of gitsm:// SRC_URI > >>> lines. > >>> Here's a subset of submodules that shows the problem I am seeing: > >>> > >>> SRC_URI = " \ > >>> > >>> git://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1 > >>> > >>> <http://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1> > >>> \ > >>> ... > >>> > >>> gitsm://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm > >>> > >>> <http://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm> > >>> > >>> \ > >>> > >>> gitsm://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit > >>> > >>> <http://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit> > >>> > >>> \ > >>> > >>> gitsm://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo > >>> > >>> <http://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo> > >>> > >>> \ > >>> > >>> gitsm://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass > >>> > >>> <http://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass> > >>> > >>> \ > >>> > >>> gitsm://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest > >>> > >>> <http://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest> > >>> > >>> \ > >>> > >>> gitsm://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch > >>> > >>> <http://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch> > >>> > >>> \ > >>> ... > >>> " > > Hi Zoltan > > This is not how you are supposed to use gitsm.. This is much simpler: > > > > SRC_URI = " > > gitsm://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1 > > " > > > > will do all the magic for you :) > > When I first created that recipe (Yocto 4.0, PyTorch 1.13.0), it didn't. > > The problems were: > * PyTorch source releases are not on https://pypi.org/project/torch > This would have been the most convenient. > * Yocto complains about using the (unstable) release tarball URL from Github. > * Using the git:// SRC_URI did not populate the submodules' directories. > This was noticed because some submodules need patches to build under > Yocto. > Without checking out submodules manually, those patches fail. > > Just checked the last one on mickledore, still a problem. > > Have Yocto master changed this? >
I used to have a kirkstone recipe using gitsm properly to fetch the submodules. Can you reproduce your issue on master? And maybe share it to help debugging? Cheers Julien > > You can look at the vulkan-sample recipes in poky for an example of > > how to use it: > > https://git.openembedded.org/openembedded-core/tree/meta/recipes-graphics/vulkan/vulkan-samples_git.bb?h=master > > > > Cheers > > Julien > > > > > >>> As you can see, there are recursively placed git submodules > >>> and do_fetch fails with an error message about SRCREV_FORMAT > >>> having to be set. > >>> > >>> The above is a limited subset of the complete list and > >>> some of the repositories (like googletest, gloo, glog and pybind11) > >>> occurs multiple times as child submodules of upper ones, > >>> with different SRCREV values in different leaf submodules. > >>> > >>> I have not found a conclusive example to use SRCREV_FORMAT > >>> with git:// + gitsm:// > >>> > >>> Can someone enlighten me how to use SRCREV_FORMAT properly > >>> for this case? Should I stick to the proven working nonstandard way > >>> and just run git submodule update --init --recursive? > >>> > >>> As an easy way out, is there a flag to git:// to process its > >>> submodules automatically? > >>> > >>> Thanks in advance, > >>> Zoltán Böszörményi > >>> > >>> > >>> > >>> > >> > >> > >> >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189394): https://lists.openembedded.org/g/openembedded-core/message/189394 Mute This Topic: https://lists.openembedded.org/mt/102038382/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
