Re: [yocto] PREMIRROR
You are very welcome. Enjoy working with YP. :rjs On 7/25/19 4:48 PM, Russell Peterson wrote: > Just tried the externalsrc feature. Works perfectly. Exactly what I > was looking for. Thanks so much, Rudolf! > > --Russ > > > On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif > mailto:rudolf.str...@ibeeto.com>> wrote: > > Inlining below. > > On 7/25/19 8:14 AM, Russell Peterson wrote: >> I think I have a somewhat better understanding of what is going on. >> >> First off, I was confused by the fact that the original error >> message I saw from do_unpack referenced the file (URL) at >> DL_DIR/git2/original_github_url. What I didn't understand at the >> time was that while that file existed,/it was actually a link/ to >> DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong >> bare repository as I thought. It was unhappy because it didn't >> see the git commit in the local repo that it was looking for. I >> use AUTOREV for SRCREV. Apparently, even with PREMIRROR set and >> matched, bitbake will still fetch the git HEAD commit hash from >> the original URL in the recipe. The local git repo doesn't >> contain that commit. When I sync the local git repo to the >> github repo things work as expected. > Correct. If the source repo or the correct commit cannot be found > from the PREMIRROR, bitbake will use SRC_URI from the recipe. You > can block that behavior by setting BB_NO_NETWORK = "1" in which > case bitbake will not be able to connect to any remote server. > However, that does not help if your PREMIRROR is a network server. > In this case use BB_ALLOWED_HOSTS to allow access to only the > servers you want. >> >> Here is my problem. What I am ultimately trying to do here is >> have 2 git repos. 1 public, 1 private. For internal development >> we use the private git repo (a local gerrit server). Just before >> we ship we update the public github repo. The recipe always >> points to the github (public) repo so we don't need to mess with >> that before a release. This way we can do nightly builds (I'm >> just talking nightly builds done by Jenkins... not developers >> using eSDK and devtool) and test. Customers using yocto will be >> given a recipe that points to github and, importantly, those >> customers that do NOT use yocto simply fetch the project from >> github and manually build it in their own environment with their >> own tools. >> > That is a common problem. I do this all the time. To do this add > to your conf/local.conf file: > > INHERIT += "externalsrc" > EXTERNALSRC_pn-myrecipe = "/path/to/source/tree" > > Alternatively, you could use a bbappend file for you recipe in a > development layer, myrecipe.bbappend: > > inherit externalsrc > EXTERNALSRC = "/path/to/source/tree" > > Now, that is exactly what devtool does for you but you can of > course do it manually. > > :rjs > > > >> I can't be the only one doing this. Is there a best practices >> here (private vs. public repo)? >> >> --Russ >> >> >> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif >> mailto:rudolf.str...@ibeeto.com>> wrote: >> >> Russell, >> >> That is exactly what devtool and the externalsrc class do. >> PREMIRROR is the wrong approach for that. >> >> :rjs >> >> On 7/24/19 12:53 PM, Russell Peterson wrote: >>> Hi, Rudolf. >>> >>> I apologize for not being clear. The idea here is that my >>> recipe points to github while, for my local development >>> environment, I set a premirror to match a specific github >>> repository and translate it to a local directory. That >>> works. The fetch matches the PREMIRROR and places a copy of >>> the LOCAL git repo in my DL_DIR. The problem is, the >>> do_unpack task is looking for the git repo in the DL_DIR >>> (git2/etc...)... but it's looking for the git repo from >>> github, not my local repo that the fetch task just put in >>> DL_DIR. >>> >>> There are numerous reasons I'm doing this including the fact >>> that I cannot put a reference to my local repo in the bb >>> file since I ship that recipe to my customers and a local >>> pathname is meaningless to them. I prefer to simply modify >>> my local.conf with a PREMIRROR value that has a specific >>> regular expression that matches this particular github repo >>> (and hence does NOT effect all recipes). This is why I >>> wanted to use the PREMIRROR function. Unfortunately, it >>> does not appear to work or at least not work as I expected. >>> >>> --Russ >>> >>> >>> >>> >>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif >>> mailto:rudolf.str...@ibeeto.com>> >>> wrote: >>> >>> Hi
Re: [yocto] PREMIRROR
Just tried the externalsrc feature. Works perfectly. Exactly what I was
looking for. Thanks so much, Rudolf!
--Russ
On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif
wrote:
> Inlining below.
> On 7/25/19 8:14 AM, Russell Peterson wrote:
>
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message I
> saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url. What I didn't understand at the time was
> that while that file existed,* it was actually a link* to
> DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong bare
> repository as I thought. It was unhappy because it didn't see the git
> commit in the local repo that it was looking for. I use AUTOREV for
> SRCREV. Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the recipe.
> The local git repo doesn't contain that commit. When I sync the local git
> repo to the github repo things work as expected.
>
> Correct. If the source repo or the correct commit cannot be found from the
> PREMIRROR, bitbake will use SRC_URI from the recipe. You can block that
> behavior by setting BB_NO_NETWORK = "1" in which case bitbake will not be
> able to connect to any remote server. However, that does not help if your
> PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS to allow
> access to only the servers you want.
>
>
> Here is my problem. What I am ultimately trying to do here is have 2 git
> repos. 1 public, 1 private. For internal development we use the private
> git repo (a local gerrit server). Just before we ship we update the public
> github repo. The recipe always points to the github (public) repo so we
> don't need to mess with that before a release. This way we can do nightly
> builds (I'm just talking nightly builds done by Jenkins... not developers
> using eSDK and devtool) and test. Customers using yocto will be given a
> recipe that points to github and, importantly, those customers that do NOT
> use yocto simply fetch the project from github and manually build it in
> their own environment with their own tools.
>
> That is a common problem. I do this all the time. To do this add to your
> conf/local.conf file:
>
> INHERIT += "externalsrc"
> EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
>
> Alternatively, you could use a bbappend file for you recipe in a
> development layer, myrecipe.bbappend:
>
> inherit externalsrc
> EXTERNALSRC = "/path/to/source/tree"
>
> Now, that is exactly what devtool does for you but you can of course do it
> manually.
>
> :rjs
>
>
>
> I can't be the only one doing this. Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif
> wrote:
>
>> Russell,
>>
>> That is exactly what devtool and the externalsrc class do. PREMIRROR is
>> the wrong approach for that.
>>
>> :rjs
>> On 7/24/19 12:53 PM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> I apologize for not being clear. The idea here is that my recipe points
>> to github while, for my local development environment, I set a premirror to
>> match a specific github repository and translate it to a local directory.
>> That works. The fetch matches the PREMIRROR and places a copy of the LOCAL
>> git repo in my DL_DIR. The problem is, the do_unpack task is looking for
>> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
>> repo from github, not my local repo that the fetch task just put in DL_DIR.
>>
>> There are numerous reasons I'm doing this including the fact that I
>> cannot put a reference to my local repo in the bb file since I ship that
>> recipe to my customers and a local pathname is meaningless to them. I
>> prefer to simply modify my local.conf with a PREMIRROR value that has a
>> specific regular expression that matches this particular github repo (and
>> hence does NOT effect all recipes). This is why I wanted to use the
>> PREMIRROR function. Unfortunately, it does not appear to work or at least
>> not work as I expected.
>>
>> --Russ
>>
>>
>>
>>
>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
>> wrote:
>>
>>> Hi Russell,
>>>
>>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>>> set a mirror for all recipes. It's a way for organizations to control where
>>> their YP builds download sources from. It's not intended to be used for a
>>> single recipe. There is no need for that. You simply set SRC_URI in your
>>> recipe to your local git repo. That is what devtool does after downloading
>>> the sources from a remote repo. If you already have the remote repo cloned
>>> locally you can just point devtool to it.
>>>
>>> You can do it manually by creating your recipe and setting SRC_URI like
>>> this:
>>>
>>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>>
>>> SRCREV = "${AUTOREV}"
>>>
>>> S = "${WORKDIR}/git"
>>>
>>>
Re: [yocto] PREMIRROR
Inlining below.
On 7/25/19 8:14 AM, Russell Peterson wrote:
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message
> I saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url. What I didn't understand at the time
> was that while that file existed,/it was actually a link/ to
> DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong bare
> repository as I thought. It was unhappy because it didn't see the git
> commit in the local repo that it was looking for. I use AUTOREV for
> SRCREV. Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the
> recipe. The local git repo doesn't contain that commit. When I sync
> the local git repo to the github repo things work as expected.
Correct. If the source repo or the correct commit cannot be found from
the PREMIRROR, bitbake will use SRC_URI from the recipe. You can block
that behavior by setting BB_NO_NETWORK = "1" in which case bitbake will
not be able to connect to any remote server. However, that does not help
if your PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS
to allow access to only the servers you want.
>
> Here is my problem. What I am ultimately trying to do here is have 2
> git repos. 1 public, 1 private. For internal development we use the
> private git repo (a local gerrit server). Just before we ship we
> update the public github repo. The recipe always points to the github
> (public) repo so we don't need to mess with that before a release.
> This way we can do nightly builds (I'm just talking nightly builds
> done by Jenkins... not developers using eSDK and devtool) and test.
> Customers using yocto will be given a recipe that points to github
> and, importantly, those customers that do NOT use yocto simply fetch
> the project from github and manually build it in their own environment
> with their own tools.
>
That is a common problem. I do this all the time. To do this add to your
conf/local.conf file:
INHERIT += "externalsrc"
EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
Alternatively, you could use a bbappend file for you recipe in a
development layer, myrecipe.bbappend:
inherit externalsrc
EXTERNALSRC = "/path/to/source/tree"
Now, that is exactly what devtool does for you but you can of course do
it manually.
:rjs
> I can't be the only one doing this. Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif
> mailto:rudolf.str...@ibeeto.com>> wrote:
>
> Russell,
>
> That is exactly what devtool and the externalsrc class do.
> PREMIRROR is the wrong approach for that.
>
> :rjs
>
> On 7/24/19 12:53 PM, Russell Peterson wrote:
>> Hi, Rudolf.
>>
>> I apologize for not being clear. The idea here is that my recipe
>> points to github while, for my local development environment, I
>> set a premirror to match a specific github repository and
>> translate it to a local directory. That works. The fetch
>> matches the PREMIRROR and places a copy of the LOCAL git repo in
>> my DL_DIR. The problem is, the do_unpack task is looking for the
>> git repo in the DL_DIR (git2/etc...)... but it's looking for the
>> git repo from github, not my local repo that the fetch task just
>> put in DL_DIR.
>>
>> There are numerous reasons I'm doing this including the fact that
>> I cannot put a reference to my local repo in the bb file since I
>> ship that recipe to my customers and a local pathname is
>> meaningless to them. I prefer to simply modify my local.conf
>> with a PREMIRROR value that has a specific regular expression
>> that matches this particular github repo (and hence does NOT
>> effect all recipes). This is why I wanted to use the PREMIRROR
>> function. Unfortunately, it does not appear to work or at least
>> not work as I expected.
>>
>> --Russ
>>
>>
>>
>>
>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
>> mailto:rudolf.str...@ibeeto.com>> wrote:
>>
>> Hi Russell,
>>
>> devtool and eSDK are different things. The purpose of
>> PREMIRRORS is to set a mirror for all recipes. It's a way for
>> organizations to control where their YP builds download
>> sources from. It's not intended to be used for a single
>> recipe. There is no need for that. You simply set SRC_URI in
>> your recipe to your local git repo. That is what devtool does
>> after downloading the sources from a remote repo. If you
>> already have the remote repo cloned locally you can just
>> point devtool to it.
>>
>> You can do it manually by creating your recipe and setting
>> SRC_URI like this:
>>
>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>
>> SRCREV =
Re: [yocto] PREMIRROR
I think I have a somewhat better understanding of what is going on.
First off, I was confused by the fact that the original error message I saw
from do_unpack referenced the file (URL) at
DL_DIR/git2/original_github_url. What I didn't understand at the time was
that while that file existed,* it was actually a link* to
DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong bare
repository as I thought. It was unhappy because it didn't see the git
commit in the local repo that it was looking for. I use AUTOREV for
SRCREV. Apparently, even with PREMIRROR set and matched, bitbake will
still fetch the git HEAD commit hash from the original URL in the recipe.
The local git repo doesn't contain that commit. When I sync the local git
repo to the github repo things work as expected.
Here is my problem. What I am ultimately trying to do here is have 2 git
repos. 1 public, 1 private. For internal development we use the private
git repo (a local gerrit server). Just before we ship we update the public
github repo. The recipe always points to the github (public) repo so we
don't need to mess with that before a release. This way we can do nightly
builds (I'm just talking nightly builds done by Jenkins... not developers
using eSDK and devtool) and test. Customers using yocto will be given a
recipe that points to github and, importantly, those customers that do NOT
use yocto simply fetch the project from github and manually build it in
their own environment with their own tools.
I can't be the only one doing this. Is there a best practices here
(private vs. public repo)?
--Russ
On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif
wrote:
> Russell,
>
> That is exactly what devtool and the externalsrc class do. PREMIRROR is
> the wrong approach for that.
>
> :rjs
> On 7/24/19 12:53 PM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> I apologize for not being clear. The idea here is that my recipe points
> to github while, for my local development environment, I set a premirror to
> match a specific github repository and translate it to a local directory.
> That works. The fetch matches the PREMIRROR and places a copy of the LOCAL
> git repo in my DL_DIR. The problem is, the do_unpack task is looking for
> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
> repo from github, not my local repo that the fetch task just put in DL_DIR.
>
> There are numerous reasons I'm doing this including the fact that I cannot
> put a reference to my local repo in the bb file since I ship that recipe to
> my customers and a local pathname is meaningless to them. I prefer to
> simply modify my local.conf with a PREMIRROR value that has a specific
> regular expression that matches this particular github repo (and hence does
> NOT effect all recipes). This is why I wanted to use the PREMIRROR
> function. Unfortunately, it does not appear to work or at least not work
> as I expected.
>
> --Russ
>
>
>
>
> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
> wrote:
>
>> Hi Russell,
>>
>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>> set a mirror for all recipes. It's a way for organizations to control where
>> their YP builds download sources from. It's not intended to be used for a
>> single recipe. There is no need for that. You simply set SRC_URI in your
>> recipe to your local git repo. That is what devtool does after downloading
>> the sources from a remote repo. If you already have the remote repo cloned
>> locally you can just point devtool to it.
>>
>> You can do it manually by creating your recipe and setting SRC_URI like
>> this:
>>
>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>
>> SRCREV = "${AUTOREV}"
>>
>> S = "${WORKDIR}/git"
>>
>> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>
>> :rjs
>>
>>
>> On 7/24/19 11:28 AM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> Thanks for the reply. Yes, I am aware of the eSDK functionality,
>> however, I have some unique requirements that I am trying to work around.
>> Regardless... what I am doing should work, no? I simply want to use a
>> local git repo (the directory itself hence protocol=file) instead of what
>> the recipe specifies. Looks like the fetch is working but the do_unpack
>> task is ignoring PREMIRRORS (or at least the localpath variable seems
>> wrong).
>>
>> --Russ
>>
>>
>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <
>> rudolf.str...@ibeeto.com> wrote:
>>
>>> Russell,
>>>
>>> You don't need PREMIRROR for this functionality. It's not exactly
>>> intended for that use.
>>>
>>> The simplest way to achieve what you are looking for is to use devtool.
>>> If I understand you correctly you are downloading sources from a remote
>>> repo on GitHub but want to have them locally to make modifications? If so
>>> use:
>>>
>>> devtool add myrecipe localsrc fetchuri
>>>
>>> from your build environment (you have to source oe-init-build-env
>>> first). devtool then fetches the source
Re: [yocto] PREMIRROR
Russell,
That is exactly what devtool and the externalsrc class do. PREMIRROR is
the wrong approach for that.
:rjs
On 7/24/19 12:53 PM, Russell Peterson wrote:
> Hi, Rudolf.
>
> I apologize for not being clear. The idea here is that my recipe
> points to github while, for my local development environment, I set a
> premirror to match a specific github repository and translate it to a
> local directory. That works. The fetch matches the PREMIRROR and
> places a copy of the LOCAL git repo in my DL_DIR. The problem is, the
> do_unpack task is looking for the git repo in the DL_DIR
> (git2/etc...)... but it's looking for the git repo from github, not my
> local repo that the fetch task just put in DL_DIR.
>
> There are numerous reasons I'm doing this including the fact that I
> cannot put a reference to my local repo in the bb file since I ship
> that recipe to my customers and a local pathname is meaningless to
> them. I prefer to simply modify my local.conf with a PREMIRROR value
> that has a specific regular expression that matches this particular
> github repo (and hence does NOT effect all recipes). This is why I
> wanted to use the PREMIRROR function. Unfortunately, it does not
> appear to work or at least not work as I expected.
>
> --Russ
>
>
>
>
> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
> mailto:rudolf.str...@ibeeto.com>> wrote:
>
> Hi Russell,
>
> devtool and eSDK are different things. The purpose of PREMIRRORS
> is to set a mirror for all recipes. It's a way for organizations
> to control where their YP builds download sources from. It's not
> intended to be used for a single recipe. There is no need for
> that. You simply set SRC_URI in your recipe to your local git
> repo. That is what devtool does after downloading the sources from
> a remote repo. If you already have the remote repo cloned locally
> you can just point devtool to it.
>
> You can do it manually by creating your recipe and setting SRC_URI
> like this:
>
> SRC_URI = "git:///local/path/${PN};protocol=file"
>
> SRCREV = "${AUTOREV}"
>
> S = "${WORKDIR}/git"
>
> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>
> :rjs
>
>
> On 7/24/19 11:28 AM, Russell Peterson wrote:
>> Hi, Rudolf.
>>
>> Thanks for the reply. Yes, I am aware of the eSDK functionality,
>> however, I have some unique requirements that I am trying to work
>> around. Regardless... what I am doing should work, no? I simply
>> want to use a local git repo (the directory itself hence
>> protocol=file) instead of what the recipe specifies. Looks like
>> the fetch is working but the do_unpack task is ignoring
>> PREMIRRORS (or at least the localpath variable seems wrong).
>>
>> --Russ
>>
>>
>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
>> mailto:rudolf.str...@ibeeto.com>> wrote:
>>
>> Russell,
>>
>> You don't need PREMIRROR for this functionality. It's not
>> exactly intended for that use.
>>
>> The simplest way to achieve what you are looking for is to
>> use devtool. If I understand you correctly you are
>> downloading sources from a remote repo on GitHub but want to
>> have them locally to make modifications? If so use:
>>
>> devtool add myrecipe localsrc fetchuri
>>
>> from your build environment (you have to source
>> oe-init-build-env first). devtool then fetches the source
>> from fetchuri into a directory localsrc as a git repo and
>> automatically creates the recipe for it.
>>
>> :rjs
>>
>>
>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>> Hello,
>>>
>>> I am looking to have bitbake pick up files for a particular
>>> recipe from a local git repository using the PREMIRROR
>>> functionality.
>>>
>>> Basically, the recipe (bb file) points to github but in my
>>> local build I add PREMIRROR_prepend = "git://.*/.*
>>> git:///local/path/BASENAME;protocol=file\n"
>>>
>>> I will probably make the git regular expression more exact
>>> for my specific github repo but this works for now.
>>>
>>> This all works (as I deleted the github download from the
>>> local download directory) because I can see in the do_fetch
>>> log and the correct (local) repo was found and placed in the
>>> DL_DIR.
>>>
>>> Problem is, do_unpack fails because it appears to be looking
>>> for the original (github) SRC_URI. Then it complains about
>>> "no up to date source found: clone or directory not
>>> available or not up to date (shallow clone not enabled)"
>>>
>>> Any help on what I am missing would be appreciated.
>>>
>>> Regards,
>>>
>>> Russell
>>>
>>>
>>>
>> --
>> -
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>
Re: [yocto] PREMIRROR
Hi, Rudolf.
I apologize for not being clear. The idea here is that my recipe points to
github while, for my local development environment, I set a premirror to
match a specific github repository and translate it to a local directory.
That works. The fetch matches the PREMIRROR and places a copy of the LOCAL
git repo in my DL_DIR. The problem is, the do_unpack task is looking for
the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
repo from github, not my local repo that the fetch task just put in DL_DIR.
There are numerous reasons I'm doing this including the fact that I cannot
put a reference to my local repo in the bb file since I ship that recipe to
my customers and a local pathname is meaningless to them. I prefer to
simply modify my local.conf with a PREMIRROR value that has a specific
regular expression that matches this particular github repo (and hence does
NOT effect all recipes). This is why I wanted to use the PREMIRROR
function. Unfortunately, it does not appear to work or at least not work
as I expected.
--Russ
On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
wrote:
> Hi Russell,
>
> devtool and eSDK are different things. The purpose of PREMIRRORS is to set
> a mirror for all recipes. It's a way for organizations to control where
> their YP builds download sources from. It's not intended to be used for a
> single recipe. There is no need for that. You simply set SRC_URI in your
> recipe to your local git repo. That is what devtool does after downloading
> the sources from a remote repo. If you already have the remote repo cloned
> locally you can just point devtool to it.
>
> You can do it manually by creating your recipe and setting SRC_URI like
> this:
>
> SRC_URI = "git:///local/path/${PN};protocol=file"
>
> SRCREV = "${AUTOREV}"
>
> S = "${WORKDIR}/git"
>
> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>
> :rjs
>
>
> On 7/24/19 11:28 AM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> Thanks for the reply. Yes, I am aware of the eSDK functionality, however,
> I have some unique requirements that I am trying to work around.
> Regardless... what I am doing should work, no? I simply want to use a
> local git repo (the directory itself hence protocol=file) instead of what
> the recipe specifies. Looks like the fetch is working but the do_unpack
> task is ignoring PREMIRRORS (or at least the localpath variable seems
> wrong).
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
> wrote:
>
>> Russell,
>>
>> You don't need PREMIRROR for this functionality. It's not exactly
>> intended for that use.
>>
>> The simplest way to achieve what you are looking for is to use devtool.
>> If I understand you correctly you are downloading sources from a remote
>> repo on GitHub but want to have them locally to make modifications? If so
>> use:
>>
>> devtool add myrecipe localsrc fetchuri
>>
>> from your build environment (you have to source oe-init-build-env first).
>> devtool then fetches the source from fetchuri into a directory localsrc as
>> a git repo and automatically creates the recipe for it.
>>
>> :rjs
>>
>>
>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>
>> Hello,
>>
>> I am looking to have bitbake pick up files for a particular recipe from a
>> local git repository using the PREMIRROR functionality.
>>
>> Basically, the recipe (bb file) points to github but in my local build I
>> add PREMIRROR_prepend = "git://.*/.*
>> git:///local/path/BASENAME;protocol=file\n"
>>
>> I will probably make the git regular expression more exact for my
>> specific github repo but this works for now.
>>
>> This all works (as I deleted the github download from the local download
>> directory) because I can see in the do_fetch log and the correct (local)
>> repo was found and placed in the DL_DIR.
>>
>> Problem is, do_unpack fails because it appears to be looking for the
>> original (github) SRC_URI. Then it complains about "no up to date source
>> found: clone or directory not available or not up to date (shallow clone
>> not enabled)"
>>
>> Any help on what I am missing would be appreciated.
>>
>> Regards,
>>
>> Russell
>>
>>
>>
>> --
>> -
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] PREMIRROR
Hi Russell,
devtool and eSDK are different things. The purpose of PREMIRRORS is to
set a mirror for all recipes. It's a way for organizations to control
where their YP builds download sources from. It's not intended to be
used for a single recipe. There is no need for that. You simply set
SRC_URI in your recipe to your local git repo. That is what devtool does
after downloading the sources from a remote repo. If you already have
the remote repo cloned locally you can just point devtool to it.
You can do it manually by creating your recipe and setting SRC_URI like
this:
SRC_URI = "git:///local/path/${PN};protocol=file"
SRCREV = "${AUTOREV}"
S = "${WORKDIR}/git"
PREMIRRORS is only relevant for do_fetch not for do_unpack.
:rjs
On 7/24/19 11:28 AM, Russell Peterson wrote:
> Hi, Rudolf.
>
> Thanks for the reply. Yes, I am aware of the eSDK functionality,
> however, I have some unique requirements that I am trying to work
> around. Regardless... what I am doing should work, no? I simply want
> to use a local git repo (the directory itself hence protocol=file)
> instead of what the recipe specifies. Looks like the fetch is working
> but the do_unpack task is ignoring PREMIRRORS (or at least the
> localpath variable seems wrong).
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
> mailto:rudolf.str...@ibeeto.com>> wrote:
>
> Russell,
>
> You don't need PREMIRROR for this functionality. It's not exactly
> intended for that use.
>
> The simplest way to achieve what you are looking for is to use
> devtool. If I understand you correctly you are downloading sources
> from a remote repo on GitHub but want to have them locally to make
> modifications? If so use:
>
> devtool add myrecipe localsrc fetchuri
>
> from your build environment (you have to source oe-init-build-env
> first). devtool then fetches the source from fetchuri into a
> directory localsrc as a git repo and automatically creates the
> recipe for it.
>
> :rjs
>
>
> On 7/23/19 1:49 PM, Russell Peterson wrote:
>> Hello,
>>
>> I am looking to have bitbake pick up files for a particular
>> recipe from a local git repository using the PREMIRROR functionality.
>>
>> Basically, the recipe (bb file) points to github but in my local
>> build I add PREMIRROR_prepend = "git://.*/.*
>> git:///local/path/BASENAME;protocol=file\n"
>>
>> I will probably make the git regular expression more exact for my
>> specific github repo but this works for now.
>>
>> This all works (as I deleted the github download from the local
>> download directory) because I can see in the do_fetch log and the
>> correct (local) repo was found and placed in the DL_DIR.
>>
>> Problem is, do_unpack fails because it appears to be looking for
>> the original (github) SRC_URI. Then it complains about "no up to
>> date source found: clone or directory not available or not up to
>> date (shallow clone not enabled)"
>>
>> Any help on what I am missing would be appreciated.
>>
>> Regards,
>>
>> Russell
>>
>>
>>
> --
> -
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
--
-
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700
signature.asc
Description: OpenPGP digital signature
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] PREMIRROR
Thanks, Yi. I did try that and I see the tar file is created... but again, the do_unpack function seems to ignore it and go directly to the github repository. --Russ On Wed, Jul 24, 2019 at 4:28 AM Yi Zhao wrote: > > On 7/24/19 4:49 AM, Russell Peterson wrote: > > Hello, > > I am looking to have bitbake pick up files for a particular recipe from a > local git repository using the PREMIRROR functionality. > > Basically, the recipe (bb file) points to github but in my local build I > add PREMIRROR_prepend = "git://.*/.* > git:///local/path/BASENAME;protocol=file\n" > > I will probably make the git regular expression more exact for my specific > github repo but this works for now. > > This all works (as I deleted the github download from the local download > directory) because I can see in the do_fetch log and the correct (local) > repo was found and placed in the DL_DIR. > > Problem is, do_unpack fails because it appears to be looking for the > original (github) SRC_URI. Then it complains about "no up to date source > found: clone or directory not available or not up to date (shallow clone > not enabled)" > > > Maybe you can use BB_GENERATE_MIRROR_TARBALLS to generate git repo tarball > and put it to your download mirror. > > See: > https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-bb-BB_GENERATE_MIRROR_TARBALLS > > and > > > https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DL_DIR > > > Regards, > > Yi > > > > Any help on what I am missing would be appreciated. > > Regards, > > Russell > > > > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] PREMIRROR
Hi, Rudolf. Thanks for the reply. Yes, I am aware of the eSDK functionality, however, I have some unique requirements that I am trying to work around. Regardless... what I am doing should work, no? I simply want to use a local git repo (the directory itself hence protocol=file) instead of what the recipe specifies. Looks like the fetch is working but the do_unpack task is ignoring PREMIRRORS (or at least the localpath variable seems wrong). --Russ On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif wrote: > Russell, > > You don't need PREMIRROR for this functionality. It's not exactly intended > for that use. > > The simplest way to achieve what you are looking for is to use devtool. If > I understand you correctly you are downloading sources from a remote repo > on GitHub but want to have them locally to make modifications? If so use: > > devtool add myrecipe localsrc fetchuri > > from your build environment (you have to source oe-init-build-env first). > devtool then fetches the source from fetchuri into a directory localsrc as > a git repo and automatically creates the recipe for it. > > :rjs > > > On 7/23/19 1:49 PM, Russell Peterson wrote: > > Hello, > > I am looking to have bitbake pick up files for a particular recipe from a > local git repository using the PREMIRROR functionality. > > Basically, the recipe (bb file) points to github but in my local build I > add PREMIRROR_prepend = "git://.*/.* > git:///local/path/BASENAME;protocol=file\n" > > I will probably make the git regular expression more exact for my specific > github repo but this works for now. > > This all works (as I deleted the github download from the local download > directory) because I can see in the do_fetch log and the correct (local) > repo was found and placed in the DL_DIR. > > Problem is, do_unpack fails because it appears to be looking for the > original (github) SRC_URI. Then it complains about "no up to date source > found: clone or directory not available or not up to date (shallow clone > not enabled)" > > Any help on what I am missing would be appreciated. > > Regards, > > Russell > > > > -- > - > Rudolf J Streif > CEO/CTO ibeeto > +1.855.442.3396 x700 > > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] PREMIRROR
Russell, You don't need PREMIRROR for this functionality. It's not exactly intended for that use. The simplest way to achieve what you are looking for is to use devtool. If I understand you correctly you are downloading sources from a remote repo on GitHub but want to have them locally to make modifications? If so use: devtool add myrecipe localsrc fetchuri from your build environment (you have to source oe-init-build-env first). devtool then fetches the source from fetchuri into a directory localsrc as a git repo and automatically creates the recipe for it. :rjs On 7/23/19 1:49 PM, Russell Peterson wrote: > Hello, > > I am looking to have bitbake pick up files for a particular recipe > from a local git repository using the PREMIRROR functionality. > > Basically, the recipe (bb file) points to github but in my local build > I add PREMIRROR_prepend = "git://.*/.* > git:///local/path/BASENAME;protocol=file\n" > > I will probably make the git regular expression more exact for my > specific github repo but this works for now. > > This all works (as I deleted the github download from the local > download directory) because I can see in the do_fetch log and the > correct (local) repo was found and placed in the DL_DIR. > > Problem is, do_unpack fails because it appears to be looking for the > original (github) SRC_URI. Then it complains about "no up to date > source found: clone or directory not available or not up to date > (shallow clone not enabled)" > > Any help on what I am missing would be appreciated. > > Regards, > > Russell > > > -- - Rudolf J Streif CEO/CTO ibeeto +1.855.442.3396 x700 signature.asc Description: OpenPGP digital signature -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] PREMIRROR
On 7/24/19 4:49 AM, Russell Peterson wrote: Hello, I am looking to have bitbake pick up files for a particular recipe from a local git repository using the PREMIRROR functionality. Basically, the recipe (bb file) points to github but in my local build I add PREMIRROR_prepend = "git://.*/.* git:///local/path/BASENAME;protocol=file\n" I will probably make the git regular expression more exact for my specific github repo but this works for now. This all works (as I deleted the github download from the local download directory) because I can see in the do_fetch log and the correct (local) repo was found and placed in the DL_DIR. Problem is, do_unpack fails because it appears to be looking for the original (github) SRC_URI. Then it complains about "no up to date source found: clone or directory not available or not up to date (shallow clone not enabled)" Maybe you can use BB_GENERATE_MIRROR_TARBALLS to generate git repo tarball and put it to your download mirror. See: https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-bb-BB_GENERATE_MIRROR_TARBALLS and https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DL_DIR Regards, Yi Any help on what I am missing would be appreciated. Regards, Russell -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] PREMIRROR
Hello, I am looking to have bitbake pick up files for a particular recipe from a local git repository using the PREMIRROR functionality. Basically, the recipe (bb file) points to github but in my local build I add PREMIRROR_prepend = "git://.*/.* git:///local/path/BASENAME;protocol=file\n" I will probably make the git regular expression more exact for my specific github repo but this works for now. This all works (as I deleted the github download from the local download directory) because I can see in the do_fetch log and the correct (local) repo was found and placed in the DL_DIR. Problem is, do_unpack fails because it appears to be looking for the original (github) SRC_URI. Then it complains about "no up to date source found: clone or directory not available or not up to date (shallow clone not enabled)" Any help on what I am missing would be appreciated. Regards, Russell -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
