Inlining below. On 7/25/19 8:14 AM, Russell Peterson wrote: > I think I have a somewhat better understanding of what is going on. > > First off, I was confused by the fact that the original error message > I saw from do_unpack referenced the file (URL) at > DL_DIR/git2/original_github_url. What I didn't understand at the time > was that while that file existed,/it was actually a link/ to > DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong bare > repository as I thought. It was unhappy because it didn't see the git > commit in the local repo that it was looking for. I use AUTOREV for > SRCREV. Apparently, even with PREMIRROR set and matched, bitbake will > still fetch the git HEAD commit hash from the original URL in the > recipe. The local git repo doesn't contain that commit. When I sync > the local git repo to the github repo things work as expected. Correct. If the source repo or the correct commit cannot be found from the PREMIRROR, bitbake will use SRC_URI from the recipe. You can block that behavior by setting BB_NO_NETWORK = "1" in which case bitbake will not be able to connect to any remote server. However, that does not help if your PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS to allow access to only the servers you want. > > Here is my problem. What I am ultimately trying to do here is have 2 > git repos. 1 public, 1 private. For internal development we use the > private git repo (a local gerrit server). Just before we ship we > update the public github repo. The recipe always points to the github > (public) repo so we don't need to mess with that before a release. > This way we can do nightly builds (I'm just talking nightly builds > done by Jenkins... not developers using eSDK and devtool) and test. > Customers using yocto will be given a recipe that points to github > and, importantly, those customers that do NOT use yocto simply fetch > the project from github and manually build it in their own environment > with their own tools. > That is a common problem. I do this all the time. To do this add to your conf/local.conf file:
INHERIT += "externalsrc" EXTERNALSRC_pn-myrecipe = "/path/to/source/tree" Alternatively, you could use a bbappend file for you recipe in a development layer, myrecipe.bbappend: inherit externalsrc EXTERNALSRC = "/path/to/source/tree" Now, that is exactly what devtool does for you but you can of course do it manually. :rjs > I can't be the only one doing this. Is there a best practices here > (private vs. public repo)? > > --Russ > > > On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif > <rudolf.str...@ibeeto.com <mailto:rudolf.str...@ibeeto.com>> wrote: > > Russell, > > That is exactly what devtool and the externalsrc class do. > PREMIRROR is the wrong approach for that. > > :rjs > > On 7/24/19 12:53 PM, Russell Peterson wrote: >> Hi, Rudolf. >> >> I apologize for not being clear. The idea here is that my recipe >> points to github while, for my local development environment, I >> set a premirror to match a specific github repository and >> translate it to a local directory. That works. The fetch >> matches the PREMIRROR and places a copy of the LOCAL git repo in >> my DL_DIR. The problem is, the do_unpack task is looking for the >> git repo in the DL_DIR (git2/etc...)... but it's looking for the >> git repo from github, not my local repo that the fetch task just >> put in DL_DIR. >> >> There are numerous reasons I'm doing this including the fact that >> I cannot put a reference to my local repo in the bb file since I >> ship that recipe to my customers and a local pathname is >> meaningless to them. I prefer to simply modify my local.conf >> with a PREMIRROR value that has a specific regular expression >> that matches this particular github repo (and hence does NOT >> effect all recipes). This is why I wanted to use the PREMIRROR >> function. Unfortunately, it does not appear to work or at least >> not work as I expected. >> >> --Russ >> >> >> >> >> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif >> <rudolf.str...@ibeeto.com <mailto:rudolf.str...@ibeeto.com>> wrote: >> >> Hi Russell, >> >> devtool and eSDK are different things. The purpose of >> PREMIRRORS is to set a mirror for all recipes. It's a way for >> organizations to control where their YP builds download >> sources from. It's not intended to be used for a single >> recipe. There is no need for that. You simply set SRC_URI in >> your recipe to your local git repo. That is what devtool does >> after downloading the sources from a remote repo. If you >> already have the remote repo cloned locally you can just >> point devtool to it. >> >> You can do it manually by creating your recipe and setting >> SRC_URI like this: >> >> SRC_URI = "git:///local/path/${PN};protocol=file" >> >> SRCREV = "${AUTOREV}" >> >> S = "${WORKDIR}/git" >> >> PREMIRRORS is only relevant for do_fetch not for do_unpack. >> >> :rjs >> >> >> On 7/24/19 11:28 AM, Russell Peterson wrote: >>> Hi, Rudolf. >>> >>> Thanks for the reply. Yes, I am aware of the eSDK >>> functionality, however, I have some unique requirements that >>> I am trying to work around. Regardless... what I am doing >>> should work, no? I simply want to use a local git repo (the >>> directory itself hence protocol=file) instead of what the >>> recipe specifies. Looks like the fetch is working but the >>> do_unpack task is ignoring PREMIRRORS (or at least the >>> localpath variable seems wrong). >>> >>> --Russ >>> >>> >>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif >>> <rudolf.str...@ibeeto.com <mailto:rudolf.str...@ibeeto.com>> >>> wrote: >>> >>> Russell, >>> >>> You don't need PREMIRROR for this functionality. It's >>> not exactly intended for that use. >>> >>> The simplest way to achieve what you are looking for is >>> to use devtool. If I understand you correctly you are >>> downloading sources from a remote repo on GitHub but >>> want to have them locally to make modifications? If so use: >>> >>> devtool add myrecipe localsrc fetchuri >>> >>> from your build environment (you have to source >>> oe-init-build-env first). devtool then fetches the >>> source from fetchuri into a directory localsrc as a git >>> repo and automatically creates the recipe for it. >>> >>> :rjs >>> >>> >>> On 7/23/19 1:49 PM, Russell Peterson wrote: >>>> Hello, >>>> >>>> I am looking to have bitbake pick up files for a >>>> particular recipe from a local git repository using the >>>> PREMIRROR functionality. >>>> >>>> Basically, the recipe (bb file) points to github but in >>>> my local build I add PREMIRROR_prepend = "git://.*/.* >>>> git:///local/path/BASENAME;protocol=file\n" >>>> >>>> I will probably make the git regular expression more >>>> exact for my specific github repo but this works for now. >>>> >>>> This all works (as I deleted the github download from >>>> the local download directory) because I can see in the >>>> do_fetch log and the correct (local) repo was found and >>>> placed in the DL_DIR. >>>> >>>> Problem is, do_unpack fails because it appears to be >>>> looking for the original (github) SRC_URI. Then it >>>> complains about "no up to date source found: clone or >>>> directory not available or not up to date (shallow >>>> clone not enabled)" >>>> >>>> Any help on what I am missing would be appreciated. >>>> >>>> Regards, >>>> >>>> Russell >>>> >>>> >>>> >>> -- >>> ----- >>> Rudolf J Streif >>> CEO/CTO ibeeto >>> +1.855.442.3396 x700 >>> >> -- >> ----- >> Rudolf J Streif >> CEO/CTO ibeeto >> +1.855.442.3396 x700 >> > -- > ----- > Rudolf J Streif > CEO/CTO ibeeto > +1.855.442.3396 x700 > -- ----- Rudolf J Streif CEO/CTO ibeeto +1.855.442.3396 x700
signature.asc
Description: OpenPGP digital signature
-- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
