Hello Matthew, On 04/22/20 02:05, macarl via [] wrote: > Hey Lazlo, > > This is a great point of discussion. Just to make sure we're on the same > page, let me paraphrase your scenario. Two features at the same time or a > improperly tested new feature were committed in BaseTools and a new release > was spun out. EDK2 was updated to have it's pip-requirements.txt to point to > the new version committed. In an ideal world, CI should catch the build break > for MdeModulePkg but let's say it didn't catch it. Your scenario is, how do > you go back to the previous version of basetools that EDK2 used or perhaps go > the the version of BaseTools that last worked. > > There are several great ways to do this. The release cadence of a pip based > BaseTools hasn't really been discussed but one option was every feature in > BaseTools would be a new release (this would be easy to do via pipelines). > The exact versioning would need to be discussed but there would likely be > some notion of breaking changes with major and minor version. This means that > you would be able to easily pip install the basetools 1:1 with the feature > commit history. In the scenario you mentioned, it would be easy to roll back > to C0, just "pip install -r the previous requirements file". If a virtual > environment is setup, this pip install would be entirely localized to the > workspace the user is operating in. The git show command you mentioned would > show the pip-requirements file. We would likely tag the pip_requirements file > to a specific release of BaseTools and there might be another file like > pip_suggestions that would allow for a looser version such as 0.3.x were any > minor version in 0.3 is fine. That said, determining what exact basetools was > used in a given build would likely be best accomplished through build > artifacts. We have a BuildToolsReport generator that specifically does this ( > https://github.com/tianocore/edk2/tree/master/BaseTools/Plugin/BuildToolsReport > ) and it would generate a report that would detail the exact pip version > used in that build. Without build artifacts, it grows a little bit more > fuzzy, but you can have reasonable confidence since it would need to at least > conform to pip suggestions and likely was on pip_requirements. To summarize, > moving back in time is simple and even trivial with minimal delay to a > developer (pip installs are usually sub-second procedures). There might even > be a part of edksetup that might do the pip install for you if that's what > the community wants. > > Going to the next step, if you wanted to debug the BaseTools at C1, you might > want to generate a diff of the basetools to try and see where the error and > compare the diff. This I think better addresses your question of > bisectability (not sure if that's a word). You could clone the basetools repo > and do a pip install -e {path to basetools}. You would checkout the version > in C1 pip_requirements and generate the diff between it and C0. Then you > could apply changes or revert changes and test them locally as the basetools > used in your edk2 repo will be the local repo you cloned. Once you're done, > simply do a pip install of your pip_requirements file and the symlinks will > be overwritten and things will go back to the way they were before. > > Hopefully that addresses your concerns.
Thanks for the explanation. So here's how I understand: * When I move around the edk2 git history, the "pip-requirements.txt" file is tracked as an integral part of that history. So, at whichever edk2 commit I've checked out the edk2 tree, I can just run "pip install -r pip-requirements.txt", and then the *matching* basetools state will be installed in my virtual environment. Is that correct? If it is, then it sounds good. It is not very different from running a git submodule update command. * Where would "pip install -r pip-requirements.txt" *fetch* the required basetools version from? Is it always a network download, or can it be performed locally? * Can you please explain the effects of the "pip install -e <basetools-git-checkout-dir>" command in more detail? I've tried googling it, not very successfully. If I understand correctly, it installs the current checkout of basetools from *that* directory, into the virtual environment. Is that right? Is this something that works "automatically", or would basetools have to be developed (in its own git repo) explicitly such that it be compatible with "pip install -e"? I'm asking because I would like to avoid a situation where a particular BaseTools checkout is not installable with "pip install -e"; that is, without "packaging up" that BaseTools state into some package / distribution format. It would be great if "pip install -e" would automatically work from any random BaseTools checkout. * Regarding the "release cadence of a pip based BaseTools", I'd like edk2's "pip-requirements.txt" to track the "BaseTools master branch" very-very closely. I'd like edk2 to consume BaseTools changes immediately, and with a fine granularity. Problems in big BaseTools releases are difficult to recover from. Thanks! Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57830): https://edk2.groups.io/g/devel/message/57830 Mute This Topic: https://groups.io/mt/73069134/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
