On Fri, 2007-03-09 at 04:38 +0100, Henryk Plötz wrote: > Am Fri, 09 Mar 2007 00:37:03 +0000 schrieb Richard Purdie: > > > My first concern is that we should really change the PV, not PR. PR > > represents a particular bitbake recipe revision and you're changing > > the underlying code which is PV. > > I see. One of the reasons for doing this was that I was striving for > genericity especially with a look on how multiple fetchers would work > together. For example uboot-gta01 is built with sources from git with > additional patches from svn. This necessitates: a) either each fetcher > needs to introduce its own revision identifier or b) all fetchers must > use a compatible scheme with a well-defined max() function (e.g. > date/time).
I can see a problem here but this doesn't really affect whether you change PV or PR though? > In any case involving more than one revision identifier it is > important that these are 'worth' less than the bitbake recipe version. Why? This is exactly the opposite behaviour to the way its defined to work. > E.g. suppose I took some fictional package with source from cvs and git > today with its revision identifiers ["20070308052342", > "eb92f613556800f7483666db09d9a237ad911d4a"] (this is the naive approach > with the sha1 for the git revision identifier, just ignore that for > now) and added some patches from an svn source. The resulting list of > revision identifiers could look something like ["1229", > "20070308052342", "eb92f613556800f7483666db09d9a237ad911d4a"] and this > would be decidedly 'less than' the original list. However, since I > changed the bitbake recipe I would also have been bumping the PR. If > the source revision was 'worth' less than the PR, then everything would > be ok. PR is defined to be worth less that PV and this assumption is hardcoded into the underlying tools and into the developers minds. When you add a new version you zero PR. Looking at the above problem, you would just have to ensure the order of the identifiers was properly controlled in PV. As long as the new SCM was at the end of the version, there shouldn't be a problem. Regarding the issue of multiple SCMs making up a PV, I have written .bb files to use multiple SCMs before, specifically http://svn.o-hand.com/view/poky/trunk/meta/packages/qemu/qemu_cvs.bb?rev=1164&view=auto where I added a srcdate tag to the URL so you could set srcdate per entry in SRC_URI. This doesn't solve your problem though. I think we need to expose more data from the fetchers such as a REVISION tag for each SCM referenced in SRC_URI so you could do something like: SRC_URI = "\ cvs://someurl;refname=scm1 \ svn://someurl;refname=scm2" PV = "x.y.z+cvs${SRCREVISION_scm1}+svn{SRCREVISION_scm2}" Setting such a PV doesn't seem like too much of a pain for these relatively rare corner case packages. You could even automate this to an extent by having: BASEPV = "x.y.z" and then the fetcher sets PV for you after examining SRC_URI (allowing the default to be overridden of course). > This could be avoided if we had compatible 'last changed' date/times > for each of the URLs and then take the most recent of those. We can't do that for git as git is timeless so this solution simply won't work. > We can't get this information for all of the fetchers, and even if we > could, it would not be reliable because it would be dependent on the > correctness of remote clocks. And even if we ignore that, then these > date/times are not compatible because they are dependent on _different_ > remote clocks that may or may not be synchronized. Agreed. > I have an idea for a partial solution to this: instead of 'last > changed' use 'last change detected'. Each fetcher would need to store > some semi-permanent local metadata for each url it handles which should > include the date/time that the last change was detected and some kind > of data to determine when a change happens (kind of like HTTP ETag). > For SVN this would be 'Last Changed Rev', for GIT this would be the > SHA-1, and for CVS ... don't know, list of files and file revisions (or > hash thereof) maybe. For CVS you have the date and time of the last time HEAD changed. > Pro: relatively reliable, generic, should apply to all version control > systems > Con: revision identifiers generated this way are not comparable between > different developers For svn and cvs, we can make it work. git is going to be a problem between different developers. I don't see any way around the problem with git (and I'm really not keen on using dates for it). > This is especially of concern since openmoko wants to share the > tarballs through CVS_TARBALL_STASH at > http://downloads.openmoko.org/sources/ so that the generation of an > interchangeable filename should be a separate problem. Having the git > hash in the tarball name should be ok. The tarball stash is easy and basically should already basically work. For CVS, we use a date (but we should make this the date/time HEAD last changed for "now" targets), for svn, we should use revision and for git, it already uses the hash in the filename. > As indicated above these are viable strategies when there is only one > fetcher with only one source involved, but they don't cooperate well. Perhaps you can see how it can be made to work above. > For git specifically there's also the problem that these sequence > numbers don't compare between developers. That's why I was suggesting a > 'last change detected' date/time above. I'm not keen on this, it sounds useful but its prone to go wrong in ways that might not immediately be obvious (clock not synced, need to be careful of timezone and git revisions can appear backwards in time after a merge). I'd much prefer a policy of an obvious local revision and anything for real use should have a tag. If its obvious, people will realise the problem, if we hide it, they won't. We could always embed the particular git revision used into the package description or something so developers can find it if they need it. > My intuition indicates that there might be an even better solution for > git (that maybe even partially applies to cvs): If git won't give us a > 'Last Changed Rev', just compute one. E.g. count the number of commits > from either the origin or at least a defined (in the recipe) base > commit on. (Changing the base commit should bump the recipe revision > and everything would be ok again.) However, I'm not familiar with git > and don't know if that's feasible (preferably without downloading the > whole repository). You'd need to download the whole repository to do that. I think this is something to perhaps ask the git developers about. > > The problem is not just limited to bitbake. The package version fields > > (PV-PR) need very careful handling since these are used in the > > resulting packages and then the packages go into feeds and/or are > > used to build images. The feed and image generation tools need to > > have package version fields they can compare. Non-integer fields are > > really just not an option. > > Right. Alternatively one might investigate into updating the tools. This means changing the defined behaviour of things like ipkg, apt, dpkg and Debian packaging policy. This is not something many developers will accept and I'm not sure it would work either. > Or > now that I think about it: Having the current date (at build time) in > the version information of the generated package is just fine, as long > as it's different from the information that is used to determine when > to rebuild. PV-PR is what it used to determine if the package should be rebuilt. At the moment, you could just set SRCDATE to the date of last change and it will then rebuild or not as appropriate. You wanted more information about the exact revisions used and I agree this would be good, hence this discussion. Regards, Richard _______________________________________________ Bitbake-dev mailing list [email protected] https://lists.berlios.de/mailman/listinfo/bitbake-dev
