Re: [RFC] dropping of $(AUTORELEASE) feature
Hi, I have another idea that might simplify the mess of dealing with PKG_RELEASE handling. 1. we use AUTORELEASE for all packages in master 2. when creating a release branch, all PKG_RELEASE lines are changed to something like 22.03-1 3. we write a script that can automatically edit a series of local commits and for each commit: - figures out affected packages - edits each affected makefile to bump PKG_RELEASE - adds a Updated-Packages: <...> tag to indicate which packages it has changed (or 'none' if no packages were affected) 4. we ask people submitting patches via email or PR to not update PKG_RELEASE themselves anymore 5. we make the server side enforce the presence of the "Updated-Packages" tag for release branches and packages, just like Signed-off-by. That way, dealing with PKG_RELEASE becomes a matter of simply running one command before doing git push. What do you think? - Felix ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
I know that autorelease has introduced some problems but it would be
good to keep a way to automatically bump releases. Every second we
could save from devs/maintainers is more time to do really useful
things.
Can't we have a central way to publish the "official list of package
releases''? It could be a file inside the git, but automatically
updated by a bot when we build packages for publishing. Any local
changes (git can track them without a history) or local commit will
introduce some "dirty tag" to the release number, similar to what
already happens with the openwrt build number (r+x-abcdef). For
example:
foo-1.3-34 (oficial built)
foo-1.3-34.2.abcd12ef+ ("2" local commits being the last one
"abcd12ef" and some extra ("+") uncommited changes)
foo-1.3-34+ (package with some extra ("+") uncommited changes)
If we have reproducible builds, we could bump that release only when
the final binary differs from the last one, skipping cosmetic changes
that do not change the final package but including external ones that
affect one or more packages. It would even avoid bumping subpackages
when the source changes when only some of them really changed.
If that would add too much noise to the history, that same official
list of packages could be published somewhere else. For example, it
could have a file with a list of commits on each branch and the
corresponding "official list of package releases". Something like:
master:
b97e5ac785960c13199239dd4821dd53f3801da3 master-0454
6f729163b18fb5860f1aa5a5a0c8861a8e3f53ad master-0454 <<< built bot
published this one
9179f484bfcb37e1c59e736b2a64c9583eb00356 master-0453
...
master-454:
foo-1.3 34
bar-1.2 45
...
The client would download that list and iteratively look for the last
commit that is found in the list and download the official list of
packages. Count the commits since that commit used for publishing
(including those in the master file or local history) and adjust the
releases. If someone checks out a tagged commit or one used to publish
packages, all releases will match the official one. If not, it will
use some extra release numbers.
There might be a lot of improvements, like sorting the commit list,
plenty of different solutions that solve this problem and maybe some
of them already exist in other build solutions (like OBS). I just
don't like the idea of returning to manual management.
Regards,
Luiz
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
Hi, > I agree that this increases maintenance burden but I believe that a > CI side solution for those > conflicting-because-of-deviating-PKG_RELEASE situations would be the > better course of action. I would also like to drop the AUTORELEASE feature. I would like to have a specific version in the tree. AUTORELEASE makes it also harder to rebuild OpenWrt for reproducible builds when the git tree is not available. What could we do to make it easier for the package maintainers? Add a linter script to warn if a Makefile or any file under a package has been modified but not the pkgversion or pkgrelease? Best, lynxis ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
On Mon, Nov 7, 2022 at 10:29 AM Jo-Philipp Wich wrote: > > Hi, > > > The AUTORELEASE has been a nice feature from the package PR maintenance > > perspective. > > > > Earlier there was constant trouble with concurrent PRs for the same package > > having the same PKG_RELEASE bump, or the maintainer doing a small change > > with > > a bump while there was an open PR with the same bump. The led to trivial > > conflicts and rebase needs for PRs. AUTORELEASE has tackled that nicely. > > it tackled it by sweeping the problem under the carpet and hiding the fact > that those concurrent PRs were indeed conflicting. If the version targeted by > a PR has been changed in the meanwhile it should not be made trivially > mergable by hiding the actual revision from the sources. > > I agree that this increases maintenance burden but I believe that a CI side > solution for those conflicting-because-of-deviating-PKG_RELEASE situations > would be the better course of action. > So, is there a proposal/conclusion on how to proceed? Or do we need/wait for a further discussion? I'm fine either way, and whatever is decided, I will implement it. I have a bunch of packages that I've been maintaining and been slowly switching to the AUTORELEASE feature. And I need to start doing some updates (for some packages that have new releases). And would like (if possible) to know what to do. I will not add AUTORELEASE (if any packages have the case). But, it will be a slow transition back to manual PKG_RELEASE. I mean, I would do it in a not-very-fast-manner. There's 469 packages using AUTORELEASE in the packages repo. And 70 in openwrt core. Core has a different speed/rythm. But for the packages feed, I think, in the course of 2-4 weeks and a few PRs, we should be able to remove it (if needed). > ~ Jo > > ___ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
Hi, > The AUTORELEASE has been a nice feature from the package PR maintenance > perspective. > > Earlier there was constant trouble with concurrent PRs for the same package > having the same PKG_RELEASE bump, or the maintainer doing a small change with > a bump while there was an open PR with the same bump. The led to trivial > conflicts and rebase needs for PRs. AUTORELEASE has tackled that nicely. it tackled it by sweeping the problem under the carpet and hiding the fact that those concurrent PRs were indeed conflicting. If the version targeted by a PR has been changed in the meanwhile it should not be made trivially mergable by hiding the actual revision from the sources. I agree that this increases maintenance burden but I believe that a CI side solution for those conflicting-because-of-deviating-PKG_RELEASE situations would be the better course of action. ~ Jo signature.asc Description: OpenPGP digital signature ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
Hi, yes, please kill it. The $(AUTORELEASE) option does not work for sources without Git history, it produces different results depending on the history, it causes package bumps for even trivial cosmetic fixes. It can also lead to situations where packages on different branches end up with the exact same version but different sets of commits applied to each branch. ~ Jo signature.asc Description: OpenPGP digital signature ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
Hannu Nyman writes: > The AUTORELEASE has been a nice feature from the package PR > maintenance perspective. > > Earlier there was constant trouble with concurrent PRs for the same > package having the same PKG_RELEASE bump, or the maintainer doing a > small change with a bump while there was an open PR with the same > bump. The led to trivial conflicts and rebase needs for > PRs. AUTORELEASE has tackled that nicely. It won't be as pretty as AUTORELEASE, but will something like AUTORELEASE=$(shell git log --format=%ct -1) do as a replacement? It should work with shallow clones and a produce a monotonic increasing integer usable in the package revision. Bjørn ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
On Mon, Nov 7, 2022 at 9:41 AM Josef Schlehofer wrote: > > > On 06. 11. 22 21:22, Hannu Nyman wrote: > > Paul Spooren kirjoitti 6.11.2022 klo 18.15: > >> While I initially thought that $(AUTORELEASE) would be a nice feature > >> to avoid the standard review comment “Please bump the PKG_RELEASE”, > >> it turned into a massive increase of bandwidth usage: Every checkout > >> of openwrt.git and package feeds needs to be a full clone instead of > >> a shallow one to have all commits in place. > >> > >> I’m suggesting dropping that feature and instead implement a (GitHub) > >> CI workflow which fails if the commit did not bump the PKG_RELEASE. > >> > >> It wasn’t a fully thought through idea and I think it’s time to > >> revert it - sorry for the inconvenience. > >> > >> Other thoughts? > > > > > > The AUTORELEASE has been a nice feature from the package PR > > maintenance perspective. > > > > Earlier there was constant trouble with concurrent PRs for the same > > package having the same PKG_RELEASE bump, or the maintainer doing a > > small change with a bump while there was an open PR with the same > > bump. The led to trivial conflicts and rebase needs for PRs. > > AUTORELEASE has tackled that nicely. > +1 > Awww. I liked the idea of AUTORELEASE. I'll admit I was a bit hesitant about the git black-magic underneath, but as long as I did not have to manage it, I was fine with it. We can try to find some other way (that git history) to manage AUTORELEASE I'm also fine to drop it, if it simplifies other things. > ___ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
On 06. 11. 22 17:15, Paul Spooren wrote: Hi, While I initially thought that $(AUTORELEASE) would be a nice feature to avoid the standard review comment “Please bump the PKG_RELEASE”, it turned into a massive increase of bandwidth usage: Every checkout of openwrt.git and package feeds needs to be a full clone instead of a shallow one to have all commits in place. What happened with the switch to CDN? There is also a possibility that users can use GitHub as a mirror to avoid increased bandwidth usage on your server. BTW: The $(AUTORELEASE) feature is nicely implemented and should not be dropped. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
On 06. 11. 22 21:22, Hannu Nyman wrote: Paul Spooren kirjoitti 6.11.2022 klo 18.15: While I initially thought that $(AUTORELEASE) would be a nice feature to avoid the standard review comment “Please bump the PKG_RELEASE”, it turned into a massive increase of bandwidth usage: Every checkout of openwrt.git and package feeds needs to be a full clone instead of a shallow one to have all commits in place. I’m suggesting dropping that feature and instead implement a (GitHub) CI workflow which fails if the commit did not bump the PKG_RELEASE. It wasn’t a fully thought through idea and I think it’s time to revert it - sorry for the inconvenience. Other thoughts? The AUTORELEASE has been a nice feature from the package PR maintenance perspective. Earlier there was constant trouble with concurrent PRs for the same package having the same PKG_RELEASE bump, or the maintainer doing a small change with a bump while there was an open PR with the same bump. The led to trivial conflicts and rebase needs for PRs. AUTORELEASE has tackled that nicely. +1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
Paul Spooren kirjoitti 6.11.2022 klo 18.15: While I initially thought that $(AUTORELEASE) would be a nice feature to avoid the standard review comment “Please bump the PKG_RELEASE”, it turned into a massive increase of bandwidth usage: Every checkout of openwrt.git and package feeds needs to be a full clone instead of a shallow one to have all commits in place. I’m suggesting dropping that feature and instead implement a (GitHub) CI workflow which fails if the commit did not bump the PKG_RELEASE. It wasn’t a fully thought through idea and I think it’s time to revert it - sorry for the inconvenience. Other thoughts? The AUTORELEASE has been a nice feature from the package PR maintenance perspective. Earlier there was constant trouble with concurrent PRs for the same package having the same PKG_RELEASE bump, or the maintainer doing a small change with a bump while there was an open PR with the same bump. The led to trivial conflicts and rebase needs for PRs. AUTORELEASE has tackled that nicely. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [RFC] dropping of $(AUTORELEASE) feature
> Le 6 nov. 2022 à 17:15, Paul Spooren a écrit : > > Hi, > > While I initially thought that $(AUTORELEASE) would be a nice feature to > avoid the standard review comment “Please bump the PKG_RELEASE”, it turned > into a massive increase of bandwidth usage: Every checkout of openwrt.git and > package feeds needs to be a full clone instead of a shallow one to have all > commits in place. > > I’m suggesting dropping that feature and instead implement a (GitHub) CI > workflow which fails if the commit did not bump the PKG_RELEASE. +1 Cheers T ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[RFC] dropping of $(AUTORELEASE) feature
Hi, While I initially thought that $(AUTORELEASE) would be a nice feature to avoid the standard review comment “Please bump the PKG_RELEASE”, it turned into a massive increase of bandwidth usage: Every checkout of openwrt.git and package feeds needs to be a full clone instead of a shallow one to have all commits in place. I’m suggesting dropping that feature and instead implement a (GitHub) CI workflow which fails if the commit did not bump the PKG_RELEASE. It wasn’t a fully thought through idea and I think it’s time to revert it - sorry for the inconvenience. Other thoughts? Best, Paul ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
