Re: [Fedora-packaging] Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-03-03 15:14, clime a écrit :
On Tue, 3 Mar 2020 at 09:22, Nicolas Mailhot
wrote:
Le 2020-03-02 14:45, clime a écrit :
> On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
> wrote:
>>
>> Le 2020-03-01 02:31, clime a écrit :
>> > On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
>> > wrote:
>> >>
>> >> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>>
>> >> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
>> >> detached file state means that fedpkg local (or checking out git state
>> >> and building in mock or copr or OBS or via plain rpmbuild -bs) will
>> >> give you the same result as launching fedpkg build.
>> >
>> > Well, I believe it doesn't. You run:
>> >
>> > 1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
>> > you haven't built that package before)
>> > 2) fedpkg build
>>
>> At that point state is undefined till the build succeeds or not. If
>> the
>> build succeeds, the buildsystem will write back a new state. Let’s
>> assume it succeeds.
>
> It's undefined if you add more elements to the equation than necessary
> (i.e. build system), otherwise it would be well-defined in this case.
>
>>
>> > 3) vim .spec and do some change in %description
>> > 4) fedpkg commit -m "description improvement"
>> > 5) fedpkg local -> produces -1.0-1.fc32
>> > 6) fedpkg push -> error because build system pushed meanwhile
>>
>> Yes, here the packager notices something else has been going on, and
>> he
>> needs to merge or rebase. He’d have the same effect if another
>> packager
>> had been doing stuff on the package, or a mass rebuild had been going
>> on. That’s the distributed decentralized aspect of git, except here
>> the
>> packager collided with himself by starting two work threads in
>> parallel
>> (one buildsys-side, one local).
>
> The problem is that you launched some process and you need to wait
> until it finishes, normally you don't need to anything like that when
> working with git.
But fedpkg build is a release to production process. It’s not a dev
staging process. In release processes, actually doing the release is
not
an inconvenient optional check.
Well, you could have a situation when somebody wants to immediately
continue working after making a release. Probably rare but could
happen and if build time of a package is long (libreOffice/firefox),
this could hit you as an inconvenience. Or do you want to push-back
immediately after srpm build when you don't yet know whether the build
will succeed? That wasn't clear to me before.
Either you lock the centralized branch while the build is ongoing or you
fail the build if someone pushes to the centralized branch before the
end of the build.
That does not stop packagers from preparing the next stage in a local
branch, only changing the state of the centralized branch at the same
time the build process is changing it (two conflicting release state
changes, option 1: buildsystem wins, option 2, packager wins, can’t have
an heisenstate where things are both released and not released).
I am not suggesting to use raw git commit messages for the changelog
but instead content of annotated tags which can be initially
prepopulated by commit messages but can be then edited to any extent
and even the way the edit window for annotated tag is prepopulated
might be configurable (to .e.g. load content from a file in which case
you would probably skip the editing part). And I see the annotated
tags as the actual releases.
The only actual release is the package built in koji. So, no matter what
mechanism you use git side:
– you must represent the real state of package builds and not a git-only
fiction
→ that means a form of buildsys write-back because builds can and will
fail
– you must produce something packagers can also build in mock or copr or
OBS or whatever
→ that means checkout-ing changelog data in a form that can be read by
rpmbuild -bs
→ ie a file not git metadata
→ with the associated conflict risk if multiple changes occur between
checkouts
You can not avoid a buildsys merge-back when doing production builds.
Merge backs by build system can be avoided, however. Why do you think
they can't be avoided?
Because they’re the real state rpm changelogs records. Because that’s
what people use rpm changelogs for. Something broke, what is the
affected packaged, what where the released package states since last
time it worked.
There is the case of mass rebuilds but this is pretty much a one-shot
event
Mass rebuilds are not an exception, they’re becoming the norm. Every SIG
that deals with modern software released as a huge number of interlinked
components needs to perform SIG-level mass rebuilds all the time
(directly in rawhide, in a side tag, whatever). koshei can now autobuild
all dependent packages when one of their requirements changed. That’s
where the project is going and has been going for several years.
All the modularity efforts in the past
Re: [Fedora-packaging] Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 3 Mar 2020 at 09:38, Nicolas Mailhot
wrote:
>
> Le 2020-03-03 07:03, clime a écrit :
>
> > Actually, that wouldn't work because prefix needs to be static, not
> > dependent on rpm macros
>
> For myself, I would oppose any rpm release process that would not take
> core rpm mecanisms like macros into account.
Being independent of rpm has one huge potential advantage. You can
apply the same approach that you apply for rpm spec files to other
kinds of spec files.
For example the fedora & containers mailing list contains a thread,
when person needs to parametrize FROM clause by branch name to
correctly reference base image:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/65GHZM4PODSO36C66ONZCYXVIN7JIXMK/
The approach I am suggesting would help there and it could help on
other places too.
>
> Recording builds in changelogs without checking they actually happened
> is bad engineering.
It's not about recording builds but instead about recording releases
which I see as something that takes place already in Git.
>
> Simulating rpm behaviour without performing actual spec evaluation in
> rpm, is also bad engineering. Especially when you know your simulation
> is horribly simplistic and approximative.
I am not trying to simulate rpm behavior. I am just trying to
introduce macros that are rendered verbatim into the spec file inside
srpm so those srpms can be rebuilt at any time when needed. rpm does
not have this feature atm. If we should put it into rpm, it would be
already third kind of rpm macro but even if it should go into rpm at
some point, it is better to have the approach heavily tested by
production use first because here it doesn't really add much
complexity. And as I mentioned, independence of rpm means you can use
it for non-rpm projects too.
It's also not simplistic: what you see in {{{ }}} tags is interpreted
by bash, it can be multi-line and you can hence do anything there you
can do with rpm macros. The only problem is, people already have some
automation in rpm macros that doesn't require the git context, instead
just e.g. the tarball content and this functionality would need to be
rewritten by using {{{ }}} if it should play together with the
git-based release generation. This cannot be avoided as much as I
would like to. If we try to avoid it, we will get a weird hybrid that
doesn't work.
It's also not approximative. You can define your own logic for
generating release. There should be some pre-made recommended macros
but in the end you are not limited by them.
>
> “Who cares if results match most of the time” is terrible workload
> optimization. You’ll make packagers waste far more time fixing the cases
> where automation guessed wrong, than you will win when it guesses right.
> Basic 80/20 rule, the 20% hard cases cost more than the 80% easy cases.
> Taking care of the 80% easy cases while making the 20% hard cases harder
> (due to automation mistakes) is not a good deal at all.
>
> Please work on approaches which are reliable by default. Reliability is
> hard even when you aim for it. When you don’t, it’s not attainable at
> all.
I think the approach I am suggesting is very reliable.
As much as I enjoy this conversation, I will need to actually try to
make something happen. Forgive me if I don't go into long disputes
hereafter. If you, however, make a detailed document summarizing the
infra changes needed for your approach I will try to comment on the
individual aspects of it and also respond with the document with the
changes needed for what I am suggesting.
Best regards!
clime
>
> Reagrds,
>
> --
> Nicolas Mailhot
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/packag...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Fedora-packaging] Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 3 Mar 2020 at 09:22, Nicolas Mailhot
wrote:
>
> Le 2020-03-02 14:45, clime a écrit :
> > On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
> > wrote:
> >>
> >> Le 2020-03-01 02:31, clime a écrit :
> >> > On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
> >> > wrote:
> >> >>
> >> >> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
> >>
> >> >> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
> >> >> detached file state means that fedpkg local (or checking out git state
> >> >> and building in mock or copr or OBS or via plain rpmbuild -bs) will
> >> >> give you the same result as launching fedpkg build.
> >> >
> >> > Well, I believe it doesn't. You run:
> >> >
> >> > 1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
> >> > you haven't built that package before)
> >> > 2) fedpkg build
> >>
> >> At that point state is undefined till the build succeeds or not. If
> >> the
> >> build succeeds, the buildsystem will write back a new state. Let’s
> >> assume it succeeds.
> >
> > It's undefined if you add more elements to the equation than necessary
> > (i.e. build system), otherwise it would be well-defined in this case.
> >
> >>
> >> > 3) vim .spec and do some change in %description
> >> > 4) fedpkg commit -m "description improvement"
> >> > 5) fedpkg local -> produces -1.0-1.fc32
> >> > 6) fedpkg push -> error because build system pushed meanwhile
> >>
> >> Yes, here the packager notices something else has been going on, and
> >> he
> >> needs to merge or rebase. He’d have the same effect if another
> >> packager
> >> had been doing stuff on the package, or a mass rebuild had been going
> >> on. That’s the distributed decentralized aspect of git, except here
> >> the
> >> packager collided with himself by starting two work threads in
> >> parallel
> >> (one buildsys-side, one local).
> >
> > The problem is that you launched some process and you need to wait
> > until it finishes, normally you don't need to anything like that when
> > working with git.
>
> But fedpkg build is a release to production process. It’s not a dev
> staging process. In release processes, actually doing the release is not
> an inconvenient optional check.
Well, you could have a situation when somebody wants to immediately
continue working after making a release. Probably rare but could
happen and if build time of a package is long (libreOffice/firefox),
this could hit you as an inconvenience. Or do you want to push-back
immediately after srpm build when you don't yet know whether the build
will succeed? That wasn't clear to me before.
>
> You're asked to autobump a “Release” field. You’re asked to automate a
> changelog which is effectively a release changelog, not dev changelog
> (the dev changelog lives in git, people want a good changelog in rpm
> proper to track release not git history).
I am not suggesting to use raw git commit messages for the changelog
but instead content of annotated tags which can be initially
prepopulated by commit messages but can be then edited to any extent
and even the way the edit window for annotated tag is prepopulated
might be configurable (to .e.g. load content from a file in which case
you would probably skip the editing part). And I see the annotated
tags as the actual releases.
From https://git-scm.com/docs/git-tag:
Annotated tags are meant for release while lightweight tags are meant
for private or temporary object labels.
They were designed to represent releases and because we are using git
I don't see why we should avoid using annotated tags for releases.
On src.fp.o, there is also already support for this: e.g.
https://src.fedoraproject.org/rpms/2ping/releases
The release needs building (koji), and distribution (bodhi + mirrors)
but that's stuff that needs to be done with the release. Scratch
builds can be used to make sure that the package builds and in the
worst case, a new release needs to be done that accounts for changed
build environment. But I don't think we need to throw the whole
concept of annotated tags = releases because of it.
Of course, everything here can be made opt-in. As an approach somebody
likes and somebody doesn't.
I think this is where our approaches really differ, you see release as
something that happens somewhere else than in git, but I see the
release as something that happens in git. And afterwards we need to
get that released content to users.
I think we will never be able to reconciliate our views because they
are different from the very base. In the end, what matters is
implementation complexity and ease of use. I can present to you the
things that need to be done for this to work in a detailed way
(although it would take a bit of time to put into a document) but I
also know it is a doable and it is actually fairly simple when not
accounting for time spending on pull requests and getting them
accepted. Your approach seems to be intuitively difficult to
implement, either locking implementation (i am
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-03-03 07:03, clime a écrit : Actually, that wouldn't work because prefix needs to be static, not dependent on rpm macros For myself, I would oppose any rpm release process that would not take core rpm mecanisms like macros into account. Recording builds in changelogs without checking they actually happened is bad engineering. Simulating rpm behaviour without performing actual spec evaluation in rpm, is also bad engineering. Especially when you know your simulation is horribly simplistic and approximative. “Who cares if results match most of the time” is terrible workload optimization. You’ll make packagers waste far more time fixing the cases where automation guessed wrong, than you will win when it guesses right. Basic 80/20 rule, the 20% hard cases cost more than the 80% easy cases. Taking care of the 80% easy cases while making the 20% hard cases harder (due to automation mistakes) is not a good deal at all. Please work on approaches which are reliable by default. Reliability is hard even when you aim for it. When you don’t, it’s not attainable at all. Reagrds, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-03-02 14:45, clime a écrit :
On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
wrote:
Le 2020-03-01 02:31, clime a écrit :
> On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
> wrote:
>>
>> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
>> detached file state means that fedpkg local (or checking out git state
>> and building in mock or copr or OBS or via plain rpmbuild -bs) will
>> give you the same result as launching fedpkg build.
>
> Well, I believe it doesn't. You run:
>
> 1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
> you haven't built that package before)
> 2) fedpkg build
At that point state is undefined till the build succeeds or not. If
the
build succeeds, the buildsystem will write back a new state. Let’s
assume it succeeds.
It's undefined if you add more elements to the equation than necessary
(i.e. build system), otherwise it would be well-defined in this case.
> 3) vim .spec and do some change in %description
> 4) fedpkg commit -m "description improvement"
> 5) fedpkg local -> produces -1.0-1.fc32
> 6) fedpkg push -> error because build system pushed meanwhile
Yes, here the packager notices something else has been going on, and
he
needs to merge or rebase. He’d have the same effect if another
packager
had been doing stuff on the package, or a mass rebuild had been going
on. That’s the distributed decentralized aspect of git, except here
the
packager collided with himself by starting two work threads in
parallel
(one buildsys-side, one local).
The problem is that you launched some process and you need to wait
until it finishes, normally you don't need to anything like that when
working with git.
But fedpkg build is a release to production process. It’s not a dev
staging process. In release processes, actually doing the release is not
an inconvenient optional check.
You're asked to autobump a “Release” field. You’re asked to automate a
changelog which is effectively a release changelog, not dev changelog
(the dev changelog lives in git, people want a good changelog in rpm
proper to track release not git history).
The *single* *only* reason good release numbers and good changelog
matter is because the result will be released to third parties. koshei,
or scratch builds, or whatever, do not care about those.
A normal git flow would lock the branch at release time to let the
release manager do his releasing (merge window closed in Linus’s terms).
That’s option 1 I gave you.
Alternatively, you can use option 2, the “I feel lucky” approach, let
everything open, and abort the release if merge-back failed. A packager
that changes his package before the production build is finished will
probably want it canceled anyway (what is the point of risking breaking
builds of other packages by pushing an unfinished package to the shared
koji buildroot)?
Either option does not stop the packager to do anything he wants in his
local branches. They just require him to merge/rebase when syncing with
the centralized master Fedora state. That’s a core element of git
architecture, be it in Fedora or elsewhere.
You can not avoid a buildsys merge-back when doing production builds.
The merge-back can be explicit and automatic (as I proposed), or you can
rely on informal meatware for it, but it will exist. Since the whole
point of the request is to lower dependence on meatware, why on hell
would you want to keep meatware as part of the whole process?
The only “cost” for the packager is to wait for his build to finish
before continuing to change the package (or merge/rebase). That’s not an
horrific cost. No one forced the packager to start a build before he was
ready. No one forced the packager to perform a production build instead
of a scratch build if he just wanted to check partial changes. If shit
happens, and he realizes before the build end something is not ok,
aborting the build on failed merge back is helping the packager.
Or am I missing something? “normally you don't need to anything like
that when working with git” is not a clear technical point.
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 3 Mar 2020 at 00:28, clime wrote:
>
> Ad. Zbyszek:
>
> > What about doing --.?
> > This means that upgrade path not affected by the number of commits or
> > builds in the older release.
>
> I was thinking how to properly implement this into rpkg-util and then
> finally, it clicked for me.
>
> First, I added the prefix parameter for git_release macro (below
> git_dir_release is used instead, which is the recommended form). Hence
> now, one can specify:
>
> Release: {{{ git_dir_release prefix="0%{?dist}" }}}
>
> which would produce release strings like:
>
> 0.fc32.1
> 0.fc32.2
> 0.fc32.3
Actually, that wouldn't work because prefix needs to be static, not
dependent on rpm macros (we would be searching for tags that contain
literally '0%{?dist}' in the last release part after the dash when
generating the current release based on past tag names). Only the
below versions that depend just on Git would work. I was hoping I can
get a fallback for cases like Nicolas or the ruby packages have but
that doesn't seem to be possible (at least to me at the moment).
>
> for each tag in f32 branch. The leading zero is not used here for
> anything but without it, we would get NVRs like somepkg-1.0-.fc32.1 -
> i.e. dash would be followed by immediate dot, which is not actually
> invalid but it is strange.
>
> Then it came to me that instead of %{dist} we can simply use branch
> name and hence (finally) drop "c" from .fcXY:
>
> Release: {{{ git_dir_release prefix="$GIT_BRANCH" }}}
>
> ("$GIT_BRANCH" is a macro variable that gives you name of the
> currently checked out branch)
>
> This will not work for cases when people put some special stuff into
> %{dist} like Nicolas showed:
> https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
> but it would work for simple cases and fallback would be possible.
See above, regrettably.
>
> Finally, one can just alias `git_dir_release prefix="$GIT_BRANCH"`
> with `git_dir_release_branched`
> (https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_464)
> and hence get the following:
>
> Release: {{{ git_dir_release_branched }}}
>
> which will be bumping release with respect to the latest tag on the
> current branch as well as the commits stacked on top of that tag (it
> will be also bumping release for uncommitted work if your working tree
> is dirty but i don't want to show it here now because NVRs are then
> quite long).
>
> I've prepared a test project for this new macro:
> https://pagure.io/hello_rpkg_release. You need the latest "rpkg" and
> "rpkg-macros" from
> https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ for it to
> work. But I also just dumped the results here:
> https://pagure.io/hello_rpkg_release/tree/result . You can see what
> NVRs are generated (by `rpkg nvr` command) at each particular point
> for the given branch (i.e. hello_rpkg_release-1.14.0-f32.2 for the
> second tagged commit in f32 branch) -- they are written down after the
> "-->" arrow. I mention there three NVRs in total:
>
> hello_rpkg_release-1.14.0-master.1 (the first tagged commit master branch
> hello_rpkg_release-1.14.0-f32.2 (the second tagged commit in the f32 branch)
> hello_rpkg_release-1.14.0-f31.1.git.1.34684da9 (untagged commit on
> top of the first tagged commit in the f31 branch)
>
> Any feedback welcome.
>
> On Mon, 2 Mar 2020 at 14:51, Neal Gompa wrote:
> >
> > On Mon, Mar 2, 2020 at 8:46 AM clime wrote:
> > >
> > > On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
> > > wrote:
> > > >
> > > > If you don’t keep things decentralized you’ll be in a word of pain when
> > > > the scm or buildsys needs to be changed for another implementation (not
> > > > to mention, that’s not a good way to collaborate with other distros).
> > > > That will happen eventually. Web apps are not eternal.
> > >
> > > Full decentralization likely means that everyone has its own git repo
> > > and we can only sync by mailing list. I think src.fp.o is a good point
> > > where things can be done and where we can agree on what the packages
> > > that Fedora produce are (meaning we need a canonical source of package
> > > sources, otherwise it would be more complex to put a distribution
> > > together).
> > >
> >
> > This is not true. Pagure accepts PRs from arbitrary Git servers just
> > fine via the remote PR feature, so we do support decentralized
> > workflows without resorting to sending patches via email or Bugzilla.
> >
> >
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Ad. Zbyszek:
> What about doing --.?
> This means that upgrade path not affected by the number of commits or
> builds in the older release.
I was thinking how to properly implement this into rpkg-util and then
finally, it clicked for me.
First, I added the prefix parameter for git_release macro (below
git_dir_release is used instead, which is the recommended form). Hence
now, one can specify:
Release: {{{ git_dir_release prefix="0%{?dist}" }}}
which would produce release strings like:
0.fc32.1
0.fc32.2
0.fc32.3
for each tag in f32 branch. The leading zero is not used here for
anything but without it, we would get NVRs like somepkg-1.0-.fc32.1 -
i.e. dash would be followed by immediate dot, which is not actually
invalid but it is strange.
Then it came to me that instead of %{dist} we can simply use branch
name and hence (finally) drop "c" from .fcXY:
Release: {{{ git_dir_release prefix="$GIT_BRANCH" }}}
("$GIT_BRANCH" is a macro variable that gives you name of the
currently checked out branch)
This will not work for cases when people put some special stuff into
%{dist} like Nicolas showed:
https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
but it would work for simple cases and fallback would be possible.
Finally, one can just alias `git_dir_release prefix="$GIT_BRANCH"`
with `git_dir_release_branched`
(https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_464)
and hence get the following:
Release: {{{ git_dir_release_branched }}}
which will be bumping release with respect to the latest tag on the
current branch as well as the commits stacked on top of that tag (it
will be also bumping release for uncommitted work if your working tree
is dirty but i don't want to show it here now because NVRs are then
quite long).
I've prepared a test project for this new macro:
https://pagure.io/hello_rpkg_release. You need the latest "rpkg" and
"rpkg-macros" from
https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ for it to
work. But I also just dumped the results here:
https://pagure.io/hello_rpkg_release/tree/result . You can see what
NVRs are generated (by `rpkg nvr` command) at each particular point
for the given branch (i.e. hello_rpkg_release-1.14.0-f32.2 for the
second tagged commit in f32 branch) -- they are written down after the
"-->" arrow. I mention there three NVRs in total:
hello_rpkg_release-1.14.0-master.1 (the first tagged commit master branch
hello_rpkg_release-1.14.0-f32.2 (the second tagged commit in the f32 branch)
hello_rpkg_release-1.14.0-f31.1.git.1.34684da9 (untagged commit on
top of the first tagged commit in the f31 branch)
Any feedback welcome.
On Mon, 2 Mar 2020 at 14:51, Neal Gompa wrote:
>
> On Mon, Mar 2, 2020 at 8:46 AM clime wrote:
> >
> > On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
> > wrote:
> > >
> > > If you don’t keep things decentralized you’ll be in a word of pain when
> > > the scm or buildsys needs to be changed for another implementation (not
> > > to mention, that’s not a good way to collaborate with other distros).
> > > That will happen eventually. Web apps are not eternal.
> >
> > Full decentralization likely means that everyone has its own git repo
> > and we can only sync by mailing list. I think src.fp.o is a good point
> > where things can be done and where we can agree on what the packages
> > that Fedora produce are (meaning we need a canonical source of package
> > sources, otherwise it would be more complex to put a distribution
> > together).
> >
>
> This is not true. Pagure accepts PRs from arbitrary Git servers just
> fine via the remote PR feature, so we do support decentralized
> workflows without resorting to sending patches via email or Bugzilla.
>
>
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Mar 2, 2020 at 8:46 AM clime wrote: > > On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel > wrote: > > > > If you don’t keep things decentralized you’ll be in a word of pain when > > the scm or buildsys needs to be changed for another implementation (not > > to mention, that’s not a good way to collaborate with other distros). > > That will happen eventually. Web apps are not eternal. > > Full decentralization likely means that everyone has its own git repo > and we can only sync by mailing list. I think src.fp.o is a good point > where things can be done and where we can agree on what the packages > that Fedora produce are (meaning we need a canonical source of package > sources, otherwise it would be more complex to put a distribution > together). > This is not true. Pagure accepts PRs from arbitrary Git servers just fine via the remote PR feature, so we do support decentralized workflows without resorting to sending patches via email or Bugzilla. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
wrote:
>
> Le 2020-03-01 02:31, clime a écrit :
> > On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
> > wrote:
> >>
> >> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>
> >> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
> >> detached file state means that fedpkg local (or checking out git state
> >> and building in mock or copr or OBS or via plain rpmbuild -bs) will
> >> give you the same result as launching fedpkg build.
> >
> > Well, I believe it doesn't. You run:
> >
> > 1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
> > you haven't built that package before)
> > 2) fedpkg build
>
> At that point state is undefined till the build succeeds or not. If the
> build succeeds, the buildsystem will write back a new state. Let’s
> assume it succeeds.
It's undefined if you add more elements to the equation than necessary
(i.e. build system), otherwise it would be well-defined in this case.
>
> > 3) vim .spec and do some change in %description
> > 4) fedpkg commit -m "description improvement"
> > 5) fedpkg local -> produces -1.0-1.fc32
> > 6) fedpkg push -> error because build system pushed meanwhile
>
> Yes, here the packager notices something else has been going on, and he
> needs to merge or rebase. He’d have the same effect if another packager
> had been doing stuff on the package, or a mass rebuild had been going
> on. That’s the distributed decentralized aspect of git, except here the
> packager collided with himself by starting two work threads in parallel
> (one buildsys-side, one local).
The problem is that you launched some process and you need to wait
until it finishes, normally you don't need to anything like that when
working with git. The src.fp.o is a synchronization point of all
developers. Now you are adding there also a build process with which
all the developers need to synchronize. I think it is inconvenient.
>
> > 7) fedpkg pull --rebase (to quickly fix the error)
> > 8) fedpkg build -> builds -1.0-2.fc32
> >
> > To get the correct NVR for the package being built by `fedpkg local`
> > you need to wait for some external process to finish first so that you
> > can pull the new state. Only then you get correct value again.
>
> Sure, that‘s the inconvenient of state: it needs syncing. And the whole
> auto-bump thing intrinsically depends on state not provided manually by
> human beings.
>
> However putting state in git would be no better, because you’d end up
> assuming a build exists before it does, in fact, build (and builds do
> fail because packagers are imperfect humans).
The only assumption is that a certain build will produce certain NEVRA
if it succeeds.
>
> And putting state in buildsys would also not be any better because
> successful production builds end up in changelogs and you want
> changelogs to be tracked.
>
> So, no matter how you re-arange things, you’ll always have a buildsys
> sync-back somewhere (you have it today when humans change their
> changelogs after a failed build; automating that makes the sync-back
> explicit).
Scratch builds can help here a lot.
>
> >> For fedpkg verrel you'll probably want to output a last (saved in
> >> detached file) and next line (probably factorizable by externalizing
> >> the dynrel bump logic in a separate command). That’s more honest
> >> anyway, next may happen or not, when you launch fedpkg verrel, it’s
> >> mere potential (the next commit may bump version and invalidate your
> >> future build).
> >
> > There is the same problem as above with `fedpkg local`, the command
> > won't be giving you correct values at all times.
>
> There is no correct value to give before production build succeeded or
> not. When fedpkg local gives you something today it is lying (just try
> it just before someone mass rebuilds, and then try to commit after this
> mass rebuild: that will fail and you’ll have to merge/rebase).
It gives you something you can rely on if push succeeds.
>
> You’re assuming a simple universe where you’re the only one doing things
> with a package. That’s not a true assumption, many packages have shared
> ownership and maintenance (to say nothing of proven-packagers and releng
> doing things on top of them).
There is only one person doing the actual build for a given release
while multiple persons can participate in development.
>
> > It doesn't need to be the consequence of "automating changelog". It's
> > only a consequence in certain implementations. If the changelog is
> > derived purely from the git state, there aren't these problems
> > anymore.
>
> Sure there are. You're assuming anything committed to git builds before
> actually trying to build it. And expecting manual packager git fixup
> when that’s not the case. Just like in the sync-back state except with
> no tooling enforcement and help.
Who will be fixing the bugs it if not human?
> >> Putting state in detached srpm source files has the following
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, 2 Mar 2020 at 11:44, Nils Philippsen wrote: > > On Fri, 2020-02-28 at 15:45 +0100, clime wrote: > > I thought the main reason not to combine update in the changelog file > > with > > code commits is to avoid conflicts when cherry picking as you > > described. > > > > I.e. i do minor update specifically in f32 and generate changelog > > file > > in the same commit. > > Then I want to do normal release update for all fedora branches. I > > start with master (as usual) > > and add e.g. add a new patch and generate the changelog file because > > i > > would like to add more info, then commit. > > Now I cannot cherry-pick that commit into f32 without conflict. > > > > In this case we wouldn't achieve one the targets of this proposal > > (afaik) => getting rid of merge conflicts in changelog and release - > > this is cherry-picking but still it would be nice not to have > > conflicts there. > > This target isn't in the document i think but i thought this is one > > of > > the goals. > > Leaving aside the advantages or disadvantages of a detached changelog > (vs. in-spec vs. "everything from the commits"), our aim is to reduce > the potential for conflicts between concurrent PRs or when cherry- > picking between branches. If what we end up with can't avoid them > completely for (what should be) corner cases, like when you have to > manually fuzz with the generated changelog, that may not be ideal, but > it's still a considerable improvement over the present situation. Right, but if we can eliminate the problem completely why not do it? clime > > Nils > -- > Nils Philippsen"Those who would give up Essential Liberty to > Software Engineer purchase a little Temporary Safety, deserve neither > Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 > PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D > old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-03-01 02:31, clime a écrit :
On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
wrote:
Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
detached file state means that fedpkg local (or checking out git state
and building in mock or copr or OBS or via plain rpmbuild -bs) will
give you the same result as launching fedpkg build.
Well, I believe it doesn't. You run:
1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
you haven't built that package before)
2) fedpkg build
At that point state is undefined till the build succeeds or not. If the
build succeeds, the buildsystem will write back a new state. Let’s
assume it succeeds.
3) vim .spec and do some change in %description
4) fedpkg commit -m "description improvement"
5) fedpkg local -> produces -1.0-1.fc32
6) fedpkg push -> error because build system pushed meanwhile
Yes, here the packager notices something else has been going on, and he
needs to merge or rebase. He’d have the same effect if another packager
had been doing stuff on the package, or a mass rebuild had been going
on. That’s the distributed decentralized aspect of git, except here the
packager collided with himself by starting two work threads in parallel
(one buildsys-side, one local).
7) fedpkg pull --rebase (to quickly fix the error)
8) fedpkg build -> builds -1.0-2.fc32
To get the correct NVR for the package being built by `fedpkg local`
you need to wait for some external process to finish first so that you
can pull the new state. Only then you get correct value again.
Sure, that‘s the inconvenient of state: it needs syncing. And the whole
auto-bump thing intrinsically depends on state not provided manually by
human beings.
However putting state in git would be no better, because you’d end up
assuming a build exists before it does, in fact, build (and builds do
fail because packagers are imperfect humans).
And putting state in buildsys would also not be any better because
successful production builds end up in changelogs and you want
changelogs to be tracked.
So, no matter how you re-arange things, you’ll always have a buildsys
sync-back somewhere (you have it today when humans change their
changelogs after a failed build; automating that makes the sync-back
explicit).
For fedpkg verrel you'll probably want to output a last (saved in
detached file) and next line (probably factorizable by externalizing
the dynrel bump logic in a separate command). That’s more honest
anyway, next may happen or not, when you launch fedpkg verrel, it’s
mere potential (the next commit may bump version and invalidate your
future build).
There is the same problem as above with `fedpkg local`, the command
won't be giving you correct values at all times.
There is no correct value to give before production build succeeded or
not. When fedpkg local gives you something today it is lying (just try
it just before someone mass rebuilds, and then try to commit after this
mass rebuild: that will fail and you’ll have to merge/rebase).
You’re assuming a simple universe where you’re the only one doing things
with a package. That’s not a true assumption, many packages have shared
ownership and maintenance (to say nothing of proven-packagers and releng
doing things on top of them).
It doesn't need to be the consequence of "automating changelog". It's
only a consequence in certain implementations. If the changelog is
derived purely from the git state, there aren't these problems
anymore.
Sure there are. You're assuming anything committed to git builds before
actually trying to build it. And expecting manual packager git fixup
when that’s not the case. Just like in the sync-back state except with
no tooling enforcement and help.
Putting state in detached srpm source files has the following super-
duper property. You can import the srpm or scm checkout between
systems, and they’ll just pick up from the point the previous system
left things, without needing a full scm import.
That's nice if there is always just a single provider of the given
package in our ecosystem. In case of copr, you can fork a certain repo
and continue building there so you will be getting bumped numbers on
top of the work done in the original repo. But once the original repo
starts producing some builds as well you will be getting that package
in two versions in two different coprs and while the original repo
might have done less builds than the forked one since the fork
happened, it can contain the package with more features because the
forked repo was simply playing around.
You will never get official Fedora git releases changed based on the
status of some copr builds. That takes a deliberate package admin action
(and that works in the proposal, just fedpkg import the last srpm built
in copr, that syncs repos).
If you want things to work on the other direction, copr will need to
read
Re: Ideas and proposal for removing changelog and release fields from spec file
On Fri, 2020-02-28 at 15:45 +0100, clime wrote: > I thought the main reason not to combine update in the changelog file > with > code commits is to avoid conflicts when cherry picking as you > described. > > I.e. i do minor update specifically in f32 and generate changelog > file > in the same commit. > Then I want to do normal release update for all fedora branches. I > start with master (as usual) > and add e.g. add a new patch and generate the changelog file because > i > would like to add more info, then commit. > Now I cannot cherry-pick that commit into f32 without conflict. > > In this case we wouldn't achieve one the targets of this proposal > (afaik) => getting rid of merge conflicts in changelog and release - > this is cherry-picking but still it would be nice not to have > conflicts there. > This target isn't in the document i think but i thought this is one > of > the goals. Leaving aside the advantages or disadvantages of a detached changelog (vs. in-spec vs. "everything from the commits"), our aim is to reduce the potential for conflicts between concurrent PRs or when cherry- picking between branches. If what we end up with can't avoid them completely for (what should be) corner cases, like when you have to manually fuzz with the generated changelog, that may not be ideal, but it's still a considerable improvement over the present situation. Nils -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, Feb 29, 2020 at 10:40:51PM +0100, Nicolas Mailhot via devel wrote: > Le samedi 29 février 2020 à 20:57 +0100, clime a écrit : > > If I, as a user, see e.g. python3-alembic-1.1.0-13 in f31 and then I > > will see python3-alembic-1.1.0-13 in f32 (this time with .fc32 > > disttag), I am going to assume nothing has changed for that package. > > And you’ll be wrong, because we do branch. Succinct, but true. To give a concrete example: foo-99-1 is in current rawhide and F32 branches. In rawhide, the mass rebuild happens, but so we get foo-99-2. But this is a noarch package and foo-99-1.fc32 and foo-99-2.fc33 are effectively identical, despite having different NVr (where "r" is just the number part without the dist tag). Now a bug reported that only affects F32 or the maintainer rebuilds in F32 only. A rebuild with the fix is labelled foo-99-2.fc32. Now the packages are functionally different, despite having same NVr. (And this is all correct packaging, according to update guidelines. The upgrade path is preserved at all times, and that is the only thing that is required.) So yeah, I stand by my original statement that > The numbers in different branches cannot > be meaningfully compared. Those numbers only make sense in the context > of a specific branch, so they should be ordered after . Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 22:42, Nicolas Mailhot via devel
wrote:
>
> Le samedi 29 février 2020 à 20:57 +0100, clime a écrit :
> > On Sat, 29 Feb 2020 at 16:24, Nicolas Mailhot via devel
> > wrote:
> > > Le samedi 29 février 2020 à 15:12 +0100, clime a écrit :
> > > > On Sat, 29 Feb 2020 at 14:47, Nicolas Mailhot via devel
> > > > wrote:
> > > > > Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
> > > > > > Does ENVR without %{dist} means something with respect to the
> > > > > > content
> > > > > > from
> > > > > > which the package was built or with respect to features that
> > > > > > it
> > > > > > offers for the given distribution version?
> > > > >
> > > > > You need to evaluate evr with a neutral dist value to decide to
> > > > > bump or
> > > > > not the auto-dynamic part of release or not. Because the whole
> > > > > point of
> > > > > the auto-dynamic part of release would be to track rebuilds
> > > > > from
> > > > > the
> > > > > same spec, all othert parts of EVR being equal
> > > > >
> > > > > Changelog-side and package build side you need the full EVR
> > > > > without
> > > > > neutralization
> > > >
> > > > Thank you very much Nicolas but you reacted to a question which
> > > > was
> > > > actually unrelated to your proposal. That particular question
> > > > about
> > > > the meaning of ENVR when you strip the distribution tag (i.e.
> > > > .fc32
> > > > or
> > > > .el7) was intended to be generic, i.e. i want to know how if
> > > > e.g.
> > > > python3-alembic-1.1.0-1 has any meaning alone without, for
> > > > example,
> > > > .fc31 appended to it (or if it should have any meaning which is
> > > > e.g.
> > > > not respected these days).
> > >
> > > And I answered you. From a changelog POW you need the full dist
> > > because
> > > the exact same stripped/neutralized Release may (and does) exist
> > > today
> > > in different branches, pointing to different spec states.
> >
> > Can you give an example of such package?
> >
> > I mean, of course, technically it is possible and not forbidden
> > anywhere in the guidelines as far as I know. But...
>
> That’s pretty much inevitable when you hit release-specific problems
> that require pushing release-specific changes or patches in one or
> several branches. No matter what clever numbering conventions you try
> to invent, things are eventually going to collide.
>
> %{dist} is not here just to prettify things. Branching is real
> branching. syncing branches is a packager optimization. It’s not
> possible in all cases. The only hard requirement is to keep evr lower
> in older branches, syncing is optional and not done when it’s just a
> lot of pain for little win.
>
> You can say “start from the -devel evr, add .number when adding a
> patch”. That only gets you as far as the first patch. What if f33 state
> needs patching one way in f32 and another in f31 (ignoring el7 and el8
> for now)? Your convention already hit its limits. And it’s just the
> first patch step.
If I wanted to be super-clean, I would simply do a minor bump for f32
and f31 on top of the f33 patch. Minor bumps do not give users any
false impressions because they are placed after the dist tag.
>
> The usual packager reaction in that case is either add spaguetti dist
> ifdefs in their spec, or just give up on syncing (I definitely prefer
> the second strategy). After all, no harm done if the branches de-sync,
> as long as cross-release upgrade pathes work.
>
> > If I, as a user, see e.g. python3-alembic-1.1.0-13 in f31 and then I
> > will see python3-alembic-1.1.0-13 in f32 (this time with .fc32
> > disttag), I am going to assume nothing has changed for that package.
>
> And you’ll be wrong, because we do branch. I we could have avoided
> branching, we’d have avoided it. Branching brings quite a lot of
> complexity. However we do branch because releases overlap but are not
> technically equal.
We still should take into account user's intuitive expectations and
the way numbers are layed out in the Release string that raises those
expectations.
Yes, we always work in a context of a certain distribution branch so
it's very inconvenient to assign to a certain number meaning that
should somehow cross boundaries of that particular branch. Maybe the
way dist-git branches are layed out (per distribution version) is not
totally perfect for everything though and we should be aware of that.
>
> What we *can* do is make fedpkg merge master work to avoid packager
> work when the branches can be synced (or re-synced). And fedpkg merge
> master should work with my proposal (as long as the older branch has
> not already autobumped further than master, but a packager that allows
> that already broke the upgrade path, and earned manual clean-up work).
>
> > If we do these build-counter, commit-counter (without the leading
> > "pivot" number) approaches, then it would imho really be better to
> > have python3-alembic-1.1.0-fc31.13 and python3-alembic-1.1.0-fc32.13
> > which is quite a
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
wrote:
>
> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
> >
> > What about fedpkg local and fedpkg verrel then?
>
> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
> detached file state means that fedpkg local (or checking out git state
> and building in mock or copr or OBS or via plain rpmbuild -bs) will
> give you the same result as launching fedpkg build.
Well, I believe it doesn't. You run:
1) fedpkg local -> produces -1.0-1.fc32 (1 in release because
you haven't built that package before)
2) fedpkg build
3) vim .spec and do some change in %description
4) fedpkg commit -m "description improvement"
5) fedpkg local -> produces -1.0-1.fc32
6) fedpkg push -> error because build system pushed meanwhile
7) fedpkg pull --rebase (to quickly fix the error)
8) fedpkg build -> builds -1.0-2.fc32
To get the correct NVR for the package being built by `fedpkg local`
you need to wait for some external process to finish first so that you
can pull the new state. Only then you get correct value again.
> Which is exactly
> what you want to make QA and testing workflows work. Those don’t live
> exclusively in koji.
>
> And because only production builds get committed back the packager can
> change his mind and stage a few more changes before doing the
> production build, without polluting the changelog builds that were
> never pushed to users.
That's a good thing, definitely.
>
> For fedpkg verrel you'll probably want to output a last (saved in
> detached file) and next line (probably factorizable by externalizing
> the dynrel bump logic in a separate command). That’s more honest
> anyway, next may happen or not, when you launch fedpkg verrel, it’s
> mere potential (the next commit may bump version and invalidate your
> future build).
There is the same problem as above with `fedpkg local`, the command
won't be giving you correct values at all times.
>
> > As for changelog, generally with build system commiting back to
> > dist-git, there is a problem that I can be concurrently pushing
> > changes while the build system tries to push its changes and build
> > system is not human so it will not know how to resolve such
> > situation.
>
> I understand, but that’s the consequence of automating changelog. Right
> now the reconciliation is done by the human packager (you can get in a
> similar situation by working on a change at the time a mass rebuild
> runs).
It doesn't need to be the consequence of "automating changelog". It's
only a consequence in certain implementations. If the changelog is
derived purely from the git state, there aren't these problems
anymore. The mass rebuilds are easier to handle than your case because
with mass rebuilds, you always put the the new changelog record "on
top" of other changelogs records. I.e. when you cannot push from build
system (while peforming a mass rebuild), you can simply pull and try
to do the same thing as before. But in your proposal it's not that
easy (unless you do the locking or failing in that case as you
described below) because you would need modify/insert into "middle" of
the changelog at the right place.
>
> > Do you have a solution for situation when I launch a build and then I
> > do another change, commit, and push (the changes to changelog can be
> > quite arbitrary here)?
>
> You can set a lock at fedpkg build time, and forbid fedpkg commit in
> that branch till the lock is released (fedpkg build in the branch
> succeeded or not). The packager can still git commit things, as long as
> he does not touch the detached changelog file. Only fedpkg commit syncs
> git state with detached changelog state.
The fact that "git commit" is different from "fedpkg commit" is not
very convenient imho but could be probably get used to. But you
generally don't except commit actions to change the repository content
under your hands before actually committing. The locking mechanism you
describe can be easily work-arounded by cloning the package repo again
and doing your stuff. The only way to avoid it would be to implement
locking server side. Either way it's not the best user experience.
>
> An alternate simpler strategy would be to mark the fedpkg build as
> failure in koji if sync-back failed. That would work too. The build
> system need not be ultra smart, just robust WRT human behaviour.
I think not being able to work with dist-git while something is
building its content is a throughput limitation and doesn't bring the
best UX.
>
> (If the packager deliberately messes with the detached changelog while
> a production build for the same branch is ongoing he deserves a build
> failure – the changelog state is undefined till the build ends, so he’s
> doing changes on quicksand. If he tried that today he’d probably have
> to rewrite his changelog if the build failed)
>
> > I think, this is not a decision of rpm upstream but an infrastructure
> > thing or a mock thing.
>
> mock and rpm
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 20:57 +0100, clime a écrit :
> On Sat, 29 Feb 2020 at 16:24, Nicolas Mailhot via devel
> wrote:
> > Le samedi 29 février 2020 à 15:12 +0100, clime a écrit :
> > > On Sat, 29 Feb 2020 at 14:47, Nicolas Mailhot via devel
> > > wrote:
> > > > Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
> > > > > Does ENVR without %{dist} means something with respect to the
> > > > > content
> > > > > from
> > > > > which the package was built or with respect to features that
> > > > > it
> > > > > offers for the given distribution version?
> > > >
> > > > You need to evaluate evr with a neutral dist value to decide to
> > > > bump or
> > > > not the auto-dynamic part of release or not. Because the whole
> > > > point of
> > > > the auto-dynamic part of release would be to track rebuilds
> > > > from
> > > > the
> > > > same spec, all othert parts of EVR being equal
> > > >
> > > > Changelog-side and package build side you need the full EVR
> > > > without
> > > > neutralization
> > >
> > > Thank you very much Nicolas but you reacted to a question which
> > > was
> > > actually unrelated to your proposal. That particular question
> > > about
> > > the meaning of ENVR when you strip the distribution tag (i.e.
> > > .fc32
> > > or
> > > .el7) was intended to be generic, i.e. i want to know how if
> > > e.g.
> > > python3-alembic-1.1.0-1 has any meaning alone without, for
> > > example,
> > > .fc31 appended to it (or if it should have any meaning which is
> > > e.g.
> > > not respected these days).
> >
> > And I answered you. From a changelog POW you need the full dist
> > because
> > the exact same stripped/neutralized Release may (and does) exist
> > today
> > in different branches, pointing to different spec states.
>
> Can you give an example of such package?
>
> I mean, of course, technically it is possible and not forbidden
> anywhere in the guidelines as far as I know. But...
That’s pretty much inevitable when you hit release-specific problems
that require pushing release-specific changes or patches in one or
several branches. No matter what clever numbering conventions you try
to invent, things are eventually going to collide.
%{dist} is not here just to prettify things. Branching is real
branching. syncing branches is a packager optimization. It’s not
possible in all cases. The only hard requirement is to keep evr lower
in older branches, syncing is optional and not done when it’s just a
lot of pain for little win.
You can say “start from the -devel evr, add .number when adding a
patch”. That only gets you as far as the first patch. What if f33 state
needs patching one way in f32 and another in f31 (ignoring el7 and el8
for now)? Your convention already hit its limits. And it’s just the
first patch step.
The usual packager reaction in that case is either add spaguetti dist
ifdefs in their spec, or just give up on syncing (I definitely prefer
the second strategy). After all, no harm done if the branches de-sync,
as long as cross-release upgrade pathes work.
> If I, as a user, see e.g. python3-alembic-1.1.0-13 in f31 and then I
> will see python3-alembic-1.1.0-13 in f32 (this time with .fc32
> disttag), I am going to assume nothing has changed for that package.
And you’ll be wrong, because we do branch. I we could have avoided
branching, we’d have avoided it. Branching brings quite a lot of
complexity. However we do branch because releases overlap but are not
technically equal.
What we *can* do is make fedpkg merge master work to avoid packager
work when the branches can be synced (or re-synced). And fedpkg merge
master should work with my proposal (as long as the older branch has
not already autobumped further than master, but a packager that allows
that already broke the upgrade path, and earned manual clean-up work).
> If we do these build-counter, commit-counter (without the leading
> "pivot" number) approaches, then it would imho really be better to
> have python3-alembic-1.1.0-fc31.13 and python3-alembic-1.1.0-fc32.13
> which is quite a huge change.
The system provides a %{dynrel} counter. The packager can stick it
before or after %{dist} (or not use it at all), the mecanism will work
the same (obviously, without autobumping if the packager does not use
%{dynrel} in his release string).
%{dynrel} is sufficient to handle autobumping. If you try to own the
whole release string, you’ll hit all kinds of complexity (starting with
pre/post release, then the ruby thing, etc)
%{dist} (especially taking %{distprefix} into account, but even without
it) does not sort well. elx/fcx does not sort. Third part packages may
use a sort-able dist or not. if you want evr to work, you put the
relevant part of r before dist, and keep it only as last resort to
distinguish between synced fedora branches.
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>
> What about fedpkg local and fedpkg verrel then?
Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
detached file state means that fedpkg local (or checking out git state
and building in mock or copr or OBS or via plain rpmbuild -bs) will
give you the same result as launching fedpkg build. Which is exactly
what you want to make QA and testing workflows work. Those don’t live
exclusively in koji.
And because only production builds get committed back the packager can
change his mind and stage a few more changes before doing the
production build, without polluting the changelog builds that were
never pushed to users.
For fedpkg verrel you'll probably want to output a last (saved in
detached file) and next line (probably factorizable by externalizing
the dynrel bump logic in a separate command). That’s more honest
anyway, next may happen or not, when you launch fedpkg verrel, it’s
mere potential (the next commit may bump version and invalidate your
future build).
> As for changelog, generally with build system commiting back to
> dist-git, there is a problem that I can be concurrently pushing
> changes while the build system tries to push its changes and build
> system is not human so it will not know how to resolve such
> situation.
I understand, but that’s the consequence of automating changelog. Right
now the reconciliation is done by the human packager (you can get in a
similar situation by working on a change at the time a mass rebuild
runs).
> Do you have a solution for situation when I launch a build and then I
> do another change, commit, and push (the changes to changelog can be
> quite arbitrary here)?
You can set a lock at fedpkg build time, and forbid fedpkg commit in
that branch till the lock is released (fedpkg build in the branch
succeeded or not). The packager can still git commit things, as long as
he does not touch the detached changelog file. Only fedpkg commit syncs
git state with detached changelog state.
An alternate simpler strategy would be to mark the fedpkg build as
failure in koji if sync-back failed. That would work too. The build
system need not be ultra smart, just robust WRT human behaviour.
(If the packager deliberately messes with the detached changelog while
a production build for the same branch is ongoing he deserves a build
failure – the changelog state is undefined till the build ends, so he’s
doing changes on quicksand. If he tried that today he’d probably have
to rewrite his changelog if the build failed)
> I think, this is not a decision of rpm upstream but an infrastructure
> thing or a mock thing.
mock and rpm upstreams excel when they work together. My premise is
that they are better qualified than us to do rpm/buildsys boundary fine
tuning.
Populating changelog from git and syncing back to fedora git are
fedpkg/koji responsability, because Fedora git is Fedora specific
infra. Handling release autobumping and build recording belongs to the
lower layers, however. They’re the same with or without Fedora git,
they belong to lower levels.
> I think your proposal wouldn't quite work for copr because it has no
> access to those repositories (which especially for src.fp.o is good
> in your proposal, otherwise copr would be modifying src.fp.o repos).
copr does not need write access to Fedora git, its builds do not
participate to the production build lineage as long as no one re-
imports them in koji (which should be a deliberate fedpkg command, not
something driven by copr itself).
copr only needs access to its own private git to remember its own last
successful build, if people want it to autobump from last successful
state. (maybe they don’t). srpm import in copr will overwrite copr
state with the state the new srpm contains, which is fine too.
Putting state in detached srpm source files has the following super-
duper property. You can import the srpm or scm checkout between
systems, and they’ll just pick up from the point the previous system
left things, without needing a full scm import. So you do not need to
deal with incompatible scms (or lack of scm).
> So if someone wanted to have a build counter in the release, yes,
> this could be an implementation although it would be much easier to
> simply catch the information from copr's database where this info is
> stored
It is simpler from the buildsys POW, but it ties state in a specific
git and buildsystem. So it will break cross-buildsys workflows.
Cross-buildsys workflows are critical for the project success because
they enable sharing work with other distros, and allow packagers to
make the best of a palette of build systems (each with its own
constrains and limitiations). Fun fact: I noticed today than one of my
old Fedora packages was rebuilt by others for AIX. This kind of cross-
pollination is one of the strengths of the rpm ecosystem. Don’t break
it by making out packages depend on Fedora git or buildsys state.
> in
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 16:24, Nicolas Mailhot via devel
wrote:
>
> Le samedi 29 février 2020 à 15:12 +0100, clime a écrit :
> > On Sat, 29 Feb 2020 at 14:47, Nicolas Mailhot via devel
> > wrote:
> > > Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
> > > > Does ENVR without %{dist} means something with respect to the
> > > > content
> > > > from
> > > > which the package was built or with respect to features that it
> > > > offers for the given distribution version?
> > >
> > > You need to evaluate evr with a neutral dist value to decide to
> > > bump or
> > > not the auto-dynamic part of release or not. Because the whole
> > > point of
> > > the auto-dynamic part of release would be to track rebuilds from
> > > the
> > > same spec, all othert parts of EVR being equal
> > >
> > > Changelog-side and package build side you need the full EVR without
> > > neutralization
> >
> > Thank you very much Nicolas but you reacted to a question which was
> > actually unrelated to your proposal. That particular question about
> > the meaning of ENVR when you strip the distribution tag (i.e. .fc32
> > or
> > .el7) was intended to be generic, i.e. i want to know how if e.g.
> > python3-alembic-1.1.0-1 has any meaning alone without, for example,
> > .fc31 appended to it (or if it should have any meaning which is e.g.
> > not respected these days).
>
> And I answered you. From a changelog POW you need the full dist because
> the exact same stripped/neutralized Release may (and does) exist today
> in different branches, pointing to different spec states.
Can you give an example of such package?
I mean, of course, technically it is possible and not forbidden
anywhere in the guidelines as far as I know. But...
If I, as a user, see e.g. python3-alembic-1.1.0-13 in f31 and then I
will see python3-alembic-1.1.0-13 in f32 (this time with .fc32
disttag), I am going to assume nothing has changed for that package. I
think that is the intuitive user's expectation here. By providing the
same NVR (except for disttag) into the new Fedora release, as user had
installed before, I can tell to the user: "Nothing has changed for you
here". I mean, that's how I would interpret it.
If that stripped NVR suddenly starts depending on build count for the
given branch, I will start getting quite random numbers where some
package in the new Fedora release looks like "nothing has changed"
(according to its stripped NVR) but in fact a lot have changed. The
similar problem is with bumping by commit count too.
If we do these build-counter, commit-counter (without the leading
"pivot" number) approaches, then it would imho really be better to
have python3-alembic-1.1.0-fc31.13 and python3-alembic-1.1.0-fc32.13
which is quite a huge change.
>
> From a dynamic release POW dist is irrelevant to compute the next bump
> point and needs stripping. All that matters is that the result evrs
> differently, ignoring dist. Keeping branches synchronized or not is a
> packager decision (choosing to desync requires specific upgrade path
> care, but it is a valid use case today).
>
> And yes that info oriented my proposal. Any other proposal will have to
> take it into account too.
>
> Regards,
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 16:12, Nicolas Mailhot via devel
wrote:
>
> Le samedi 29 février 2020 à 13:49 +0100, clime a écrit :
> > On Sat, 29 Feb 2020 at 10:15, Nicolas Mailhot via devel
> > wrote:
> > > Hi,
> > >
> > > Anyway, here is what I would personnaly consider a robust,
> > > inclusive,
> > > and future-proof design:
> >
> > I will need to ask some questions to really understand it.
> >
> > > — a %{use_dynstate} rpm variable enables dynamic changelog/release
> > > behaviour
> > > — probably initialy set to false distro-wide, to let volunteers
> > > test
> > > the thing by setting it to true iin their specs,
> > > — then to true (opt-out), when kinks have been fixed, and
> > > FPC/FESCO
> > > greenlights general availability
> > >
> > > – when activated, last changelog, evr and %{dynrel} state are saved
> > > in one or several detached files
> >
> > You mean the last changelog, evr, and %{dynrel} are stored once
> > %{use_dynstate} is set and and after one invokes fedpkg commit?
> > I think there should be some explicit action (e.g. the commit) to
> > generate the files after I set the %{use_dynstate} value to true in
> > the spec file.
>
> Once a spec sets %{use_dynstate} to true changelog, last computed
> changelog, ev, neutralized-r, and %{dynrel} are taken from detached
> files. neutralized-r is the evaluation of Release with %{dist} set to a
> neutral value (for example “-”).
>
> State is stored in files included in the srpm to be able to import to
> and from Fedora git (pretty much a hard requirement if tooling is to be
> kept Fedora-git neutral, which is itself a hard requirement to be able
> to interact with packaging work existing outside Fedora git).
>
> You need the last computed ev and neutralized-r to decide whether
> %{dynrel} can be kept, needs bumping, or should be reset to 1.
>
> I don’t trust mixed mode, KISS, changelog is detached or not, don’t try
> to have it both ways, here lies madness and confusion.
>
> > How is %{dynrel} computed here at this stage
>
> Nothing is computed at this stage.
>
> Detached changelog state, is a commit property (except for build date).
> It is computed at fedpkg commit time:
> – if detached changelog data is absent
>→ move in-spec changelog data to the detached file if data exists,
>→ otherwise init detached data to nothing
> – after the first fedpkg commit with %{use_dynstate} set to true, the
>detached changelog file exists and the in-spec changelog has been
>removed.
> – others have detailed possible fedpkg commit strategies to stuff
>changelog with new info
>
> %{dynrel} and build changelog info are a build property. They are
> computed at build time:
> – if computed ev ≠ last saved ev or last %{dynrel} is undefined
>→ reset %{dynrel} to 1
> – otherwise if computed neutralized-r = last neutralized-r
>→ bump %{dynrel}
> – otherwise, do nothing, something already took care of things
> – save the new computed ev, computed neutralized-r and dynrel to the
>detached files
> – add a build line with the full evr (and full dist) to detached
>changelog
> – at build time, there is no relationship with magic git state or
>magic buildsys info, the srpms are 100% autonomous as before. The
>detached changelog has already been populated manually or by distro
>automation or manual processes when it reaches build stage and a new
>build is recorded (with a new dynrel value, if necessary).
>
> > Is the intention here that with each new build of the same package,
> > the value of %{dynrel} is bumped and committed back?
>
> It is bumped (or reseted to 1) whenever comparison with last saved
> state shows a bump or reset is needed.
>
> Build-time state changes need to be commited back, of course (otherwise
> the produced srpm needs to be re-imported manually)
>
> > That means the evr file read from the repository needs to contain
> > somehow outdated values at this point (when srpm is being built in
> > build system), otherwise %{dynrel} would be always bumped?
>
> Final dynrel state is not computed before a build, yes. The build will
> decide if it needs to bump dynrel, or can reset it to 1. I had
> forgotten to document the reset, sorry.
>
> There is no need to compute a new dynrel before actual build, the
> packager may yet change ev or r.
>
> > > – a build line is added to the detached changelog, using the
> > > current
> > > date and full evr (without %{dist} neutralization)
> > > — that probably requires defining a rpm variable to track whom
> > > the
> > > build is attributed to
> > > – it can default to "Anonymous coward" or whatever if not
> > > explicitely
> > > set
> >
> > I think today's changelogs do not contain name of the person who
> > built the pacakge
>
> Today’s changelog mixes who made spec changes and who built the result.
> That what confuses you. Confusion is anthetical with automation. This
> proposal clearly documents that changelog can change at build time
> (fedpkg build)
Re: Ideas and proposal for removing changelog and release fields from spec file
On 2/27/20 12:08 AM, Dan Čermák wrote: For the changelog: yes please, generate it from the commit log! They are more or less the same for all my packages and I'm getting tired of copy pasting the same text into %changelog and git commit. Do you know about fedpkg commit --clog ? -- Orion Poplawski Manager of NWRA Technical Systems 720-772-5637 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nwra.com Boulder, CO 80301 https://www.nwra.com/ smime.p7s Description: S/MIME Cryptographic Signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 16:11 +0100, Nicolas Mailhot a écrit : > > Build-time state changes need to be commited back, of course > (otherwise the produced srpm needs to be re-imported manually) Probably, only on *successful* production build however. We don’t need to record failed or scratch builds in our package changelogs Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 15:12 +0100, clime a écrit :
> On Sat, 29 Feb 2020 at 14:47, Nicolas Mailhot via devel
> wrote:
> > Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
> > > Does ENVR without %{dist} means something with respect to the
> > > content
> > > from
> > > which the package was built or with respect to features that it
> > > offers for the given distribution version?
> >
> > You need to evaluate evr with a neutral dist value to decide to
> > bump or
> > not the auto-dynamic part of release or not. Because the whole
> > point of
> > the auto-dynamic part of release would be to track rebuilds from
> > the
> > same spec, all othert parts of EVR being equal
> >
> > Changelog-side and package build side you need the full EVR without
> > neutralization
>
> Thank you very much Nicolas but you reacted to a question which was
> actually unrelated to your proposal. That particular question about
> the meaning of ENVR when you strip the distribution tag (i.e. .fc32
> or
> .el7) was intended to be generic, i.e. i want to know how if e.g.
> python3-alembic-1.1.0-1 has any meaning alone without, for example,
> .fc31 appended to it (or if it should have any meaning which is e.g.
> not respected these days).
And I answered you. From a changelog POW you need the full dist because
the exact same stripped/neutralized Release may (and does) exist today
in different branches, pointing to different spec states.
From a dynamic release POW dist is irrelevant to compute the next bump
point and needs stripping. All that matters is that the result evrs
differently, ignoring dist. Keeping branches synchronized or not is a
packager decision (choosing to desync requires specific upgrade path
care, but it is a valid use case today).
And yes that info oriented my proposal. Any other proposal will have to
take it into account too.
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 13:49 +0100, clime a écrit :
> On Sat, 29 Feb 2020 at 10:15, Nicolas Mailhot via devel
> wrote:
> > Hi,
> >
> > Anyway, here is what I would personnaly consider a robust,
> > inclusive,
> > and future-proof design:
>
> I will need to ask some questions to really understand it.
>
> > — a %{use_dynstate} rpm variable enables dynamic changelog/release
> > behaviour
> > — probably initialy set to false distro-wide, to let volunteers
> > test
> > the thing by setting it to true iin their specs,
> > — then to true (opt-out), when kinks have been fixed, and
> > FPC/FESCO
> > greenlights general availability
> >
> > – when activated, last changelog, evr and %{dynrel} state are saved
> > in one or several detached files
>
> You mean the last changelog, evr, and %{dynrel} are stored once
> %{use_dynstate} is set and and after one invokes fedpkg commit?
> I think there should be some explicit action (e.g. the commit) to
> generate the files after I set the %{use_dynstate} value to true in
> the spec file.
Once a spec sets %{use_dynstate} to true changelog, last computed
changelog, ev, neutralized-r, and %{dynrel} are taken from detached
files. neutralized-r is the evaluation of Release with %{dist} set to a
neutral value (for example “-”).
State is stored in files included in the srpm to be able to import to
and from Fedora git (pretty much a hard requirement if tooling is to be
kept Fedora-git neutral, which is itself a hard requirement to be able
to interact with packaging work existing outside Fedora git).
You need the last computed ev and neutralized-r to decide whether
%{dynrel} can be kept, needs bumping, or should be reset to 1.
I don’t trust mixed mode, KISS, changelog is detached or not, don’t try
to have it both ways, here lies madness and confusion.
> How is %{dynrel} computed here at this stage
Nothing is computed at this stage.
Detached changelog state, is a commit property (except for build date).
It is computed at fedpkg commit time:
– if detached changelog data is absent
→ move in-spec changelog data to the detached file if data exists,
→ otherwise init detached data to nothing
– after the first fedpkg commit with %{use_dynstate} set to true, the
detached changelog file exists and the in-spec changelog has been
removed.
– others have detailed possible fedpkg commit strategies to stuff
changelog with new info
%{dynrel} and build changelog info are a build property. They are
computed at build time:
– if computed ev ≠ last saved ev or last %{dynrel} is undefined
→ reset %{dynrel} to 1
– otherwise if computed neutralized-r = last neutralized-r
→ bump %{dynrel}
– otherwise, do nothing, something already took care of things
– save the new computed ev, computed neutralized-r and dynrel to the
detached files
– add a build line with the full evr (and full dist) to detached
changelog
– at build time, there is no relationship with magic git state or
magic buildsys info, the srpms are 100% autonomous as before. The
detached changelog has already been populated manually or by distro
automation or manual processes when it reaches build stage and a new
build is recorded (with a new dynrel value, if necessary).
> Is the intention here that with each new build of the same package,
> the value of %{dynrel} is bumped and committed back?
It is bumped (or reseted to 1) whenever comparison with last saved
state shows a bump or reset is needed.
Build-time state changes need to be commited back, of course (otherwise
the produced srpm needs to be re-imported manually)
> That means the evr file read from the repository needs to contain
> somehow outdated values at this point (when srpm is being built in
> build system), otherwise %{dynrel} would be always bumped?
Final dynrel state is not computed before a build, yes. The build will
decide if it needs to bump dynrel, or can reset it to 1. I had
forgotten to document the reset, sorry.
There is no need to compute a new dynrel before actual build, the
packager may yet change ev or r.
> > – a build line is added to the detached changelog, using the
> > current
> > date and full evr (without %{dist} neutralization)
> > — that probably requires defining a rpm variable to track whom
> > the
> > build is attributed to
> > – it can default to "Anonymous coward" or whatever if not
> > explicitely
> > set
>
> I think today's changelogs do not contain name of the person who
> built the pacakge
Today’s changelog mixes who made spec changes and who built the result.
That what confuses you. Confusion is anthetical with automation. This
proposal clearly documents that changelog can change at build time
(fedpkg build) and between builds (fedpkg commit).
From a package consumer POW, the only attribution and timestamp that
matters is who build the final package and when, because the builder is
taking the responsability of pushing a result to users.
A clean changelog is:
%changelog
*
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 14:47, Nicolas Mailhot via devel
wrote:
>
> Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
> >
> > Does ENVR without %{dist} means something with respect to the content
> > from
> > which the package was built or with respect to features that it
> > offers for the given distribution version?
>
> You need to evaluate evr with a neutral dist value to decide to bump or
> not the auto-dynamic part of release or not. Because the whole point of
> the auto-dynamic part of release would be to track rebuilds from the
> same spec, all othert parts of EVR being equal
>
> Changelog-side and package build side you need the full EVR without
> neutralization
Thank you very much Nicolas but you reacted to a question which was
actually unrelated to your proposal. That particular question about
the meaning of ENVR when you strip the distribution tag (i.e. .fc32 or
.el7) was intended to be generic, i.e. i want to know how if e.g.
python3-alembic-1.1.0-1 has any meaning alone without, for example,
.fc31 appended to it (or if it should have any meaning which is e.g.
not respected these days).
I posted a separate set of questions to understand your proposal which
is very interesting by the way but the description was a bit fuzzy at
some points so I needed more explanation. If you could rather react
there, it would be awesome.
Thank you again
clime
>
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le samedi 29 février 2020 à 14:28 +0100, clime a écrit :
>
> Does ENVR without %{dist} means something with respect to the content
> from
> which the package was built or with respect to features that it
> offers for the given distribution version?
You need to evaluate evr with a neutral dist value to decide to bump or
not the auto-dynamic part of release or not. Because the whole point of
the auto-dynamic part of release would be to track rebuilds from the
same spec, all othert parts of EVR being equal
Changelog-side and package build side you need the full EVR without
neutralization
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 18:07, Zbigniew Jędrzejewski-Szmek
wrote:
>
> On Tue, Feb 25, 2020 at 05:42:11PM +0100, Miro Hrončok wrote:
> > On 25. 02. 20 9:50, Pierre-Yves Chibon wrote:
> > >Upgrade path may be problematic if you update Fn to a version in less
> > >commit
> > >than the update for Fn-1 (ie: you update F32 to 1.0 in 1 commit and update
> > >F31
> > >to 1.0 in 2 commits, suddenly you have F32 with 1.0-1 and F31 with 1.0-2).
> >
> > I don't consider that an issue. It's not super elegant, but since we
> > do distro-sync on upgrades, it shuld be fine.
>
> Hmm, I don't do distro-sync and in general I think upgrade path is
> something that should be preserved.
>
> What about doing --.?
This is a very good point really. Either it should have been always like that or
we will lose something by removing that number just after the last dash.
I.e. what's the definition of Release number (the number just after
the last dash).
Does it have a definition? I was looking for it in packaging
guidelines but couldn't
find it.
Does ENVR without %{dist} means something with respect to the content from
which the package was built or with respect to features that it offers
for the given
distribution version?
Thank you
clime
> This means that upgrade path not affected by the number of commits or
> builds in the older release.
>
> The numbers in different branches cannot
> be meaningfully compared. Those numbers only make sense in the context
> of a specific branch, so they should be ordered after .
>
> Zbyszek
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Sat, 29 Feb 2020 at 10:15, Nicolas Mailhot via devel
wrote:
>
> Hi,
>
> Anyway, here is what I would personnaly consider a robust, inclusive,
> and future-proof design:
I will need to ask some questions to really understand it.
>
> — a %{use_dynstate} rpm variable enables dynamic changelog/release
> behaviour
> — probably initialy set to false distro-wide, to let volunteers test
> the thing by setting it to true iin their specs,
> — then to true (opt-out), when kinks have been fixed, and FPC/FESCO
> greenlights general availability
>
> – when activated, last changelog, evr and %{dynrel} state are saved in
> one or several detached files
You mean the last changelog, evr, and %{dynrel} are stored once
%{use_dynstate} is set and and after one invokes fedpkg commit?
I think there should be some explicit action (e.g. the commit) to generate
the files after I set the %{use_dynstate} value to true in the spec file.
How is %{dynrel} computed here at this stage - does it have something
to do with the number of commits from the latest version change or
similar?
> — evr computed using a fixed neutral %{dist} value (for example “-”
> since it’s forbidden in %{dist})
>
> – those files are given standard rpm variable names and added to
> Sources:
> – manual packager Source900: %{dynstate} or whatever
> – rpm later updated to allow source file declarations via macros at
> to eliminate boilerplate (I need this in forge and go packages)
> – or the detached files are set in stone as separate Tags with a
> default value overridable via the %{dynstate} variable in a new rpm
> version
>
> – the packager adds %{dynrel} wherever he sees fit in his Release
> string
>
> – at fedpkg commit time changelog state is updated with info taken from
> fedora git state, *without* evr and build date
> – that’s Fedora-specific integration, exact commit/tag syntax to mark
> things to inject or ignore TBD Fedora-side
> – fedpkg commit sets a tag in git to mark anything older can now be
> ignored for changelog generation purposes
> – detached changelog state remains packager-editable, like the in-
> spec changelog today. That allows pruning useless no longer relevant
> stuff and fixing grammar errors
>
> — at rpmbuild -bs stage
> – evr is computed using the same neutral %{dist} value as before, and
> the saved %{dynrel} value
> – if it is equal to the previously saved evr %{dynrel} is bumped and
> saved in the detached file
Is the intention here that with each new build of the same package,
the value of %{dynrel} is bumped and committed back?
Only if other parts of release (other from %{dist} and %{dynrel}) or version or
release have changed, nothing is done? That means the evr file read from the
repository needs to contain somehow outdated values at this point (when srpm
is being built in build system), otherwise %{dynrel} would be always bumped?
> – a build line is added to the detached changelog, using the current
> date and full evr (without %{dist} neutralization)
> — that probably requires defining a rpm variable to track whom the
> build is attributed to
> – it can default to "Anonymous coward" or whatever if not explicitely
> set
I think today's changelogs do not contain name of the person who
built the pacakge (unless it is a mass rebuild), do you think something
like this is needed? Usually a person responsible for release of the
package (for the given "evr") is mentioned in the changelog record.
> – Koji, OBS and Copr can set it to whoever asked them to build the
> package
> – the result constitutes the new srpm (either first or second level
> srpm as upstream rpm sees fit)
You mean there would be different kinds of built srpms? Or otherwise
i don't under why upstream rpm (i understand it as
https://github.com/rpm-software-management/rpm)
should be involved here. What is meant by the "levels"?
> – that’s generic non Fedora-specific behaviour that work as well in
> rpmbuild -bs, mock, copr, koji, OBS, whatever, with or without Fedora
> git presence
> – Koji/copr need to commit the new dynamic dynrel/changelog state to
> git (a build-striggered state change is commited by the build system)
For copr, it is not possible, because it has read-only access to git
repositories being built.
> And yes that requires some work rpm-side. That is necessary to maintain
> the rpm decentralized design and keep srpms independent from anyone’s
> git state. Personnaly, I don’t see the point of pretending we can move
> our infra forward without ever touching rpm.
I think there are good examples where some things can be done without
rpm changes - e.g. the simple-koji-ci to do automatic scratch builds
on a new commit.
> The cardinal sin of
> Modularity was attempting to create an overlay over existing rpm/dnf
> behaviour, without changing this behaviour when it needed improving.
>
> Contrat that with dynamic buildrequires or weak deps that slotted into
> our workflows with hardly any ripple. Though
Re: Ideas and proposal for removing changelog and release fields from spec file
Hi,
Anyway, here is what I would personnaly consider a robust, inclusive,
and future-proof design:
— a %{use_dynstate} rpm variable enables dynamic changelog/release
behaviour
— probably initialy set to false distro-wide, to let volunteers test
the thing by setting it to true iin their specs,
— then to true (opt-out), when kinks have been fixed, and FPC/FESCO
greenlights general availability
– when activated, last changelog, evr and %{dynrel} state are saved in
one or several detached files
— evr computed using a fixed neutral %{dist} value (for example “-”
since it’s forbidden in %{dist})
– those files are given standard rpm variable names and added to
Sources:
– manual packager Source900: %{dynstate} or whatever
– rpm later updated to allow source file declarations via macros at
to eliminate boilerplate (I need this in forge and go packages)
– or the detached files are set in stone as separate Tags with a
default value overridable via the %{dynstate} variable in a new rpm
version
– the packager adds %{dynrel} wherever he sees fit in his Release
string
– at fedpkg commit time changelog state is updated with info taken from
fedora git state, *without* evr and build date
– that’s Fedora-specific integration, exact commit/tag syntax to mark
things to inject or ignore TBD Fedora-side
– fedpkg commit sets a tag in git to mark anything older can now be
ignored for changelog generation purposes
– detached changelog state remains packager-editable, like the in-
spec changelog today. That allows pruning useless no longer relevant
stuff and fixing grammar errors
— at rpmbuild -bs stage
– evr is computed using the same neutral %{dist} value as before, and
the saved %{dynrel} value
– if it is equal to the previously saved evr %{dynrel} is bumped and
saved in the detached file
– a build line is added to the detached changelog, using the current
date and full evr (without %{dist} neutralization)
— that probably requires defining a rpm variable to track whom the
build is attributed to
– it can default to "Anonymous coward" or whatever if not explicitely
set
– Koji, OBS and Copr can set it to whoever asked them to build the
package
– the result constitutes the new srpm (either first or second level
srpm as upstream rpm sees fit)
– that’s generic non Fedora-specific behaviour that work as well in
rpmbuild -bs, mock, copr, koji, OBS, whatever, with or without Fedora
git presence
– Koji/copr need to commit the new dynamic dynrel/changelog state to
git (a build-striggered state change is commited by the build system)
And yes that requires some work rpm-side. That is necessary to maintain
the rpm decentralized design and keep srpms independent from anyone’s
git state. Personnaly, I don’t see the point of pretending we can move
our infra forward without ever touching rpm. The cardinal sin of
Modularity was attempting to create an overlay over existing rpm/dnf
behaviour, without changing this behaviour when it needed improving.
Contrat that with dynamic buildrequires or weak deps that slotted into
our workflows with hardly any ripple. Though they were major feature
changes. But, they were done with rpm upstream, instead of trying to
bypass it.
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Fri, 28 Feb 2020 at 15:07, Zbigniew Jędrzejewski-Szmek wrote: > > On Fri, Feb 28, 2020 at 12:31:23PM +0100, Vít Ondruch wrote: > > > > Dne 27. 02. 20 v 16:11 Zbigniew Jędrzejewski-Szmek napsal(a): > > > Hi, > > > > > > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > > >> For the changelog, we believe we have a potential good solution: > > >> - The changelog will be automatically generated using an external > > >> `changelog` > > >> file as well as the commit history > > > ... > > >> If you wanted to edit the changelog, you would: > > >> - Generate the changelog locally via a command like: > > >> `fedpkg generate-changelog > changelog` > > >> - Edit `changelog` as desired > > >> - Commit and push the changes > > > This means that a separate commit needs to be done *after* on top > > > of the commits doing the actual changes. > > > > > > It would not need any extra commit, if you modified manually the > > changelog with the last entry. Which should be not problem, considering > > that `fedpkg generate-changelog > changelog` would be executed locally > > and you can amend the last commit with the changelog. > > Yes, but... Normally I'd do a cycle of "hack + build + commit". > Now if I do changelog generation, I'd either have to repeat it after > every step, or do it once at the end and then the last commit would > suddenly contain a changelog update for multiple previous commits. Either > way, this is not elegant and rather brittle. I thought the main reason not to combine update in the changelog file with code commits is to avoid conflicts when cherry picking as you described. I.e. i do minor update specifically in f32 and generate changelog file in the same commit. Then I want to do normal release update for all fedora branches. I start with master (as usual) and add e.g. add a new patch and generate the changelog file because i would like to add more info, then commit. Now I cannot cherry-pick that commit into f32 without conflict. In this case we wouldn't achieve one the targets of this proposal (afaik) => getting rid of merge conflicts in changelog and release - this is cherry-picking but still it would be nice not to have conflicts there. This target isn't in the document i think but i thought this is one of the goals. clime > > Zbyszek > > > > It's a bit disappointing, but > > > on its own would not be too bad, since we can have fedpkg provide a > > > higher level command that combines generate-changelog and build... > > > > > > One important feature will work: > > > being able to cherry-pick commit between branches w/o spurious > > > conflicts. As long as the "real" change to the spec file are done in > > > separate commits, and the changelog commit is another commit on top, > > > then when cherry-picking to another branch, the "real" commits would > > > be cherry-pickable, and then the changelog commit would be recreated > > > using the automation, OK. > > > > > > But it doesn't work quite as nicely with something similar: merging > > > branches. A simple 'git merge' would result in conflicts. > > > > > > Also, if an amendment to the changelog is done, and the same change > > > needs to be done in the changelog in a different branch, trying to > > > cherry-pick the fix commit would most likely result in a merge > > > conflict. > > > > > > Considering these three drawbacks (separate commit step and resulting > > > log noise, merge conflicts), I'd very much hope for a solution where > > > the changelog is never stored in the version control, and is always > > > autogenerated at srpm creation time. We should never try to chery-pick > > > commits that have changelog entries with actual date or e-v-r texts, > > > since this will inevitably lead to merge conflicts. > > > > > > > > > A different approach: > > > - we have 'fedpkg generate-changelog' (which does all the git log > > > extraction that was described, I think that part is OK), > > > - the output from this command included in the srpm at srpm generation > > > time and never committed to version control, > > > - the output is annotated with the source commits hashes, so we can > > > see where each line came from. > > > > > > At any time, the packager can run 'fedpkg generate-changelog' to see > > > how the output looks like. If they see something they don't like, if > > > the commits haven't been pushed out yet, they can immediately run > > > 'git commit --amend' and recheck. If they have been pushed out, an > > > override > > > to the changelog could be committed as part of a commit message text. > > > > > > Git-changelog-suppress: adb42e > > > or > > > Git-changelog-suppress: adb42e..efefedeadb > > > or > > > Git-changelog-replace: adb42e > > > Some other text with typos fixed that completely overrides whatever > > > was generated from adb42e. > > > or > > > Git-changelog-append: adb42e > > > (additional clarification for commit adb42e, e.g. bug #12345) > > > > > > This
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 28. 02. 20 v 15:06 Zbigniew Jędrzejewski-Szmek napsal(a): > On Fri, Feb 28, 2020 at 12:31:23PM +0100, Vít Ondruch wrote: >> Dne 27. 02. 20 v 16:11 Zbigniew Jędrzejewski-Szmek napsal(a): >>> Hi, >>> >>> On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: For the changelog, we believe we have a potential good solution: - The changelog will be automatically generated using an external `changelog` file as well as the commit history >>> ... If you wanted to edit the changelog, you would: - Generate the changelog locally via a command like: `fedpkg generate-changelog > changelog` - Edit `changelog` as desired - Commit and push the changes >>> This means that a separate commit needs to be done *after* on top >>> of the commits doing the actual changes. >> >> It would not need any extra commit, if you modified manually the >> changelog with the last entry. Which should be not problem, considering >> that `fedpkg generate-changelog > changelog` would be executed locally >> and you can amend the last commit with the changelog. > Yes, but... Normally I'd do a cycle of "hack + build + commit". > Now if I do changelog generation, I'd either have to repeat it after > every step, or do it once at the end and then the last commit would > suddenly contain a changelog update for multiple previous commits. Either > way, this is not elegant and rather brittle. But the point of automation is to not edit changelog at all, isn't it? If you are going to edit it, then the automation probably did not work good enough or you have pushed something you should not have. Changelog is brittle even now. Everybody is free to edit changelog from release to release and there are no restrictions what can be done there. The situation with automation won't be worse then it is now. Vít > > Zbyszek > >>> It's a bit disappointing, but >>> on its own would not be too bad, since we can have fedpkg provide a >>> higher level command that combines generate-changelog and build... >>> >>> One important feature will work: >>> being able to cherry-pick commit between branches w/o spurious >>> conflicts. As long as the "real" change to the spec file are done in >>> separate commits, and the changelog commit is another commit on top, >>> then when cherry-picking to another branch, the "real" commits would >>> be cherry-pickable, and then the changelog commit would be recreated >>> using the automation, OK. >>> >>> But it doesn't work quite as nicely with something similar: merging >>> branches. A simple 'git merge' would result in conflicts. >>> >>> Also, if an amendment to the changelog is done, and the same change >>> needs to be done in the changelog in a different branch, trying to >>> cherry-pick the fix commit would most likely result in a merge >>> conflict. >>> >>> Considering these three drawbacks (separate commit step and resulting >>> log noise, merge conflicts), I'd very much hope for a solution where >>> the changelog is never stored in the version control, and is always >>> autogenerated at srpm creation time. We should never try to chery-pick >>> commits that have changelog entries with actual date or e-v-r texts, >>> since this will inevitably lead to merge conflicts. >>> >>> >>> A different approach: >>> - we have 'fedpkg generate-changelog' (which does all the git log >>> extraction that was described, I think that part is OK), >>> - the output from this command included in the srpm at srpm generation >>> time and never committed to version control, >>> - the output is annotated with the source commits hashes, so we can >>> see where each line came from. >>> >>> At any time, the packager can run 'fedpkg generate-changelog' to see >>> how the output looks like. If they see something they don't like, if >>> the commits haven't been pushed out yet, they can immediately run >>> 'git commit --amend' and recheck. If they have been pushed out, an override >>> to the changelog could be committed as part of a commit message text. >>> >>> Git-changelog-suppress: adb42e >>> or >>> Git-changelog-suppress: adb42e..efefedeadb >>> or >>> Git-changelog-replace: adb42e >>> Some other text with typos fixed that completely overrides whatever >>> was generated from adb42e. >>> or >>> Git-changelog-append: adb42e >>> (additional clarification for commit adb42e, e.g. bug #12345) >>> >>> This would have the following advantages: >>> - git log is the sole source of truth >>> - there are no cherry-pick or merge conflicts >>> - there is no separate "now I'm done and I need to do another commit >>> before building" thing. >>> >>> The assumption is that those Git-changelog-* macros would only be >>> used occasionally, if the bad changelog entry was not noticed before >>> pushing the changes out. >>> >>> (One nitpick: when cherry-picking between branches, hashes of the >>> cherry-picked commits change, so "adb42e" in the example above >>> would
Re: Ideas and proposal for removing changelog and release fields from spec file
On Fri, Feb 28, 2020 at 12:31:23PM +0100, Vít Ondruch wrote: > > Dne 27. 02. 20 v 16:11 Zbigniew Jędrzejewski-Szmek napsal(a): > > Hi, > > > > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > >> For the changelog, we believe we have a potential good solution: > >> - The changelog will be automatically generated using an external > >> `changelog` > >> file as well as the commit history > > ... > >> If you wanted to edit the changelog, you would: > >> - Generate the changelog locally via a command like: > >> `fedpkg generate-changelog > changelog` > >> - Edit `changelog` as desired > >> - Commit and push the changes > > This means that a separate commit needs to be done *after* on top > > of the commits doing the actual changes. > > > It would not need any extra commit, if you modified manually the > changelog with the last entry. Which should be not problem, considering > that `fedpkg generate-changelog > changelog` would be executed locally > and you can amend the last commit with the changelog. Yes, but... Normally I'd do a cycle of "hack + build + commit". Now if I do changelog generation, I'd either have to repeat it after every step, or do it once at the end and then the last commit would suddenly contain a changelog update for multiple previous commits. Either way, this is not elegant and rather brittle. Zbyszek > > It's a bit disappointing, but > > on its own would not be too bad, since we can have fedpkg provide a > > higher level command that combines generate-changelog and build... > > > > One important feature will work: > > being able to cherry-pick commit between branches w/o spurious > > conflicts. As long as the "real" change to the spec file are done in > > separate commits, and the changelog commit is another commit on top, > > then when cherry-picking to another branch, the "real" commits would > > be cherry-pickable, and then the changelog commit would be recreated > > using the automation, OK. > > > > But it doesn't work quite as nicely with something similar: merging > > branches. A simple 'git merge' would result in conflicts. > > > > Also, if an amendment to the changelog is done, and the same change > > needs to be done in the changelog in a different branch, trying to > > cherry-pick the fix commit would most likely result in a merge > > conflict. > > > > Considering these three drawbacks (separate commit step and resulting > > log noise, merge conflicts), I'd very much hope for a solution where > > the changelog is never stored in the version control, and is always > > autogenerated at srpm creation time. We should never try to chery-pick > > commits that have changelog entries with actual date or e-v-r texts, > > since this will inevitably lead to merge conflicts. > > > > > > A different approach: > > - we have 'fedpkg generate-changelog' (which does all the git log > > extraction that was described, I think that part is OK), > > - the output from this command included in the srpm at srpm generation > > time and never committed to version control, > > - the output is annotated with the source commits hashes, so we can > > see where each line came from. > > > > At any time, the packager can run 'fedpkg generate-changelog' to see > > how the output looks like. If they see something they don't like, if > > the commits haven't been pushed out yet, they can immediately run > > 'git commit --amend' and recheck. If they have been pushed out, an override > > to the changelog could be committed as part of a commit message text. > > > > Git-changelog-suppress: adb42e > > or > > Git-changelog-suppress: adb42e..efefedeadb > > or > > Git-changelog-replace: adb42e > > Some other text with typos fixed that completely overrides whatever > > was generated from adb42e. > > or > > Git-changelog-append: adb42e > > (additional clarification for commit adb42e, e.g. bug #12345) > > > > This would have the following advantages: > > - git log is the sole source of truth > > - there are no cherry-pick or merge conflicts > > - there is no separate "now I'm done and I need to do another commit > > before building" thing. > > > > The assumption is that those Git-changelog-* macros would only be > > used occasionally, if the bad changelog entry was not noticed before > > pushing the changes out. > > > > (One nitpick: when cherry-picking between branches, hashes of the > > cherry-picked commits change, so "adb42e" in the example above > > would stop working. This is handled by using 'git cherry-pick -x'. > > 'fedpkg generate-changelog' would look at any hash in a > > "(cherry picked from commit ...)" line.) > > > > > > As how to hook this up with rpm, > > looking at > > https://pagure.io/packaging-committee/pull-request/942#comment-108542, > > a macro like %generate_changelog could be provided that would > > simply shell out to 'fedpkg generate-changelog'. > > > > > > I'll comment separate on the -release part. > > > > Zbyszek > >
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 27. 02. 20 v 16:11 Zbigniew Jędrzejewski-Szmek napsal(a): > Hi, > > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: >> For the changelog, we believe we have a potential good solution: >> - The changelog will be automatically generated using an external `changelog` >> file as well as the commit history > ... >> If you wanted to edit the changelog, you would: >> - Generate the changelog locally via a command like: >> `fedpkg generate-changelog > changelog` >> - Edit `changelog` as desired >> - Commit and push the changes > This means that a separate commit needs to be done *after* on top > of the commits doing the actual changes. It would not need any extra commit, if you modified manually the changelog with the last entry. Which should be not problem, considering that `fedpkg generate-changelog > changelog` would be executed locally and you can amend the last commit with the changelog. Vít > It's a bit disappointing, but > on its own would not be too bad, since we can have fedpkg provide a > higher level command that combines generate-changelog and build... > > One important feature will work: > being able to cherry-pick commit between branches w/o spurious > conflicts. As long as the "real" change to the spec file are done in > separate commits, and the changelog commit is another commit on top, > then when cherry-picking to another branch, the "real" commits would > be cherry-pickable, and then the changelog commit would be recreated > using the automation, OK. > > But it doesn't work quite as nicely with something similar: merging > branches. A simple 'git merge' would result in conflicts. > > Also, if an amendment to the changelog is done, and the same change > needs to be done in the changelog in a different branch, trying to > cherry-pick the fix commit would most likely result in a merge > conflict. > > Considering these three drawbacks (separate commit step and resulting > log noise, merge conflicts), I'd very much hope for a solution where > the changelog is never stored in the version control, and is always > autogenerated at srpm creation time. We should never try to chery-pick > commits that have changelog entries with actual date or e-v-r texts, > since this will inevitably lead to merge conflicts. > > > A different approach: > - we have 'fedpkg generate-changelog' (which does all the git log > extraction that was described, I think that part is OK), > - the output from this command included in the srpm at srpm generation > time and never committed to version control, > - the output is annotated with the source commits hashes, so we can > see where each line came from. > > At any time, the packager can run 'fedpkg generate-changelog' to see > how the output looks like. If they see something they don't like, if > the commits haven't been pushed out yet, they can immediately run > 'git commit --amend' and recheck. If they have been pushed out, an override > to the changelog could be committed as part of a commit message text. > > Git-changelog-suppress: adb42e > or > Git-changelog-suppress: adb42e..efefedeadb > or > Git-changelog-replace: adb42e > Some other text with typos fixed that completely overrides whatever > was generated from adb42e. > or > Git-changelog-append: adb42e > (additional clarification for commit adb42e, e.g. bug #12345) > > This would have the following advantages: > - git log is the sole source of truth > - there are no cherry-pick or merge conflicts > - there is no separate "now I'm done and I need to do another commit > before building" thing. > > The assumption is that those Git-changelog-* macros would only be > used occasionally, if the bad changelog entry was not noticed before > pushing the changes out. > > (One nitpick: when cherry-picking between branches, hashes of the > cherry-picked commits change, so "adb42e" in the example above > would stop working. This is handled by using 'git cherry-pick -x'. > 'fedpkg generate-changelog' would look at any hash in a > "(cherry picked from commit ...)" line.) > > > As how to hook this up with rpm, > looking at > https://pagure.io/packaging-committee/pull-request/942#comment-108542, > a macro like %generate_changelog could be provided that would > simply shell out to 'fedpkg generate-changelog'. > > > I'll comment separate on the -release part. > > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 16:13, Zbigniew Jędrzejewski-Szmek wrote: > > Hi, > > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > > For the changelog, we believe we have a potential good solution: > > - The changelog will be automatically generated using an external > > `changelog` > > file as well as the commit history > ... > > If you wanted to edit the changelog, you would: > > - Generate the changelog locally via a command like: > > `fedpkg generate-changelog > changelog` > > - Edit `changelog` as desired > > - Commit and push the changes > > This means that a separate commit needs to be done *after* on top > of the commits doing the actual changes. It's a bit disappointing, but > on its own would not be too bad, since we can have fedpkg provide a > higher level command that combines generate-changelog and build... > > One important feature will work: > being able to cherry-pick commit between branches w/o spurious > conflicts. As long as the "real" change to the spec file are done in > separate commits, and the changelog commit is another commit on top, > then when cherry-picking to another branch, the "real" commits would > be cherry-pickable, and then the changelog commit would be recreated > using the automation, OK. > > But it doesn't work quite as nicely with something similar: merging > branches. A simple 'git merge' would result in conflicts. > > Also, if an amendment to the changelog is done, and the same change > needs to be done in the changelog in a different branch, trying to > cherry-pick the fix commit would most likely result in a merge > conflict. > > Considering these three drawbacks (separate commit step and resulting > log noise, merge conflicts), I'd very much hope for a solution where > the changelog is never stored in the version control, and is always > autogenerated at srpm creation time. We should never try to chery-pick > commits that have changelog entries with actual date or e-v-r texts, > since this will inevitably lead to merge conflicts. > > > A different approach: > - we have 'fedpkg generate-changelog' (which does all the git log > extraction that was described, I think that part is OK), > - the output from this command included in the srpm at srpm generation > time and never committed to version control, > - the output is annotated with the source commits hashes, so we can > see where each line came from. > > At any time, the packager can run 'fedpkg generate-changelog' to see > how the output looks like. If they see something they don't like, if > the commits haven't been pushed out yet, they can immediately run > 'git commit --amend' and recheck. If they have been pushed out, an override > to the changelog could be committed as part of a commit message text. > > Git-changelog-suppress: adb42e > or > Git-changelog-suppress: adb42e..efefedeadb > or > Git-changelog-replace: adb42e > Some other text with typos fixed that completely overrides whatever > was generated from adb42e. > or > Git-changelog-append: adb42e > (additional clarification for commit adb42e, e.g. bug #12345) > > This would have the following advantages: > - git log is the sole source of truth > - there are no cherry-pick or merge conflicts > - there is no separate "now I'm done and I need to do another commit > before building" thing. > > The assumption is that those Git-changelog-* macros would only be > used occasionally, if the bad changelog entry was not noticed before > pushing the changes out. > > (One nitpick: when cherry-picking between branches, hashes of the > cherry-picked commits change, so "adb42e" in the example above > would stop working. This is handled by using 'git cherry-pick -x'. > 'fedpkg generate-changelog' would look at any hash in a > "(cherry picked from commit ...)" line.) Hello Zbyszek, it's good that it solves conflicts when cherry-picking and merging and git log is the source of truth but simply dropping a line that i don't want in the changelog or appending something to it seems to be quite painful to do. One advantage you mention is: > - there is no separate "now I'm done and I need to do another commit > before building" thing. But if you want to modify the automatic product, you would need one or more additional commits to do that. Imho, one would prefer to be able to do that simply by editing a text file (even if temporary). Annotated tags give you that ability by design - the tag annotation can be prepopulated and then freely edited, which is much more pleasant in my opinion. It doesn't clutter commit space and you can point other branches to that particular annotation by merging if desired. But the cherry-picking workflow, on the other hand, would require creating a new tag specific for that branch (that would suggest the tags should contain branchname/%dist at the end as fedpkg does it now). That's were things start getting clumsy unless there is something like fedpkg build --annotation-from
Re: Ideas and proposal for removing changelog and release fields from spec file
Le jeudi 27 février 2020 à 17:38 +0100, clime a écrit :
> On Thu, 27 Feb 2020 at 16:26, Nicolas Mailhot via devel
> wrote:
> > Le 2020-02-27 12:59, clime a écrit :
> > Hi,
> >
> > > can you, please, show an example of such package? I was searching
> > > through some
> > > golang packages because I was curious how it works but couldn't
> > > find
> > > an example
> >
> > A Go example:
> > https://src.fedoraproject.org/rpms/golang-x-build
> >
> > A non-Go example:
> > https://src.fedoraproject.org/rpms/impallari-dancing-script-fonts
> >
> > In both cases the spec calls a macro (not the same one in both
> > specs) that causes one or several distprefixNUMBER variables to be
> > computed
> >
> > The result then ends in Release via
> > https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
> >
> > The font spec example calls the macro in redhat-rpm-config directly
> > (%forgemeta), the Go example calls it through a Go-specific macro
> > in go-rpm-macros (%gometa)
>
> Okay, thanks :)!
>
> At least my idea was that there should be a spec file preprocessor,
> evaluating the macros that need git context, that would produce valid
> rpm spec that will be evaluated by the rpm engine in the end so
> everything related to %{dist} generation that is used in those
> packages will work unchanged.
>
> A small tweak in those spec files to allow dynamic release bumping
> could be:
>
> Release: 0.<>%{?dist}
>
> or alternatively:
>
> Release: <>%{?dist} depending on the
> implementation of the dynamic_release macro.
Yes, something like that would be possible, my worry was essentially
about not snipping commit info from automated changelogs.
%{dynrel} should default to 1 if buildsys info is not available
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 17:29, Nils Philippsen wrote: > > On Thu, 2020-02-27 at 15:10 +0100, clime wrote: > > Another thing to consider is whether we want a transparent build > > system where all the description of what will happen when a spec file > > is sent to it is included in the specfile itself or whether we want > > But we don't have that today: think of macros defined externally to the > SPEC file and RPM which we use, think of "underspecified" build > dependencies where what happens to be available and fulfil the BRs will > be used and ultimately influence how the package is built. What happens > when you build a package is already dependent on outside sources. > > > some auto-magic where e.g. 'Release' field or %changelog are > > automatically inserted when missing (something like that is > > impossible > > right now due to rpm constraints but just to make a point). > > Again, this is opt-in. And the way this will be made opt-in is that > people who want to use the feature use a macro, e.g. > "%automatic_release", so that it is clear that some value will be > filled in there. The idea isn't to have gutted spec files that don't > build anywhere else than in our build system, one fixed requirement is > that local builds using fedpkg or rpmbuild must continue to work. Yes, that sounds good! I just wouldn't like too much when a build system would be fiddling with spec files without some explicit macro that everyone can see _in_ the spec file telling it explicitly to do it - like e.g. changing values of some fields that packager manually specified for some reason, e.g. auto correction. > > Nils > -- > Nils Philippsen"Those who would give up Essential Liberty to > Software Engineer purchase a little Temporary Safety, deserve neither > Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 > PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D > old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 05:42:11PM +0100, Miro Hrončok wrote: > On 25. 02. 20 9:50, Pierre-Yves Chibon wrote: > >Upgrade path may be problematic if you update Fn to a version in less commit > >than the update for Fn-1 (ie: you update F32 to 1.0 in 1 commit and update > >F31 > >to 1.0 in 2 commits, suddenly you have F32 with 1.0-1 and F31 with 1.0-2). > > I don't consider that an issue. It's not super elegant, but since we > do distro-sync on upgrades, it shuld be fine. Hmm, I don't do distro-sync and in general I think upgrade path is something that should be preserved. What about doing --.? This means that upgrade path not affected by the number of commits or builds in the older release. The numbers in different branches cannot be meaningfully compared. Those numbers only make sense in the context of a specific branch, so they should be ordered after . Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 16:26, Nicolas Mailhot via devel
wrote:
>
> Le 2020-02-27 12:59, clime a écrit :
> Hi,
>
> >
> > can you, please, show an example of such package? I was searching
> > through some
> > golang packages because I was curious how it works but couldn't find
> > an example
>
> A Go example:
> https://src.fedoraproject.org/rpms/golang-x-build
>
> A non-Go example:
> https://src.fedoraproject.org/rpms/impallari-dancing-script-fonts
>
> In both cases the spec calls a macro (not the same one in both specs)
> that causes one or several disprefixNUMBER variables to be computed
>
> The result then ends in Release via
> https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
>
> The font spec example calls the macro in redhat-rpm-config directly, the
> go example calls it through a Go-specific macro in fonts-rpm-macros
Okay, thanks :)!
At least my idea was that there should be a spec file preprocessor,
evaluating the macros that need git context, that would produce valid
rpm spec that will be evaluated by the rpm engine in the end so
everything related to %{dist} generation that is used in those
packages will work unchanged.
A small tweak in those spec files to allow dynamic release bumping could be:
Release: 0.<>%{?dist}
or alternatively:
Release: <>%{?dist} depending on the
implementation of the dynamic_release macro.
clime
>
> Regards,
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Wed, 2020-02-26 at 23:07 -0500, Neal Gompa wrote:
> On Wed, Feb 26, 2020 at 8:05 PM Robert-André Mauchin <
> zebo...@gmail.com> wrote:
> > On Monday, 24 February 2020 17:48:36 CET Pierre-Yves Chibon wrote:
> > > However, for the release field, we are struggling a little bit
> > > more, two
> > > options are more appealing to us:
> > >
> > > A) The release field is automatically generated using two
> > > elements:
> > > - the number of commits at this version
> > > - the number of builds at this commit
> > > - the dist-tag being added after them
> > > The release field would thus look like:
> > > ``. > > commit>%{dist}``
> > >
> > > So we could have: (A, B, C and D being different commits)
> > > A -- version: 0.9 -- release: ?
> > >
> > > |
> > >
> > > B -- version: 1.0 -- release: 1.0
> > >
> > > |
> > >
> > > C -- version: 1.0 -- release: 2.0
> > >
> > > |
> > >
> > > D -- version: 1.1 -- release: 1.0
> > >
> > > A rebuild of the commit D, would lead to a release 1.1 (assuming
> > > the first
> > > one succeeded)/
> > >
> > > Pros:
> > > - Easy to replicate locally
> > > - Every changelog entry can be mapped to a `version-release` (No
> > > changes to
> > > the packaging guidelines)
> > > - Allows triggering two builds from the same commit without any
> > > manual
> > > change (simplifies mass-rebuilds)
> > > - Easy to link a certain build with a specific commit
> > > - Easy to “guess” the next release value before triggering the
> > > build
> > >
> > > Cons:
> > > - Old packages which are no longer receiving upstream releases
> > > may need
> > > special care (for example if they are at the release -48 but
> > > only had
> > > 45 commits leading to this)
> > > - Relies on information from the build system for the build
> > > number (but
> > > can be closely simulated locally since the info is only
> > > used for
> > > rebuilds)
> > > - Upgrade path may be problematic if Fn is upgraded to version
> > > X with
> > > one commit while Fn-1 is upgrade to version X with 2 commits (or
> > > more)
> > >
> > > B) The release field is automatically generated taking existing
> > > builds in
> > > all current Fedora releases (ie: rawhide, Fn, Fn-1...) into
> > > account. This
> > > means for builds of the same epoch:version, find a new release
> > > that (if at
> > > all possible) ensures upgradability from older Fedora versions to
> > > newer
> > > ones, adhering to the structure of a release tag documented in
> > > the
> > > Versioning Guidelines[1]. Going from the (RPM-wise) "latest
> > > build" that the
> > > new one should surpass, this can mean bumping in the front
> > > (`pkgrel`) or
> > > the back (`minorbump`).
> > >
> > > This allows packages from "very stable" upstreams who haven't
> > > released in
> > > years to still benefit from automatically generated releases.
> > >
> > > The following examples would use a macro for the release field as
> > > outlined
> > > in a separate document[2].
> > >
> > > Example #1 ("normal" release progression):
> > > Rawhide has: 2.0-1.fc32
> > > F31 has: 1.0-1.fc31
> > > F30 has: 1.0-1.fc30
> > >
> > > Next release in F31: 1.0-2.fc32
> > >
> > >
> > > Example #2 ("hotfix" in an older release, selected by an
> > > alternative macro
> > > (or option) in the spec file):
> > > Rawhide has: 2.0-1.fc32
> > > F31 has: 1.0-1.fc31
> > > F30 has: 1.0-1.fc30
> > >
> > > Next release in F30: 1.0-1.fc30.1
> > >
> > > Example #3 (pre-release, selected by an alternative macro (or
> > > option) in
> > > the spec file):
> > > Rawhide has: 2.0-1.fc32
> > > F31 has: 1.0-1.fc31
> > > F30 has: 1.0-1.fc30
> > >
> > > Next release in Rawhide: 3.0-0.1.20200224git1234abcd
> > >
> > >
> > > Pros:
> > > - Allows triggering two builds from the same commit without
> > > manual
> > > intervention
> > > - Emulates what many maintainers do manually today for most use
> > > cases
> > > - Can cater for pre-releases (e.g.: by offering different
> > > macros or macro
> > > options for the different use cases)
> > >
> > > Cons:
> > > - Needs the build system for information about builds in this
> > > and other
> > > Fedora versions
> > > - Not easy to reproduce locally because we don’t have machine-
> > > consumable
> > > knowledge about other builds in e.g. the dist-git repo
> > > - Does not allow to generate changelog entries with `version-
> > > release`
> > > information for all commits (and this will require a change
> > > in our
> > > packaging guidelines)
> > >
> > >
> > > So these are the results of our current investigations, we are
> > > very much
> > > eager to get your feedback on them and even more eager if you
> > > have ideas
> > > on how to approach/solve some of the challenges mentioned here.
> > >
> > >
> > > Looking forward for the discussion,
> > >
> > > Pierre
> > > For Adam, Nils and myself
> > >
> > >
> > >
> >
> > How would that work with "complex" releases? For example
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 2020-02-27 at 15:10 +0100, clime wrote: > Another thing to consider is whether we want a transparent build > system where all the description of what will happen when a spec file > is sent to it is included in the specfile itself or whether we want But we don't have that today: think of macros defined externally to the SPEC file and RPM which we use, think of "underspecified" build dependencies where what happens to be available and fulfil the BRs will be used and ultimately influence how the package is built. What happens when you build a package is already dependent on outside sources. > some auto-magic where e.g. 'Release' field or %changelog are > automatically inserted when missing (something like that is > impossible > right now due to rpm constraints but just to make a point). Again, this is opt-in. And the way this will be made opt-in is that people who want to use the feature use a macro, e.g. "%automatic_release", so that it is clear that some value will be filled in there. The idea isn't to have gutted spec files that don't build anywhere else than in our build system, one fixed requirement is that local builds using fedpkg or rpmbuild must continue to work. Nils -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-02-27 12:59, clime a écrit : Hi, can you, please, show an example of such package? I was searching through some golang packages because I was curious how it works but couldn't find an example A Go example: https://src.fedoraproject.org/rpms/golang-x-build A non-Go example: https://src.fedoraproject.org/rpms/impallari-dancing-script-fonts In both cases the spec calls a macro (not the same one in both specs) that causes one or several disprefixNUMBER variables to be computed The result then ends in Release via https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488 The font spec example calls the macro in redhat-rpm-config directly, the go example calls it through a Go-specific macro in fonts-rpm-macros Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Hi, On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > For the changelog, we believe we have a potential good solution: > - The changelog will be automatically generated using an external `changelog` > file as well as the commit history ... > If you wanted to edit the changelog, you would: > - Generate the changelog locally via a command like: > `fedpkg generate-changelog > changelog` > - Edit `changelog` as desired > - Commit and push the changes This means that a separate commit needs to be done *after* on top of the commits doing the actual changes. It's a bit disappointing, but on its own would not be too bad, since we can have fedpkg provide a higher level command that combines generate-changelog and build... One important feature will work: being able to cherry-pick commit between branches w/o spurious conflicts. As long as the "real" change to the spec file are done in separate commits, and the changelog commit is another commit on top, then when cherry-picking to another branch, the "real" commits would be cherry-pickable, and then the changelog commit would be recreated using the automation, OK. But it doesn't work quite as nicely with something similar: merging branches. A simple 'git merge' would result in conflicts. Also, if an amendment to the changelog is done, and the same change needs to be done in the changelog in a different branch, trying to cherry-pick the fix commit would most likely result in a merge conflict. Considering these three drawbacks (separate commit step and resulting log noise, merge conflicts), I'd very much hope for a solution where the changelog is never stored in the version control, and is always autogenerated at srpm creation time. We should never try to chery-pick commits that have changelog entries with actual date or e-v-r texts, since this will inevitably lead to merge conflicts. A different approach: - we have 'fedpkg generate-changelog' (which does all the git log extraction that was described, I think that part is OK), - the output from this command included in the srpm at srpm generation time and never committed to version control, - the output is annotated with the source commits hashes, so we can see where each line came from. At any time, the packager can run 'fedpkg generate-changelog' to see how the output looks like. If they see something they don't like, if the commits haven't been pushed out yet, they can immediately run 'git commit --amend' and recheck. If they have been pushed out, an override to the changelog could be committed as part of a commit message text. Git-changelog-suppress: adb42e or Git-changelog-suppress: adb42e..efefedeadb or Git-changelog-replace: adb42e Some other text with typos fixed that completely overrides whatever was generated from adb42e. or Git-changelog-append: adb42e (additional clarification for commit adb42e, e.g. bug #12345) This would have the following advantages: - git log is the sole source of truth - there are no cherry-pick or merge conflicts - there is no separate "now I'm done and I need to do another commit before building" thing. The assumption is that those Git-changelog-* macros would only be used occasionally, if the bad changelog entry was not noticed before pushing the changes out. (One nitpick: when cherry-picking between branches, hashes of the cherry-picked commits change, so "adb42e" in the example above would stop working. This is handled by using 'git cherry-pick -x'. 'fedpkg generate-changelog' would look at any hash in a "(cherry picked from commit ...)" line.) As how to hook this up with rpm, looking at https://pagure.io/packaging-committee/pull-request/942#comment-108542, a macro like %generate_changelog could be provided that would simply shell out to 'fedpkg generate-changelog'. I'll comment separate on the -release part. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 13:23, clime wrote: > > On Thu, 27 Feb 2020 at 12:42, David Kaufmann wrote: > > > > On Thu, Feb 27, 2020 at 12:21:48PM +0100, clime wrote: > > > On Thu, 27 Feb 2020 at 10:00, David Kaufmann wrote: > > >> Another idea would be generating a changelog-entry from git history when > > >> creating an update in bodhi, and there is no pre-existing > > >> changelog-entry for the current version. > > > > > > But Bodhi changelogs is not what user can read on his/her machine when > > > examining e.g. dnf check-update --changelogs. These are imho rpm > > > changelogs. So the rpm spec changelogs are the most important. > > > > Ah, sorry, what I meant to say is bodhi modifying the .spec file, adding > > the changelog line, and committing it, before building the package. > > (similarly like the releng commits when rebuilding stuff for new > > releases) > > > > But when I am thinking more about it I think it won't work, as bodhi > > would need to rebuild the package then. (afaik it does not, as koji > > built it already) > > Ah, right, I understand now. Thank you for the explanation! > > > > > An alternative might be something like modifying "fedpkg build" to check > > for missing changelogs and ask something like: > > > > "your package does not contain a changelog entry for . should > > we add the following entryies to the changelog for ?: > commit-messages>" > > > > maybe even with the option of modifying the lines? > > That would be possible, yes, but then fedpkg build would need > to create additional commit to maintain what user have inputed > so that it is remembered somewhere and present in the resulting rpm, > i.e. it would probably need to change repository and do a push during > the build launching. > > I had a similar idea but executed earlier. When` fedpkg tag` is invoked > (packagers usually don't invoke it today), a test-editor window is opened that > can be prepopulated from commit message but it could also be prepopulated > from an external file depending on configuration. Here user can compose > the final changelog record for the given release and this record is then > stored > in the newly created tag (in its annotation) and from there it is > automatically > (through a dedicated macro if used) rendered into spec file when needed > (e.g. when srpm is built). > > It requires one more step (i.e. fepkg commit -> fedpkg tag -> fedpkg > push -> fedpkg build) > but the build can be also launched automatically if a new tag is > pushed so the number > of steps would be the same. Actually, I thought about your idea doing it during `fedpkg build` (create the changelog record and push) and I started to like it more and more because people wouldn't need to run any extra commands compared to today when they would like to use the new macros. This could be only an opt-in feature (create_changelog_and_push_before_build or similar) cause some people will not want this. There are two ways how to create such record and push it without actually changing HEAD commit (which would be good cause otherwise we would be building different commit then we launched the command on, which would lead to confusion) 1) git notes: mutable text records that can be attached to any commit probably or tag in a git repo (see $ man git notes) and they have a separate history 2) git's annoted tags: tags with immutable text records and timestamps that can be signed and are checksummed. This is a direction of thought where the changelog is stored in git metadata. Proposal with the external changelog file in the repository is, instead, a direction where the changelog is stored in git data (well partially data and partially metadata because of the commit message fallback) - it's a bit different. Another thing to consider is whether we want a transparent build system where all the description of what will happen when a spec file is sent to it is included in the specfile itself or whether we want some auto-magic where e.g. 'Release' field or %changelog are automatically inserted when missing (something like that is impossible right now due to rpm constraints but just to make a point). > > A disadvantage is that the spec file is not standalone with respect to > important information. > It can be, however, made standalone at any time by "rendering" it, > i.e. process when the > macros are replaced by the actual information pulled from git (the > command to do it > could be e.g. fedpkg spec). > > > > > All the best, > > Astra > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 12:42, David Kaufmann wrote: > > On Thu, Feb 27, 2020 at 12:21:48PM +0100, clime wrote: > > On Thu, 27 Feb 2020 at 10:00, David Kaufmann wrote: > >> Another idea would be generating a changelog-entry from git history when > >> creating an update in bodhi, and there is no pre-existing > >> changelog-entry for the current version. > > > > But Bodhi changelogs is not what user can read on his/her machine when > > examining e.g. dnf check-update --changelogs. These are imho rpm > > changelogs. So the rpm spec changelogs are the most important. > > Ah, sorry, what I meant to say is bodhi modifying the .spec file, adding > the changelog line, and committing it, before building the package. > (similarly like the releng commits when rebuilding stuff for new > releases) > > But when I am thinking more about it I think it won't work, as bodhi > would need to rebuild the package then. (afaik it does not, as koji > built it already) Ah, right, I understand now. Thank you for the explanation! > > An alternative might be something like modifying "fedpkg build" to check > for missing changelogs and ask something like: > > "your package does not contain a changelog entry for . should > we add the following entryies to the changelog for ?: commit-messages>" > > maybe even with the option of modifying the lines? That would be possible, yes, but then fedpkg build would need to create additional commit to maintain what user have inputed so that it is remembered somewhere and present in the resulting rpm, i.e. it would probably need to change repository and do a push during the build launching. I had a similar idea but executed earlier. When` fedpkg tag` is invoked (packagers usually don't invoke it today), a test-editor window is opened that can be prepopulated from commit message but it could also be prepopulated from an external file depending on configuration. Here user can compose the final changelog record for the given release and this record is then stored in the newly created tag (in its annotation) and from there it is automatically (through a dedicated macro if used) rendered into spec file when needed (e.g. when srpm is built). It requires one more step (i.e. fepkg commit -> fedpkg tag -> fedpkg push -> fedpkg build) but the build can be also launched automatically if a new tag is pushed so the number of steps would be the same. A disadvantage is that the spec file is not standalone with respect to important information. It can be, however, made standalone at any time by "rendering" it, i.e. process when the macros are replaced by the actual information pulled from git (the command to do it could be e.g. fedpkg spec). > > All the best, > Astra > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 10:47, Nicolas Mailhot via devel
wrote:
>
> Le 2020-02-27 09:52, Miro Hrončok a écrit :
> > On 27. 02. 20 9:20, Pierre-Yves Chibon wrote:
> >>> How would that work with "complex" releases? For example release
> >>> containing
> >>> prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go
> >>> package
> >>> have no version, so depend heavily on the Release tag to signal what
> >>> is the
> >>> snapshot date and git commit packaged.
> >> This is something that we will need to investigate and clarify a
> >> little more,
> >> the answer may very well be: it won't, but let's investigate this
> >> first.
> >
> > There are three ways of of there I can think of ATM:
> >
> > 1. (as said by Pierre) make it opt-in only and don't handle this
> > 2. (as said by Neal) don't do this, use 0.1~beta.2-
> > 3. allow to keep the Release filed if it uses %{baserelease}
> >
> > %{baserelease} is already respected by rpmdev-bumpspec (and hence mass
> > rebuilds)
> >
> >
> > %global baserelease 8
> > Release:
> > iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
> >
> > bumpspec does:
> >
> > %global baserelease 9
> > Release:
> > iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
>
> Go (and other) packages do not set Version nor Release. They set
> distprefixNUMBER, which is defined in
> https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
Hello Nicolas,
can you, please, show an example of such package? I was searching through some
golang packages because I was curious how it works but couldn't find
an example (i.e. those packages had Version
and Release set, e.g.
https://src.fedoraproject.org/rpms/golang-github-anacrolix-missinggo/blob/master/f/golang-github-anacrolix-missinggo.spec).
>
> This automation must be kept or migrated to something else if people do
> not want to keep it ("just write bogus upstream versions manually is not
> a forward path)
>
> If changelog is automated, distprefixNUMBER should end up somewhere in
> it, because that’s important information not present in Version
>
> https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
> also defines some things equivalent to a distpostfix. They probably
> belong in changelog too.
>
> Regards,
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, Feb 27, 2020 at 12:21:48PM +0100, clime wrote: > On Thu, 27 Feb 2020 at 10:00, David Kaufmann wrote: >> Another idea would be generating a changelog-entry from git history when >> creating an update in bodhi, and there is no pre-existing >> changelog-entry for the current version. > > But Bodhi changelogs is not what user can read on his/her machine when > examining e.g. dnf check-update --changelogs. These are imho rpm > changelogs. So the rpm spec changelogs are the most important. Ah, sorry, what I meant to say is bodhi modifying the .spec file, adding the changelog line, and committing it, before building the package. (similarly like the releng commits when rebuilding stuff for new releases) But when I am thinking more about it I think it won't work, as bodhi would need to rebuild the package then. (afaik it does not, as koji built it already) An alternative might be something like modifying "fedpkg build" to check for missing changelogs and ask something like: "your package does not contain a changelog entry for . should we add the following entryies to the changelog for ?: " maybe even with the option of modifying the lines? All the best, Astra signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 10:00, David Kaufmann wrote:
>
> On Thu, Feb 27, 2020 at 08:08:49AM +0100, Dan Čermák wrote:
> > For the changelog: yes please, generate it from the commit log! They are
> > more or less the same for all my packages and I'm getting tired of copy
> > pasting the same text into %changelog and git commit.
>
> Another idea would be generating a changelog-entry from git history when
> creating an update in bodhi, and there is no pre-existing
> changelog-entry for the current version.
>
> This would not break the option to build a package from just one file.
> Having it all in one file is a big bonus to fedora, you can just
> download that file and build your package and not worry about the whole
> git-workflow, or having to check if you downloaded all files (not
> completely true in case of patch files).
Yes, I also thing having all relevant information in a single file is
a nice feature.
At the same time, usually you need to download/clone the whole repository
to build the package.
>
> It also would remove the need to copy messages from git log to the
> changelog. (some people complained about that - not only this message)
But Bodhi changelogs is not what user can read on his/her machine when
examining e.g. dnf check-update --changelogs. These are imho rpm
changelogs. So the rpm spec changelogs are the most important.
>
>
> I'm also not really a fan of "git as single source of truth" (has been
> mentioned a few times in this thread) - for me git is just a tool
> ensuring that git history was not modified.
>
> The *actual* source of truth is still the .spec file in the commit that is
> used to build the package - nobody is ever looking at old commits except
> for checking for malicious changes. (at least for spec files, with code
> it is useful for bisecting bugs).
>
>
> For end-users it might be useful to get the changelog alone (for that it
> does not matter if it is generated or copied from the .spec), but I
> never had any use for the changelog without the .spec file, as this
> gives me the context to the changes in the .spec file.
>
> But after all I do not care too much about how changelog is created, as
> long as the previous functionality is still preserved - my git log
> messages contain information about the .spec file changes while the
> changelog contains changes about the functionality of the package.
> ("what have I changed" (git) vs. "what has changed for you" (cl))
I think it is a great point. Commit message very often just say something
like new upstream release. In the final changelog it would be nice to
additionally e.g. provide a link to the upstream release summary - but
it's also more work.
Anyway, the messages will change if you really try to explain to user
what has changed. That's true.
>
> All the best,
> Astra
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 27. 02. 20 11:57, clime wrote:
If i understand correctly, this would rely on the locally undefined
%{baserelease} macro, which is later provided auto-magically by the
build system. Should this macro be populated also locally if not
defined?
Yes, by fedpkg. Now should the specs be buildable standalone? Possibly by
defining baserelease_default value in some SRPM macro package.
So the specs would have:
Release: beta.2.%{dynamic_release}
And %{dynamic_release} would be defined as:
%{?_buildsystem_dynamic_release}%{?!_buildsystem_dynamic_release:1}
And fedpkg would populate %_buildsystem_dynamic_release as it does with %fedora
etc.
We would just need to teach bumpspec (or more generally releng rebuild scripts)
to not try to manually bump releases with %{dynamic_release}.
I already mentioned it earlier in this thread but I think the use of
bare rpm macro is problematic because they either need to have git
context (metadata) around to evaluate or they would even need to do
remote network requests. Do you disagree?
No. I agree.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, 27 Feb 2020 at 09:53, Miro Hrončok wrote:
>
> On 27. 02. 20 9:20, Pierre-Yves Chibon wrote:
> >> How would that work with "complex" releases? For example release containing
> >> prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go package
> >> have no version, so depend heavily on the Release tag to signal what is the
> >> snapshot date and git commit packaged.
> > This is something that we will need to investigate and clarify a little
> > more,
> > the answer may very well be: it won't, but let's investigate this first.
>
> There are three ways of of there I can think of ATM:
>
> 1. (as said by Pierre) make it opt-in only and don't handle this
> 2. (as said by Neal) don't do this, use 0.1~beta.2-
> 3. allow to keep the Release filed if it uses %{baserelease}
>
> %{baserelease} is already respected by rpmdev-bumpspec (and hence mass
> rebuilds)
>
>
>%global baserelease 8
>Release: iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
>
> bumpspec does:
>
>%global baserelease 9
>Release: iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
>
> So we can reuse that and say: If the Release is manually defined in spec and
> uses %{baserelease} and %{baserelease} is not defined in spec, the automation
> sets the %{baserelease} value instead of setting the release directly.
Hello Miro,
interesting idea. I have some notes:
If i understand correctly, this would rely on the locally undefined
%{baserelease} macro, which is later provided auto-magically by the
build system. Should this macro be populated also locally if not
defined?
I already mentioned it earlier in this thread but I think the use of
bare rpm macro is problematic because they either need to have git
context (metadata) around to evaluate or they would even need to do
remote network requests. Do you disagree?
But anyway, what packager provides statically in the Release filed
without using a dedicated macros should imho stay there and not be
somehow magically deleted and overridden by build system. So the
solution might be simply:
Release: beta.2.<>
or it might be:
Release: <>
if the macro for the release computation needs to know the value of
the static prefix (that would be the case for git_release macro
defined in rpkg-util).
>
> --
> Miro Hrončok
> --
> Phone: +420777974800
> IRC: mhroncok
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-02-27 09:52, Miro Hrončok a écrit :
On 27. 02. 20 9:20, Pierre-Yves Chibon wrote:
How would that work with "complex" releases? For example release
containing
prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go
package
have no version, so depend heavily on the Release tag to signal what
is the
snapshot date and git commit packaged.
This is something that we will need to investigate and clarify a
little more,
the answer may very well be: it won't, but let's investigate this
first.
There are three ways of of there I can think of ATM:
1. (as said by Pierre) make it opt-in only and don't handle this
2. (as said by Neal) don't do this, use 0.1~beta.2-
3. allow to keep the Release filed if it uses %{baserelease}
%{baserelease} is already respected by rpmdev-bumpspec (and hence mass
rebuilds)
%global baserelease 8
Release:
iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
bumpspec does:
%global baserelease 9
Release:
iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
Go (and other) packages do not set Version nor Release. They set
distprefixNUMBER, which is defined in
https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
This automation must be kept or migrated to something else if people do
not want to keep it ("just write bogus upstream versions manually is not
a forward path)
If changelog is automated, distprefixNUMBER should end up somewhere in
it, because that’s important information not present in Version
https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-release.spec#_488
also defines some things equivalent to a distpostfix. They probably
belong in changelog too.
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-02-27 08:35, Nicolas Mailhot via devel a écrit : Le mercredi 26 février 2020 à 23:07 -0500, Neal Gompa a écrit : You don't use Release for upstream versioning, even for snapshots. For your examples: * 0-0.1.beta.2 -> 0~beta.2-1 * 0-0.1.20120225gitd6c789a -> 0~git20120225.d6c789a- Sorry but no You are attempting to redefine the meaning of Version (*upstream* version) to accommodate your release simplification plans To expand a little: as soon as you are in commit land, upstream ordering is effectively undefined. It is a packager (hopefully educated) judgment call to impose a Fedora packaging order over an upstream commit sequence, to decide if a commit looks like more a pre or post release, do decide to take the commit in a specific upstream branch or not, etc. Upstream commit ordering is undefined. Upstream did not pledge to evolve commits in a specific non-API breaking linear direction. They can make API breaks, revert them, reorder, fix Fedora problems in a topic branch, merge topic branches in no particular order, give up on topic branches and re-do them some other way, rebase, fork, etc. The Fedora packager may make a mistake in its educated guess and target a different branch or commit point in its next package release. Even if we had deep integration with upstream’s code repositories (which we have not today, and which would cause deep problems every time upstream shuffles its commits or re-hosts or forks, losing some history, or when an upstream repo URL goes away for another reason) we could not reliably automate an evr ordering from upstream’s commit history (we’d have to effectively epoch every single commit change, and that is moving the problem from release to version to epoch instead of merely moving it from releasez to version). Release to upstream commit mapping is a downstream Fedora invention. It has no place in Version. Please stop pretending upstream made the commit choice for us. If it was ready to make this choice, it would have tagged a specific release point. Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, Feb 27, 2020 at 08:08:49AM +0100, Dan Čermák wrote:
> For the changelog: yes please, generate it from the commit log! They are
> more or less the same for all my packages and I'm getting tired of copy
> pasting the same text into %changelog and git commit.
Another idea would be generating a changelog-entry from git history when
creating an update in bodhi, and there is no pre-existing
changelog-entry for the current version.
This would not break the option to build a package from just one file.
Having it all in one file is a big bonus to fedora, you can just
download that file and build your package and not worry about the whole
git-workflow, or having to check if you downloaded all files (not
completely true in case of patch files).
It also would remove the need to copy messages from git log to the
changelog. (some people complained about that - not only this message)
I'm also not really a fan of "git as single source of truth" (has been
mentioned a few times in this thread) - for me git is just a tool
ensuring that git history was not modified.
The *actual* source of truth is still the .spec file in the commit that is
used to build the package - nobody is ever looking at old commits except
for checking for malicious changes. (at least for spec files, with code
it is useful for bisecting bugs).
For end-users it might be useful to get the changelog alone (for that it
does not matter if it is generated or copied from the .spec), but I
never had any use for the changelog without the .spec file, as this
gives me the context to the changes in the .spec file.
But after all I do not care too much about how changelog is created, as
long as the previous functionality is still preserved - my git log
messages contain information about the .spec file changes while the
changelog contains changes about the functionality of the package.
("what have I changed" (git) vs. "what has changed for you" (cl))
All the best,
Astra
signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 27. 02. 20 9:20, Pierre-Yves Chibon wrote:
How would that work with "complex" releases? For example release containing
prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go package
have no version, so depend heavily on the Release tag to signal what is the
snapshot date and git commit packaged.
This is something that we will need to investigate and clarify a little more,
the answer may very well be: it won't, but let's investigate this first.
There are three ways of of there I can think of ATM:
1. (as said by Pierre) make it opt-in only and don't handle this
2. (as said by Neal) don't do this, use 0.1~beta.2-
3. allow to keep the Release filed if it uses %{baserelease}
%{baserelease} is already respected by rpmdev-bumpspec (and hence mass rebuilds)
%global baserelease 8
Release: iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
bumpspec does:
%global baserelease 9
Release: iamcrazy1234568andIknowit.%{baserelease}.whatnot~foo666%{?dist}
So we can reuse that and say: If the Release is manually defined in spec and
uses %{baserelease} and %{baserelease} is not defined in spec, the automation
sets the %{baserelease} value instead of setting the release directly.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Thu, Feb 27, 2020 at 02:04:41AM +0100, Robert-André Mauchin wrote:
> On Monday, 24 February 2020 17:48:36 CET Pierre-Yves Chibon wrote:
> > However, for the release field, we are struggling a little bit more, two
> > options are more appealing to us:
> >
> > A) The release field is automatically generated using two elements:
> > - the number of commits at this version
> > - the number of builds at this commit
> > - the dist-tag being added after them
> > The release field would thus look like:
> > ``.%{dist}``
> >
> > So we could have: (A, B, C and D being different commits)
> > A -- version: 0.9 -- release: ?
> >
> > |
> >
> > B -- version: 1.0 -- release: 1.0
> >
> > |
> >
> > C -- version: 1.0 -- release: 2.0
> >
> > |
> >
> > D -- version: 1.1 -- release: 1.0
> >
> > A rebuild of the commit D, would lead to a release 1.1 (assuming the first
> > one succeeded)/
> >
> > Pros:
> > - Easy to replicate locally
> > - Every changelog entry can be mapped to a `version-release` (No changes to
> > the packaging guidelines)
> > - Allows triggering two builds from the same commit without any manual
> > change (simplifies mass-rebuilds)
> > - Easy to link a certain build with a specific commit
> > - Easy to “guess” the next release value before triggering the build
> >
> > Cons:
> > - Old packages which are no longer receiving upstream releases may need
> > special care (for example if they are at the release -48 but only had
> > 45 commits leading to this)
> > - Relies on information from the build system for the build number (but
> > can be closely simulated locally since the info is only used for
> > rebuilds)
> > - Upgrade path may be problematic if Fn is upgraded to version X with
> > one commit while Fn-1 is upgrade to version X with 2 commits (or more)
> >
> > B) The release field is automatically generated taking existing builds in
> > all current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This
> > means for builds of the same epoch:version, find a new release that (if at
> > all possible) ensures upgradability from older Fedora versions to newer
> > ones, adhering to the structure of a release tag documented in the
> > Versioning Guidelines[1]. Going from the (RPM-wise) "latest build" that the
> > new one should surpass, this can mean bumping in the front (`pkgrel`) or
> > the back (`minorbump`).
> >
> > This allows packages from "very stable" upstreams who haven't released in
> > years to still benefit from automatically generated releases.
> >
> > The following examples would use a macro for the release field as outlined
> > in a separate document[2].
> >
> > Example #1 ("normal" release progression):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in F31: 1.0-2.fc32
> >
> >
> > Example #2 ("hotfix" in an older release, selected by an alternative macro
> > (or option) in the spec file):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in F30: 1.0-1.fc30.1
> >
> > Example #3 (pre-release, selected by an alternative macro (or option) in
> > the spec file):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in Rawhide: 3.0-0.1.20200224git1234abcd
> >
> >
> > Pros:
> > - Allows triggering two builds from the same commit without manual
> > intervention
> > - Emulates what many maintainers do manually today for most use cases
> > - Can cater for pre-releases (e.g.: by offering different macros or macro
> > options for the different use cases)
> >
> > Cons:
> > - Needs the build system for information about builds in this and other
> > Fedora versions
> > - Not easy to reproduce locally because we don’t have machine-consumable
> > knowledge about other builds in e.g. the dist-git repo
> > - Does not allow to generate changelog entries with `version-release`
> > information for all commits (and this will require a change in our
> > packaging guidelines)
> How would that work with "complex" releases? For example release containing
> prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go package
> have no version, so depend heavily on the Release tag to signal what is the
> snapshot date and git commit packaged.
This is something that we will need to investigate and clarify a little more,
the answer may very well be: it won't, but let's investigate this first.
We are very well aware that the proposal in its current form will not address
all packages. The idea being to improve the situation for some packages, see how
this works and see how we can grow this number of packages. Reading this thread
you can already see that some packagers are not interested in such a system
regardless of how it is, which is fine, we want it to be opt-in.
Pierre
___
devel mailing list -- devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le mercredi 26 février 2020 à 23:07 -0500, Neal Gompa a écrit :
>
> You don't use Release for upstream versioning, even for snapshots.
> For
> your examples:
>
> * 0-0.1.beta.2 -> 0~beta.2-1
> * 0-0.1.20120225gitd6c789a -> 0~git20120225.d6c789a-
Sorry but no
You are attempting to redefine the meaning of Version (*upstream*
version) to accomodate your release simplification plans
As I wrote the list months ago the upstream Version (Version,
%{version}) is zero 0 nil not 0~git20120225.d6c789a-1
The git20120225.d6c789a is a Fedora downstream construct
You can not break the data model with automation the way you break it
for humans. Automation does not care about your feelings. Automation
input is O as upstream Version. It can add downstream constructs to
Release, it can not rewrite Version (bad bad idea to attempt rewriting
a core rpm Tag in macros anyway)
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Hi list, Stephen John Smoogen writes: > On Mon, 24 Feb 2020 at 11:50, Pierre-Yves Chibon > wrote: > >> Good Morning Everyone, >> >> This topic has already been discussed a few times over the past month, but >> Adam >> Saleh, Nils Philippsen and myself have had the opportunity to invest some >> time >> on it with the hope of making the packager's life simpler as well as >> making it >> easier to build automation around our package maintenance. >> > > > Going through the various discussions, I think we are running into a > classic sausage factory problem.. everyone wants to have their favourite > meat, but the only way to make it work is mix them all together, grind it > up and come up with something else. I have to agree with Stephen here. As someone who has been heavily involved in building packages on OBS, where the Release field is generated automatically by the build server: I don't see a problem in automatically generating the release field, I would actually very much welcome it. But then none of my packages fall into the "Beaufort D'Ete" category, so I might be missing something here. For the changelog: yes please, generate it from the commit log! They are more or less the same for all my packages and I'm getting tired of copy pasting the same text into %changelog and git commit. Cheers, Dan signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Wed, Feb 26, 2020 at 8:05 PM Robert-André Mauchin wrote:
>
> On Monday, 24 February 2020 17:48:36 CET Pierre-Yves Chibon wrote:
> > However, for the release field, we are struggling a little bit more, two
> > options are more appealing to us:
> >
> > A) The release field is automatically generated using two elements:
> > - the number of commits at this version
> > - the number of builds at this commit
> > - the dist-tag being added after them
> > The release field would thus look like:
> > ``.%{dist}``
> >
> > So we could have: (A, B, C and D being different commits)
> > A -- version: 0.9 -- release: ?
> >
> > |
> >
> > B -- version: 1.0 -- release: 1.0
> >
> > |
> >
> > C -- version: 1.0 -- release: 2.0
> >
> > |
> >
> > D -- version: 1.1 -- release: 1.0
> >
> > A rebuild of the commit D, would lead to a release 1.1 (assuming the first
> > one succeeded)/
> >
> > Pros:
> > - Easy to replicate locally
> > - Every changelog entry can be mapped to a `version-release` (No changes to
> > the packaging guidelines)
> > - Allows triggering two builds from the same commit without any manual
> > change (simplifies mass-rebuilds)
> > - Easy to link a certain build with a specific commit
> > - Easy to “guess” the next release value before triggering the build
> >
> > Cons:
> > - Old packages which are no longer receiving upstream releases may need
> > special care (for example if they are at the release -48 but only had
> > 45 commits leading to this)
> > - Relies on information from the build system for the build number (but
> > can be closely simulated locally since the info is only used for
> > rebuilds)
> > - Upgrade path may be problematic if Fn is upgraded to version X with
> > one commit while Fn-1 is upgrade to version X with 2 commits (or more)
> >
> > B) The release field is automatically generated taking existing builds in
> > all current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This
> > means for builds of the same epoch:version, find a new release that (if at
> > all possible) ensures upgradability from older Fedora versions to newer
> > ones, adhering to the structure of a release tag documented in the
> > Versioning Guidelines[1]. Going from the (RPM-wise) "latest build" that the
> > new one should surpass, this can mean bumping in the front (`pkgrel`) or
> > the back (`minorbump`).
> >
> > This allows packages from "very stable" upstreams who haven't released in
> > years to still benefit from automatically generated releases.
> >
> > The following examples would use a macro for the release field as outlined
> > in a separate document[2].
> >
> > Example #1 ("normal" release progression):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in F31: 1.0-2.fc32
> >
> >
> > Example #2 ("hotfix" in an older release, selected by an alternative macro
> > (or option) in the spec file):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in F30: 1.0-1.fc30.1
> >
> > Example #3 (pre-release, selected by an alternative macro (or option) in
> > the spec file):
> > Rawhide has: 2.0-1.fc32
> > F31 has: 1.0-1.fc31
> > F30 has: 1.0-1.fc30
> >
> > Next release in Rawhide: 3.0-0.1.20200224git1234abcd
> >
> >
> > Pros:
> > - Allows triggering two builds from the same commit without manual
> > intervention
> > - Emulates what many maintainers do manually today for most use cases
> > - Can cater for pre-releases (e.g.: by offering different macros or macro
> > options for the different use cases)
> >
> > Cons:
> > - Needs the build system for information about builds in this and other
> > Fedora versions
> > - Not easy to reproduce locally because we don’t have machine-consumable
> > knowledge about other builds in e.g. the dist-git repo
> > - Does not allow to generate changelog entries with `version-release`
> > information for all commits (and this will require a change in our
> > packaging guidelines)
> >
> >
> > So these are the results of our current investigations, we are very much
> > eager to get your feedback on them and even more eager if you have ideas
> > on how to approach/solve some of the challenges mentioned here.
> >
> >
> > Looking forward for the discussion,
> >
> > Pierre
> > For Adam, Nils and myself
> >
> >
> >
>
> How would that work with "complex" releases? For example release containing
> prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go package
> have no version, so depend heavily on the Release tag to signal what is the
> snapshot date and git commit packaged.
>
You don't use Release for upstream versioning, even for snapshots. For
your examples:
* 0-0.1.beta.2 -> 0~beta.2-1
* 0-0.1.20120225gitd6c789a -> 0~git20120225.d6c789a-1
--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe
Re: Ideas and proposal for removing changelog and release fields from spec file
On Monday, 24 February 2020 17:48:36 CET Pierre-Yves Chibon wrote:
> However, for the release field, we are struggling a little bit more, two
> options are more appealing to us:
>
> A) The release field is automatically generated using two elements:
> - the number of commits at this version
> - the number of builds at this commit
> - the dist-tag being added after them
> The release field would thus look like:
> ``.%{dist}``
>
> So we could have: (A, B, C and D being different commits)
> A -- version: 0.9 -- release: ?
>
> |
>
> B -- version: 1.0 -- release: 1.0
>
> |
>
> C -- version: 1.0 -- release: 2.0
>
> |
>
> D -- version: 1.1 -- release: 1.0
>
> A rebuild of the commit D, would lead to a release 1.1 (assuming the first
> one succeeded)/
>
> Pros:
> - Easy to replicate locally
> - Every changelog entry can be mapped to a `version-release` (No changes to
> the packaging guidelines)
> - Allows triggering two builds from the same commit without any manual
> change (simplifies mass-rebuilds)
> - Easy to link a certain build with a specific commit
> - Easy to “guess” the next release value before triggering the build
>
> Cons:
> - Old packages which are no longer receiving upstream releases may need
> special care (for example if they are at the release -48 but only had
> 45 commits leading to this)
> - Relies on information from the build system for the build number (but
> can be closely simulated locally since the info is only used for
> rebuilds)
> - Upgrade path may be problematic if Fn is upgraded to version X with
> one commit while Fn-1 is upgrade to version X with 2 commits (or more)
>
> B) The release field is automatically generated taking existing builds in
> all current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This
> means for builds of the same epoch:version, find a new release that (if at
> all possible) ensures upgradability from older Fedora versions to newer
> ones, adhering to the structure of a release tag documented in the
> Versioning Guidelines[1]. Going from the (RPM-wise) "latest build" that the
> new one should surpass, this can mean bumping in the front (`pkgrel`) or
> the back (`minorbump`).
>
> This allows packages from "very stable" upstreams who haven't released in
> years to still benefit from automatically generated releases.
>
> The following examples would use a macro for the release field as outlined
> in a separate document[2].
>
> Example #1 ("normal" release progression):
> Rawhide has: 2.0-1.fc32
> F31 has: 1.0-1.fc31
> F30 has: 1.0-1.fc30
>
> Next release in F31: 1.0-2.fc32
>
>
> Example #2 ("hotfix" in an older release, selected by an alternative macro
> (or option) in the spec file):
> Rawhide has: 2.0-1.fc32
> F31 has: 1.0-1.fc31
> F30 has: 1.0-1.fc30
>
> Next release in F30: 1.0-1.fc30.1
>
> Example #3 (pre-release, selected by an alternative macro (or option) in
> the spec file):
> Rawhide has: 2.0-1.fc32
> F31 has: 1.0-1.fc31
> F30 has: 1.0-1.fc30
>
> Next release in Rawhide: 3.0-0.1.20200224git1234abcd
>
>
> Pros:
> - Allows triggering two builds from the same commit without manual
> intervention
> - Emulates what many maintainers do manually today for most use cases
> - Can cater for pre-releases (e.g.: by offering different macros or macro
> options for the different use cases)
>
> Cons:
> - Needs the build system for information about builds in this and other
> Fedora versions
> - Not easy to reproduce locally because we don’t have machine-consumable
> knowledge about other builds in e.g. the dist-git repo
> - Does not allow to generate changelog entries with `version-release`
> information for all commits (and this will require a change in our
> packaging guidelines)
>
>
> So these are the results of our current investigations, we are very much
> eager to get your feedback on them and even more eager if you have ideas
> on how to approach/solve some of the challenges mentioned here.
>
>
> Looking forward for the discussion,
>
> Pierre
> For Adam, Nils and myself
>
>
>
How would that work with "complex" releases? For example release containing
prerelease info like 0.1.beta.2 or 0.1.20120225gitd6c789a? Many Go package
have no version, so depend heavily on the Release tag to signal what is the
snapshot date and git commit packaged.
Best regards,
Robert-André
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 06:13:21PM +0100, Miro Hrončok wrote: > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > >However, for the release field, we are struggling a little bit more, two > >options > >are more appealing to us: > > Can we please have a "git is the only source of truth" version of > this? I.e. "Compute the release field from the number of commits > since the last version change" in the document. It seem to only have > one con (breaks if two builds are triggered from the same commit) > which is the status quo. This. And a "just use the git changelog" feature as well. In fact both should be the default. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Wed, 2020-02-26 at 11:29 +0100, Nicolas Mailhot via devel wrote: > Le 2020-02-26 11:14, Nils Philippsen a écrit : > > > Well, if we officially were to break with the upgrade path > > constraints, > > we'd have to document this. While we're at it, we should then > > document > > that Rawhide users should use "dnf distro-sync". > > That won’t work because (for example) rawhide is pulling forward > *right* > *now*, while F32 is not finished, so you have a *huge* time windows > during which there is nothing to distro-sync with, and you still need > to > keep compat between rawhide and future-stable numbering. Heh, I wasn't volunteering to push this. If you ask me, we should stick with having a clean upgrade path. Nils -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, 24 Feb 2020 at 11:50, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > This topic has already been discussed a few times over the past month, but > Adam > Saleh, Nils Philippsen and myself have had the opportunity to invest some > time > on it with the hope of making the packager's life simpler as well as > making it > easier to build automation around our package maintenance. > Going through the various discussions, I think we are running into a classic sausage factory problem.. everyone wants to have their favourite meat, but the only way to make it work is mix them all together, grind it up and come up with something else. Would it make sense to classify packages into 3 different groupings (like cheese)? Processed cheese: packages which are automatically built with a processed tool that updates most of the items. Just gets a bit of work every now and then from a packager afterwords for major changes or side-cases found. Everything in the package is fully automated with build tools filling in all the needed data for it. Cheddar cheese: packages which need more work and care, but are still pretty 'standard'... some may be sharper and need some aging but might be ok with some automated parts (%changelog and %release and a couple of things). Beaufort D'Ete (etc etc): packages which need a lot of work on the side of the maintainer to make it work for multiple reasons. Each packager has their own way of dealing with these packages which makes each a regional cheese. Currently a lot of packagers have to spend their time making cheeze-wiz packages so that they can even get to working on their specialist cheese. But because they are focused on wanting to get that specialist package out they are worried that any change will make all packages cheeze-wiz. I am wondering if we just make that part of the system separate with people knowing that they can label a package into a specific mental bucket, they can focus their energies on their Comté vieux. -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 26. 02. 20 v 12:16 clime napsal(a):
> Few more notes:
> - having just: Release: means every commit bumps
> release and hence every commit should likely generate a new record
> into changelog => changelog basically becomes dumped `git log` (in
> rpm-compatible format) - i.e. there is no capability to group multiple
> changes into one changelog record (grouping them implicitly by version
> changes will be tricky because changelogs will become mutable)
If you care, you'll be able to manually modify the changelog. But I
agree that some automatic grouping would be nice. May be there could
also be grouping hint, similarly to `RPM-Changelog: exclude`.
Vít
> - it doesn't help in jumping from specific package name to a specific
> commit in dist-git unless the macro also generates short git hash into
> the release but it also doesn't make things worse compared to now
> - for the solution with external changelog file with the commit
> fallback - it might be slightly confusing for someone who wants to
> look through the changes for a package in dist-git - at first, one
> should look at commits but from a certain commit switch to the file =>
> probably everyone will just stick to reading the commits
>
> I very much like simplicity of the approach but i think it is good to
> realize that the proposed solution is a direct translation of: "RPM
> Changelog and Release, you are not needed. Stand aside and don't make
> any problems!" If this is what packagers usually want, then it is a
> good solution. I think it also depends on what rpm consumers generally
> want.
>
> Then from the document:
> Get the release field from the annotation of the git tag
> (-) requires manual work on behalf of the maintainer
>
> That doesn't need to be the case with a client-side tooling
> that generates release automatically into a tag name.
>
> (-) Fragile: this means parsing using a regex the git annotation to
> extract the release information
>
> I would suggest putting release directly into a tag name, not into its
> annotation (i.e. subject or body). Basically tag names = NVRs.
>
> There is nothing fragile on parsing release from such name. In python,
> it is just standard rsplit('-', 1) and taking the last component.
>
> There are two issues here:
> 1) epochs: they will need to be put into the tag-name as /NVR
> because tag names cannot contain colons
> 2) tilde for prereleases: git tags cannot contain tilde character
> ('~'). But i personally believe we could find better ways to mark a
> prerelease. It used to be forbidden in Fedora...
>
> clime
>
> On Tue, 25 Feb 2020 at 22:20, James Cassell
> wrote:
>>
>> On Tue, Feb 25, 2020, at 2:53 PM, Matthew Miller wrote:
>>> On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote:
So these are the results of our current investigations, we are very much
eager
to get your feedback on them and even more eager if you have ideas on how
to
approach/solve some of the challenges mentioned here.
>>> This all sounds great. I'd also love for there to be a standard way of
>>> tagging specific git commit log messages as meant for user consumption, and
>>> use that to prepopulate the bodhi release notes field (with an eventual eye
>>> towards making this the single source of Fedora package change information).
>>>
>> Indeed, it is unfortunate that the Bodhi info for EOL releases is currently
>> lost.
>>
>> V/r,
>> James Cassell
>> ___
>> devel mailing list -- devel@lists.fedoraproject.org
>> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 25. 02. 20 v 15:03 Pierre-Yves Chibon napsal(a): > On Tue, Feb 25, 2020 at 12:59:39PM +0100, Vít Ondruch wrote: > > [About the release field] > >> I am not really sure about this. How do you envision this is going to be >> implemented? Is there going to be "release" file, similarly to >> changelog, which would help me to override this? I just wanted to stress out that I believe it would be good idea to use similar approach for changelog as well as for release. I.e. both should be opt-in, stored externally, overridable, and the automation should kick in only if they were not touched by the last commit. Vít >> Because, currently, it >> seems that both methods are going to need a lot of information from the >> build system. > The first method using number of commits and number of builds, pull limited > information from the build system (just: how many builds succeed before this > one > with this version and release?). > The second method does rely heavily on the build system. > >> They will need to parse .spec file etc. I don't see this >> to be able to handle even a bit more complex stuff like e.g. >> https://src.fedoraproject.org/rpms/ruby/blob/master/f/ruby.spec#_26 > We've been trying to find something that fit the majority of the packages but > we > are well aware that there are and will always be some special snowflakes that > will not be able to adhere to this. > I'd be happy if this works makes life/packaging easier for 50% of our packages > (I'd even probably be happy if it's 20%). > Once we have the simple case out and we can test it, see how it behaves, what > its limits are, we can start building on more complex cases, but I don't > really > believe on building the perfect solution in one go. > > > Pierre > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Few more notes:
- having just: Release: means every commit bumps
release and hence every commit should likely generate a new record
into changelog => changelog basically becomes dumped `git log` (in
rpm-compatible format) - i.e. there is no capability to group multiple
changes into one changelog record (grouping them implicitly by version
changes will be tricky because changelogs will become mutable)
- it doesn't help in jumping from specific package name to a specific
commit in dist-git unless the macro also generates short git hash into
the release but it also doesn't make things worse compared to now
- for the solution with external changelog file with the commit
fallback - it might be slightly confusing for someone who wants to
look through the changes for a package in dist-git - at first, one
should look at commits but from a certain commit switch to the file =>
probably everyone will just stick to reading the commits
I very much like simplicity of the approach but i think it is good to
realize that the proposed solution is a direct translation of: "RPM
Changelog and Release, you are not needed. Stand aside and don't make
any problems!" If this is what packagers usually want, then it is a
good solution. I think it also depends on what rpm consumers generally
want.
Then from the document:
Get the release field from the annotation of the git tag
(-) requires manual work on behalf of the maintainer
That doesn't need to be the case with a client-side tooling
that generates release automatically into a tag name.
(-) Fragile: this means parsing using a regex the git annotation to
extract the release information
I would suggest putting release directly into a tag name, not into its
annotation (i.e. subject or body). Basically tag names = NVRs.
There is nothing fragile on parsing release from such name. In python,
it is just standard rsplit('-', 1) and taking the last component.
There are two issues here:
1) epochs: they will need to be put into the tag-name as /NVR
because tag names cannot contain colons
2) tilde for prereleases: git tags cannot contain tilde character
('~'). But i personally believe we could find better ways to mark a
prerelease. It used to be forbidden in Fedora...
clime
On Tue, 25 Feb 2020 at 22:20, James Cassell wrote:
>
>
> On Tue, Feb 25, 2020, at 2:53 PM, Matthew Miller wrote:
> > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote:
> > > So these are the results of our current investigations, we are very much
> > > eager
> > > to get your feedback on them and even more eager if you have ideas on how
> > > to
> > > approach/solve some of the challenges mentioned here.
> >
> > This all sounds great. I'd also love for there to be a standard way of
> > tagging specific git commit log messages as meant for user consumption, and
> > use that to prepopulate the bodhi release notes field (with an eventual eye
> > towards making this the single source of Fedora package change information).
> >
>
> Indeed, it is unfortunate that the Bodhi info for EOL releases is currently
> lost.
>
> V/r,
> James Cassell
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-02-26 11:14, Nils Philippsen a écrit : Well, if we officially were to break with the upgrade path constraints, we'd have to document this. While we're at it, we should then document that Rawhide users should use "dnf distro-sync". That won’t work because (for example) rawhide is pulling forward *right* *now*, while F32 is not finished, so you have a *huge* time windows during which there is nothing to distro-sync with, and you still need to keep compat between rawhide and future-stable numbering. If you don’t keep compat you can not solve post-beta-freeze issues by pulling in future-stable the rawhide packages that solve those issues. And there will be some of those. Otherwise beta would be the same thing as definite release. Reagrds, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, 2020-02-24 at 18:13 +0100, Miro Hrončok wrote: > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > > However, for the release field, we are struggling a little bit > > more, two options > > are more appealing to us: > > Can we please have a "git is the only source of truth" version of > this? I.e. > "Compute the release field from the number of commits since the last > version > change" in the document. It seem to only have one con (breaks if two > builds are > triggered from the same commit) which is the status quo. The <#commits>.<#builds> approach also doesn't address "snapshot" prereleases, i.e. which can't be dealt with having appending "~prereleasename" to the version, which is a second con (or a "pro" of the "try to emulate traditional manual release numbers" approach, if you will). Nils -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 2020-02-25 at 17:45 +0100, Miro Hrončok wrote: > On 25. 02. 20 15:27, Fabio Valentini wrote: > > Side note: I've been meaning to propose dropping Epoch because of > > this > > "we don't care about upgrade path anymore", but I've not gotten > > around > > to do that yet We still need Epoch to have the ability to back-pedal on a (hypothetical) horribly broken version update in a stable release which wasn't caught during testing. > Unfortunately, that breaks rawhide users. Well, if we officially were to break with the upgrade path constraints, we'd have to document this. While we're at it, we should then document that Rawhide users should use "dnf distro-sync". Nils -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint: D0C1 1576 CDA6 5B6E BBAE 95B2 7D53 7FCA E9F6 395D old: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020, at 2:53 PM, Matthew Miller wrote: > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > > So these are the results of our current investigations, we are very much > > eager > > to get your feedback on them and even more eager if you have ideas on how to > > approach/solve some of the challenges mentioned here. > > This all sounds great. I'd also love for there to be a standard way of > tagging specific git commit log messages as meant for user consumption, and > use that to prepopulate the bodhi release notes field (with an eventual eye > towards making this the single source of Fedora package change information). > Indeed, it is unfortunate that the Bodhi info for EOL releases is currently lost. V/r, James Cassell ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > So these are the results of our current investigations, we are very much eager > to get your feedback on them and even more eager if you have ideas on how to > approach/solve some of the challenges mentioned here. This all sounds great. I'd also love for there to be a standard way of tagging specific git commit log messages as meant for user consumption, and use that to prepopulate the bodhi release notes field (with an eventual eye towards making this the single source of Fedora package change information). -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 25. 02. 20 15:27, Fabio Valentini wrote: Side note: I've been meaning to propose dropping Epoch because of this "we don't care about upgrade path anymore", but I've not gotten around to do that yet Unfortunately, that breaks rawhide users. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 25. 02. 20 9:50, Pierre-Yves Chibon wrote: Upgrade path may be problematic if you update Fn to a version in less commit than the update for Fn-1 (ie: you update F32 to 1.0 in 1 commit and update F31 to 1.0 in 2 commits, suddenly you have F32 with 1.0-1 and F31 with 1.0-2). I don't consider that an issue. It's not super elegant, but since we do distro-sync on upgrades, it shuld be fine. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 3:12 PM Pierre-Yves Chibon wrote: > > On Tue, Feb 25, 2020 at 02:55:37PM +0100, Fabio Valentini wrote: > > On Tue, Feb 25, 2020 at 2:47 PM Remi Collet > > wrote: > > > > > > Le 24/02/2020 à 17:48, Pierre-Yves Chibon a écrit : > > > > > > > - You can easily opt-in by using the macros > > > > > > Please keep opt-in as a mandatory need for such a change. > > > > > > > > > To be clear, I will be (perhaps the only) one to not use it. > > > > > > > > > For now spec file are self-contained, which is nice. > > > > > > I don't like the idea of generated / external stuff related > > > to "storage" or "build system" > > > > > > > > > Sorry, to be again the old bad guy which don't like changes. > > > > > > Remi > > > > FWIW, I agree. Maybe I'm getting old as well >:-D > > > > I don't think it's a good idea to use any information from outside the > > dist-git repository as a source of truth for anything. > > The big benefit of only using the git repository as source of > > information is that it is immutable, reproducible, and cannot be > > changed after commits have been pushed. > > The git repository data is also available for working on packages > > *offline*, in contrast to having to ask koji for the number of builds > > since X ... > > The way I see it is this: > With the number of commits+number of build idea, you get the same results > locally and in bodhi. > Locally fedpkg build or rpmbuild -ba will override the existing RPM > In koji, it will simply append a .1 to the release to avoid overriding the > existing RPM. > But the content and release, except for two characters, will be the same. (snip) > That being said, there seems to be a consensus forming about wanting to rely > only on number of commits (though, we still have the upgrade path issue to > sort > out). Hi Pierre, After reporting a few upgrade path bugs for (I think) fedora 28 and 29, I was told that "we don't care about upgrade path anymore", since "dnf system-upgrade" operates in "distro-sync" mode by default, since a few releases ago. So I don't see upgrade path as a (big) concern here. There may be package downgrades at system-upgrade time, but that's already the case today - most of the time because either people forget to build for fedora-branched after the branch point, or because they forget to submit bodhi updates after update-testing activation point. Whereas those two are "real" downgrades, any downgrades caused by the new commit counting would only be "downgrade by number but upgrade in content". Fabio Side note: I've been meaning to propose dropping Epoch because of this "we don't care about upgrade path anymore", but I've not gotten around to do that yet ️ > Pierre > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 02:55:37PM +0100, Fabio Valentini wrote: > On Tue, Feb 25, 2020 at 2:47 PM Remi Collet wrote: > > > > Le 24/02/2020 à 17:48, Pierre-Yves Chibon a écrit : > > > > > - You can easily opt-in by using the macros > > > > Please keep opt-in as a mandatory need for such a change. > > > > > > To be clear, I will be (perhaps the only) one to not use it. > > > > > > For now spec file are self-contained, which is nice. > > > > I don't like the idea of generated / external stuff related > > to "storage" or "build system" > > > > > > Sorry, to be again the old bad guy which don't like changes. > > > > Remi > > FWIW, I agree. Maybe I'm getting old as well >:-D > > I don't think it's a good idea to use any information from outside the > dist-git repository as a source of truth for anything. > The big benefit of only using the git repository as source of > information is that it is immutable, reproducible, and cannot be > changed after commits have been pushed. > The git repository data is also available for working on packages > *offline*, in contrast to having to ask koji for the number of builds > since X ... The way I see it is this: With the number of commits+number of build idea, you get the same results locally and in bodhi. Locally fedpkg build or rpmbuild -ba will override the existing RPM In koji, it will simply append a .1 to the release to avoid overriding the existing RPM. But the content and release, except for two characters, will be the same. That being said, there seems to be a consensus forming about wanting to rely only on number of commits (though, we still have the upgrade path issue to sort out). Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 12:59:39PM +0100, Vít Ondruch wrote: > > Dne 24. 02. 20 v 17:48 Pierre-Yves Chibon napsal(a): > > Good Morning Everyone, > > > > This topic has already been discussed a few times over the past month, but > > Adam > > Saleh, Nils Philippsen and myself have had the opportunity to invest some > > time > > on it with the hope of making the packager's life simpler as well as making > > it > > easier to build automation around our package maintenance. > > > > We have investigated a few ideas, the full list with their pros and cons > > can be > > found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev > > If you have any questions about some of these, please ask them, we'll be > > happy > > to answer them and potentially complete this document. > > > > > > For both the release and the changelog fields we believe using RPM macros > > would > > satisfy the requirements we have for opting-in/out: > > - You can easily opt-in by using the macros > > - You can easily opt-out by removing the macros from your spec file > > > > > > For the changelog, we believe we have a potential good solution: > > - The changelog will be automatically generated using an external > > `changelog` > > file as well as the commit history > > - When you opt-in, you will simply move the existing changelog to an > > external > > file `changelog`, which is placed in the dist-git repo and add the > > appropriate macro. > > > I assume you are aware about my PR: > > > https://pagure.io/packaging-committee/pull-request/942 > > > > - Upon building, the macro will generate the changelog using all the > > commits > > of the repo up to the last commit touching the `changelog` file. > > > I proposed, that the changelog file is either included in the repository > or it is not included (probably listed in .gitignore). So it is either > used as it is or if later, then it is generated. > > You propose mixture if I understand it correctly. I.e. the changelog > file is always present in the dist-git and it is always is used. But if > there are more recent commits without changelog modifications, they are > prepended to the changelog, but the changelog file itself stays > untouched. E.g. if my latest commit modifies the changelog, the > changelog as it is present in the repo will be used without any > modifications. > > I like it. It is opt-in, because I have to place some macro into .spec > file and it is even more or less bacward compatible, because the > automation kicks in only if I don't maintain the changelog manually. That is correct. I was aware of the PR to the FPC which actually contributed to this idea (figuring out the last commit that changed the `changelog` file is way easier than the last commit that changed %changelog in the spec file). So thanks for this! [About the release field] > I am not really sure about this. How do you envision this is going to be > implemented? Is there going to be "release" file, similarly to > changelog, which would help me to override this? Because, currently, it > seems that both methods are going to need a lot of information from the > build system. The first method using number of commits and number of builds, pull limited information from the build system (just: how many builds succeed before this one with this version and release?). The second method does rely heavily on the build system. > They will need to parse .spec file etc. I don't see this > to be able to handle even a bit more complex stuff like e.g. > https://src.fedoraproject.org/rpms/ruby/blob/master/f/ruby.spec#_26 We've been trying to find something that fit the majority of the packages but we are well aware that there are and will always be some special snowflakes that will not be able to adhere to this. I'd be happy if this works makes life/packaging easier for 50% of our packages (I'd even probably be happy if it's 20%). Once we have the simple case out and we can test it, see how it behaves, what its limits are, we can start building on more complex cases, but I don't really believe on building the perfect solution in one go. Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 2:47 PM Remi Collet wrote: > > Le 24/02/2020 à 17:48, Pierre-Yves Chibon a écrit : > > > - You can easily opt-in by using the macros > > Please keep opt-in as a mandatory need for such a change. > > > To be clear, I will be (perhaps the only) one to not use it. > > > For now spec file are self-contained, which is nice. > > I don't like the idea of generated / external stuff related > to "storage" or "build system" > > > Sorry, to be again the old bad guy which don't like changes. > > Remi FWIW, I agree. Maybe I'm getting old as well >:-D I don't think it's a good idea to use any information from outside the dist-git repository as a source of truth for anything. The big benefit of only using the git repository as source of information is that it is immutable, reproducible, and cannot be changed after commits have been pushed. The git repository data is also available for working on packages *offline*, in contrast to having to ask koji for the number of builds since X ... Fabio > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 24/02/2020 à 17:48, Pierre-Yves Chibon a écrit : > - You can easily opt-in by using the macros Please keep opt-in as a mandatory need for such a change. To be clear, I will be (perhaps the only) one to not use it. For now spec file are self-contained, which is nice. I don't like the idea of generated / external stuff related to "storage" or "build system" Sorry, to be again the old bad guy which don't like changes. Remi ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 25. 02. 20 v 9:55 Pierre-Yves Chibon napsal(a): > On Mon, Feb 24, 2020 at 10:05:31PM +0100, Miroslav Suchý wrote: >> Dne 24. 02. 20 v 18:13 Miro Hrončok napsal(a): >>> Can we please have a "git is the only source of truth" version of this? >>> I.e. "Compute the release field from the number >>> of commits since the last version change" in the document. It seem to only >>> have one con (breaks if two builds are >>> triggered from the same commit) which is the status quo. >>> >> +1 >> >> Or the combination of: >> Get the release field from the annotation of the git tag >> and >> Get the release field using the number of tags since the last version >> change >> If the regexp does not match, just count the tags (or the commits). > We addressed git tags in the hackmd doc and we were not very thrilled with the > idea. It seems fragile (parsing/regex) and if the changelog is included in the > git tag, packagers still have to maintain 3 different changelogs. We should keep git tags for Bodhi updates O:-) Vít > >> And BTW I like the changelog proposal. > Thanks :) > It's a bit of a tricky situation since we tend to consider both release and > changelog separately but they are also very much related since the release > appears in the changelog. > So if we want to keep the current changelog format, we need to consider them > together (of we accept to change the structure of the changelog) > > > Pierre > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 24. 02. 20 v 17:48 Pierre-Yves Chibon napsal(a):
> Good Morning Everyone,
>
> This topic has already been discussed a few times over the past month, but
> Adam
> Saleh, Nils Philippsen and myself have had the opportunity to invest some time
> on it with the hope of making the packager's life simpler as well as making it
> easier to build automation around our package maintenance.
>
> We have investigated a few ideas, the full list with their pros and cons can
> be
> found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
> If you have any questions about some of these, please ask them, we'll be happy
> to answer them and potentially complete this document.
>
>
> For both the release and the changelog fields we believe using RPM macros
> would
> satisfy the requirements we have for opting-in/out:
> - You can easily opt-in by using the macros
> - You can easily opt-out by removing the macros from your spec file
>
>
> For the changelog, we believe we have a potential good solution:
> - The changelog will be automatically generated using an external `changelog`
> file as well as the commit history
> - When you opt-in, you will simply move the existing changelog to an
> external
> file `changelog`, which is placed in the dist-git repo and add the
> appropriate macro.
I assume you are aware about my PR:
https://pagure.io/packaging-committee/pull-request/942
> - Upon building, the macro will generate the changelog using all the commits
> of the repo up to the last commit touching the `changelog` file.
I proposed, that the changelog file is either included in the repository
or it is not included (probably listed in .gitignore). So it is either
used as it is or if later, then it is generated.
You propose mixture if I understand it correctly. I.e. the changelog
file is always present in the dist-git and it is always is used. But if
there are more recent commits without changelog modifications, they are
prepended to the changelog, but the changelog file itself stays
untouched. E.g. if my latest commit modifies the changelog, the
changelog as it is present in the repo will be used without any
modifications.
I like it. It is opt-in, because I have to place some macro into .spec
file and it is even more or less bacward compatible, because the
automation kicks in only if I don't maintain the changelog manually.
> Of all
> these commits it will only consider the commits following these rules:
> - There are generally two approaches regarding what to include by default:
> 1. commits touching only the `sources`, `.gitignore`, `dead.package`
> files, `tests` folder will be ignored by default, i.e. a blacklist
> 2. only commits touching the spec file or patches will be included by
> default, i.e. a whitelist
> ==> Which approach do you think is better/will work in most cases?
> - empty commits (not touching any files) will be included on the
> assumption
> that this is their purpose
> - commits containing "magic keyword" (#changelog_exclude,
> #changelog_include?) will be ignored or included as the case may be
> - Finally the content of the changelog file specified will be appended to
> the
> changelog generated from the git history
>
>
> If you wanted to edit the changelog, you would:
> - Generate the changelog locally via a command like:
> `fedpkg generate-changelog > changelog`
This also helps to completely modify the changelog and possible fix some
past errors in git changelog or what not.
> - Edit `changelog` as desired
> - Commit and push the changes
>
> Since the macro will only consider the commits up to the first commit touching
> `changelog` (in other words, the macro will only consider the commits after
> this
> one) and include `changelog` file as is, your edits will appear in the RPM
> changelog as you want.
>
> One thing to note is that rebuilds from the same commit will have the same
> %changelog, they will not get a new entry. If you want a new changelog entry,
> you have to create a new commit with the desired changelog entry as commit
> message (the commit itself can be empty).
>
>
>
> However, for the release field, we are struggling a little bit more, two
> options
> are more appealing to us:
>
> A) The release field is automatically generated using two elements:
> - the number of commits at this version
> - the number of builds at this commit
> - the dist-tag being added after them
> The release field would thus look like:
> ``.%{dist}``
>
> So we could have: (A, B, C and D being different commits)
> A -- version: 0.9 -- release: ?
> |
> B -- version: 1.0 -- release: 1.0
> |
> C -- version: 1.0 -- release: 2.0
> |
> D -- version: 1.1 -- release: 1.0
>
> A rebuild of the commit D, would lead to a release 1.1 (assuming the first one
> succeeded)/
>
> Pros:
> - Easy to replicate locally
> - Every changelog entry can be mapped to a `version-release`
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 25 Feb 2020 at 12:47, clime wrote:
>
> Hey pingou!
>
> On Tue, 25 Feb 2020 at 10:26, Pierre-Yves Chibon wrote:
> >
> > On Tue, Feb 25, 2020 at 12:18:24AM +0100, clime wrote:
> > > Hello!
> > >
> > > What is the point of including number of builds into release? I think
> > > the Miro's approach solves it.
> > > Or is there any other problem except soname bumps?
> >
> > It makes it easier to do rebuilds which means it makes it easier and
> > simpler to
> > do mass-rebuilds.
> >
> > > Ad. document - annotated git tags:
> > > (-) Editing the changelog would mean allowing to remove the git tags,
> > > thus leading to potential issue in build reproducibility
> > >
> > > That doesn't need to be the case. In rpkg-util, this was resolved by
> > > introducing arguments since_tag and until_tag
> > > for git_changelog macro
> > > (https://docs.pagure.org/rpkg-util/macro_reference.html#git-macros).
> > > That's
> > > how it can be prevented for some annotated tag to contribute to changelog.
> > >
> > > Example:
> > >
> > > {{{ git_changelog since_tag=name-1.3-1 }}}
> > >
> > > * Mon Feb 24 2020 clime 1.2-1
> > > - manual changelog entry that is used instead of a tag annotation
> > >
> > > {{{ git_changelog until_tag=name-1.1-1 }}}
> >
> > Interesting idea.
> >
> > > Removing already pushed git tags is probably not a good idea anyway:
> > > https://git-scm.com/docs/git-tag#_on_re_tagging
> >
> > We very much agree that removing git tags is not a good idea, that's why we
> > listed it as a con :)
> >
> > > Ad. the following approach for calculating release:
> > > - Compute the release field from the number of commits since the last
> > > version change: -%{dist}
> > >
> > > I think it is a good idea. In rpkg-util, it is done similarly, except
> > > the release part has more subparts than just
> > > two (including %{dist}).
> >
> > If you make the build system provide the ${dirty_appendix} and drop the
> > ${pivot}
> > (because we want to generate the release, so there is no input specified),
> > you
> > get very close to what we described.
>
> ${dirty_appendix} is a part intended for development purposes, i.e. to be able
> to have versioned working tree when working locally without committing. It
> will be empty if working tree is clean, which should always be the
> case in a build system
> (there is more info here
> https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_9
> on this).
>
> I am guessing you think about number-of-builds as the
> ${dirty_appendix} ? That would
> need to be a normal rpm macro provided by a build system (similarly to
> %dist). Anyway,
> if the number-of-builds part is just to optimize mass rebuild process,
> I would try to look
> if we can tweak the build system to be able to do the mass rebuilds
> without making
> new commits back to dist-git and even without that %number-of-builds macro. In
> my opinion, it should be possible. There is already BUILDTIME in rpm headers
> that can be used to differentiate between newer and older rpm with otherwise
> the
> same nevr.
>
> In rpkg-util, ${pivot} (the main release number) is generated
> automatically (when the git_release
> macro is employed) but packager can also hard-set it at any time to
> any value. I think with
> the pivot number, the upgrade-path problem would be solved cause the
> commit-count appendix
> is only used as delta since the last pivot change, which is marked by
> an annotated tag in the git
> tree. So the commit (having an annotated tag on it) in older and newer
> Fedora release (branch)
> would generate an rpm with the same NVR and just the %dist decides
> then (there can
> also be two separate tags if %dist is used in the Release string and
> if it is configured
> not to be empty as i do it now rpkg-util :) but this is another discussion).
>
> The pivot is also useful for post/pre-releases and other cases described here:
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_more_complex_versioning
>
> It is then useful to have something that can be configured by hand.
>
> >
> > > The full description is here:
> > > https://pagure.io/rpkg-util/blob/master/f/man/rpkg_man_page.py#_262
> > > but the main difference is that it counts commits from the latest
> > > annotated tag which contains (in its name)
> > > the current version and the current release number which are extracted
> > > from the spec file when
> > > creating the tag unless they are specified manually on command line.
> > > Commit count is only appended
> > > to it if we build from commit which is on top of some annotated tag
> > > (i.e. it is itself untagged).
> > >
> > > Going by just a textual version change in a spec file might be slightly
> > > tricky.
> > > You would need to go through all the past commits that touched that spec
> > > file,
> > > keep checking these out and look if the version is changed when compared
> > > to the
> > > one parsed from the head commit. Possible though.
> >
> > This is
Re: Ideas and proposal for removing changelog and release fields from spec file
Hey pingou!
On Tue, 25 Feb 2020 at 10:26, Pierre-Yves Chibon wrote:
>
> On Tue, Feb 25, 2020 at 12:18:24AM +0100, clime wrote:
> > Hello!
> >
> > What is the point of including number of builds into release? I think
> > the Miro's approach solves it.
> > Or is there any other problem except soname bumps?
>
> It makes it easier to do rebuilds which means it makes it easier and simpler
> to
> do mass-rebuilds.
>
> > Ad. document - annotated git tags:
> > (-) Editing the changelog would mean allowing to remove the git tags,
> > thus leading to potential issue in build reproducibility
> >
> > That doesn't need to be the case. In rpkg-util, this was resolved by
> > introducing arguments since_tag and until_tag
> > for git_changelog macro
> > (https://docs.pagure.org/rpkg-util/macro_reference.html#git-macros).
> > That's
> > how it can be prevented for some annotated tag to contribute to changelog.
> >
> > Example:
> >
> > {{{ git_changelog since_tag=name-1.3-1 }}}
> >
> > * Mon Feb 24 2020 clime 1.2-1
> > - manual changelog entry that is used instead of a tag annotation
> >
> > {{{ git_changelog until_tag=name-1.1-1 }}}
>
> Interesting idea.
>
> > Removing already pushed git tags is probably not a good idea anyway:
> > https://git-scm.com/docs/git-tag#_on_re_tagging
>
> We very much agree that removing git tags is not a good idea, that's why we
> listed it as a con :)
>
> > Ad. the following approach for calculating release:
> > - Compute the release field from the number of commits since the last
> > version change: -%{dist}
> >
> > I think it is a good idea. In rpkg-util, it is done similarly, except
> > the release part has more subparts than just
> > two (including %{dist}).
>
> If you make the build system provide the ${dirty_appendix} and drop the
> ${pivot}
> (because we want to generate the release, so there is no input specified), you
> get very close to what we described.
${dirty_appendix} is a part intended for development purposes, i.e. to be able
to have versioned working tree when working locally without committing. It
will be empty if working tree is clean, which should always be the
case in a build system
(there is more info here
https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_9
on this).
I am guessing you think about number-of-builds as the
${dirty_appendix} ? That would
need to be a normal rpm macro provided by a build system (similarly to
%dist). Anyway,
if the number-of-builds part is just to optimize mass rebuild process,
I would try to look
if we can tweak the build system to be able to do the mass rebuilds
without making
new commits back to dist-git and even without that %number-of-builds macro. In
my opinion, it should be possible. There is already BUILDTIME in rpm headers
that can be used to differentiate between newer and older rpm with otherwise the
same nevr.
In rpkg-util, ${pivot} (the main release number) is generated
automatically (when the git_release
macro is employed) but packager can also hard-set it at any time to
any value. I think with
the pivot number, the upgrade-path problem would be solved cause the
commit-count appendix
is only used as delta since the last pivot change, which is marked by
an annotated tag in the git
tree. So the commit (having an annotated tag on it) in older and newer
Fedora release (branch)
would generate an rpm with the same NVR and just the %dist decides
then (there can
also be two separate tags if %dist is used in the Release string and
if it is configured
not to be empty as i do it now rpkg-util :) but this is another discussion).
The pivot is also useful for post/pre-releases and other cases described here:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_more_complex_versioning
It is then useful to have something that can be configured by hand.
>
> > The full description is here:
> > https://pagure.io/rpkg-util/blob/master/f/man/rpkg_man_page.py#_262
> > but the main difference is that it counts commits from the latest
> > annotated tag which contains (in its name)
> > the current version and the current release number which are extracted
> > from the spec file when
> > creating the tag unless they are specified manually on command line.
> > Commit count is only appended
> > to it if we build from commit which is on top of some annotated tag
> > (i.e. it is itself untagged).
> >
> > Going by just a textual version change in a spec file might be slightly
> > tricky.
> > You would need to go through all the past commits that touched that spec
> > file,
> > keep checking these out and look if the version is changed when compared to
> > the
> > one parsed from the head commit. Possible though.
>
> This is basically what I had in mind.
Yes, it takes slightly more computational time than being navigated by tags
themselves that can be used for version bumps as well as release bumps.
>
> > To go back to your original post:
> > > For both the release and the changelog fields we believe using RPM
Re: Ideas and proposal for removing changelog and release fields from spec file
Le 2020-02-25 10:24, Pierre-Yves Chibon a écrit :
If you make the build system provide the ${dirty_appendix} and drop the
${pivot}
(because we want to generate the release, so there is no input
specified), you
get very close to what we described.
BTW, regardless of how things up, we have existing logic that computes
dist prefixes (unpstream git commit numbers…) and appendixes
(bootstrap…).
The easy way to do things is to let things as they stand today and limit
the automation to the main release number, but that still requires
adding those things to changelog entries (if we want to trace them in
changelogs; we probably want because a project that does not releases
will be stuck at version 0 forever, only the distprefix disambiguates
what version 0 means)
OTOH, if people feel ambitious, and want to reexamine all the dist
prefix/appendix logic, there is quite a lot of work to audit the
hundreds of packages that use them today and define a migration plan and
logic. Because this amount of spec logic is not going to rewrite itself,
and expecting someone else (packagers) to do it will throw a non trivial
part of the distro under the bus.
(The correct way to do things would be to evolve rpm to have dedicated
fields for all the things we crammed into Release in the past. That’s a
non trivial endeavour.)
Regards,
--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 12:18:24AM +0100, clime wrote:
> Hello!
>
> What is the point of including number of builds into release? I think
> the Miro's approach solves it.
> Or is there any other problem except soname bumps?
It makes it easier to do rebuilds which means it makes it easier and simpler to
do mass-rebuilds.
> Ad. document - annotated git tags:
> (-) Editing the changelog would mean allowing to remove the git tags,
> thus leading to potential issue in build reproducibility
>
> That doesn't need to be the case. In rpkg-util, this was resolved by
> introducing arguments since_tag and until_tag
> for git_changelog macro
> (https://docs.pagure.org/rpkg-util/macro_reference.html#git-macros).
> That's
> how it can be prevented for some annotated tag to contribute to changelog.
>
> Example:
>
> {{{ git_changelog since_tag=name-1.3-1 }}}
>
> * Mon Feb 24 2020 clime 1.2-1
> - manual changelog entry that is used instead of a tag annotation
>
> {{{ git_changelog until_tag=name-1.1-1 }}}
Interesting idea.
> Removing already pushed git tags is probably not a good idea anyway:
> https://git-scm.com/docs/git-tag#_on_re_tagging
We very much agree that removing git tags is not a good idea, that's why we
listed it as a con :)
> Ad. the following approach for calculating release:
> - Compute the release field from the number of commits since the last
> version change: -%{dist}
>
> I think it is a good idea. In rpkg-util, it is done similarly, except
> the release part has more subparts than just
> two (including %{dist}).
If you make the build system provide the ${dirty_appendix} and drop the ${pivot}
(because we want to generate the release, so there is no input specified), you
get very close to what we described.
> The full description is here:
> https://pagure.io/rpkg-util/blob/master/f/man/rpkg_man_page.py#_262
> but the main difference is that it counts commits from the latest
> annotated tag which contains (in its name)
> the current version and the current release number which are extracted
> from the spec file when
> creating the tag unless they are specified manually on command line.
> Commit count is only appended
> to it if we build from commit which is on top of some annotated tag
> (i.e. it is itself untagged).
>
> Going by just a textual version change in a spec file might be slightly
> tricky.
> You would need to go through all the past commits that touched that spec file,
> keep checking these out and look if the version is changed when compared to
> the
> one parsed from the head commit. Possible though.
This is basically what I had in mind.
> To go back to your original post:
> > For both the release and the changelog fields we believe using RPM macros
> > would
> > satisfy the requirements we have for opting-in/out:
>
> By using such RPM macros, you would lose ability to rebuild srpms
> because it will
> be only possible to build them when the git context is present but not when
> they
> become a standalone thing. I.e. things like this will not work:
>
> https://github.com/rpm-software-management/mock/blob/cfe34c8a57/mock/py/mockbuild/backend.py#L270
>
> That's why I think that such macros should be of a different kind:
> macros that are computed
> once when srpm is created and result of which is put _verbatim_ into
> the spec file so that
> there is no (re)computation later when srpm is being built and when
> the git context is already
> missing.
We had a discussion with Neal about this topic on #fedora-apps yesterday and I
believe one of the outcome of this discussion was that we should indeed use this
approach, for the reasons you're mentioning.
Thanks for your feedbacks and thoughts (and links!)
Pierre
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, Feb 25, 2020 at 09:06:35AM +0100, Petr Pisar wrote: > On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > > - commits containing "magic keyword" (#changelog_exclude, > > #changelog_include?) will be ignored or included as the case may be > > Could we please use the usual git commit keyword syntax? I.e. the e-mail > header format (Signed-off-by:). E.g.: > > RPM-Changelog: include > RPM-Changelog: exclude > > > If you wanted to edit the changelog, you would: > > - Generate the changelog locally via a command like: > > `fedpkg generate-changelog > changelog` > > - Edit `changelog` as desired > > - Commit and push the changes > > > If the changelog file name is hard-coded, I recommend "fedpkg > generate-changelog" command to write directly into the "changelog" file. Two good ideas, thanks :) Pierre signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 10:05:31PM +0100, Miroslav Suchý wrote: > Dne 24. 02. 20 v 18:13 Miro Hrončok napsal(a): > > Can we please have a "git is the only source of truth" version of this? > > I.e. "Compute the release field from the number > > of commits since the last version change" in the document. It seem to only > > have one con (breaks if two builds are > > triggered from the same commit) which is the status quo. > > > > +1 > > Or the combination of: > Get the release field from the annotation of the git tag > and > Get the release field using the number of tags since the last version change > If the regexp does not match, just count the tags (or the commits). We addressed git tags in the hackmd doc and we were not very thrilled with the idea. It seems fragile (parsing/regex) and if the changelog is included in the git tag, packagers still have to maintain 3 different changelogs. > And BTW I like the changelog proposal. Thanks :) It's a bit of a tricky situation since we tend to consider both release and changelog separately but they are also very much related since the release appears in the changelog. So if we want to keep the current changelog format, we need to consider them together (of we accept to change the structure of the changelog) Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 07:30:15PM +0100, Nicolas Mailhot via devel wrote: > Le lundi 24 février 2020 à 18:13 +0100, Miro Hrončok a écrit : > > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > > > However, for the release field, we are struggling a little bit > > > more, two options > > > are more appealing to us: > > > > Can we please have a "git is the only source of truth" version of > > this? > > Packages have a life in review before they get a Fedora git (sometimes > the life can be years in copr, OBS or elsewhere as our review process > is none too fast) I guess this means the first version that lands in Fedora will not have a -1 release, is that a real problem? Don't we already have this anyway since often we bump the release and changelog during the review process to address the reviewer's comments? Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 06:13:21PM +0100, Miro Hrončok wrote: > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > > However, for the release field, we are struggling a little bit more, two > > options > > are more appealing to us: > > Can we please have a "git is the only source of truth" version of this? I.e. > "Compute the release field from the number of commits since the last version > change" in the document. It seem to only have one con (breaks if two builds > are triggered from the same commit) which is the status quo. It has another con which is indeed not mentioned in this section but in the next one (n_commits+n_build) (I've fixed that). Upgrade path may be problematic if you update Fn to a version in less commit than the update for Fn-1 (ie: you update F32 to 1.0 in 1 commit and update F31 to 1.0 in 2 commits, suddenly you have F32 with 1.0-1 and F31 with 1.0-2). > If you need to rebuild for a libpingouisawesome soname bump, you just do an > empty commit with the explanation. > > If you merge that empty commit to a branch that did not need it, it would > have a bogus changelog entry (status quo). If you care, you would not merge > but cherry-pick anything thta comes next (which is now much easier given the > benefit of not having the %changelog and release). > > With the proposed solution that includes "successful build count" you always > bump and build even if it is not needed and also you make the release number > depend on a specific build system, which I think is kinda wrong. One of the benefit of the approach using number of commits + number of builds (when it's > 1) is that mass-rebuild could be made even simpler and faster. Instead of doing git commit + koji build, you would simply do koji build. Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Tue, 25 Feb 2020 at 09:31, clime wrote: > > Hello Adam! > > On Tue, 25 Feb 2020 at 08:58, Adam Saleh wrote: > > > > Nice, I have been trying to fight through the 'git context already missing' > > with pure lua rpm macros, > > and so far was hitting walls left and right :-) > > > > Will look at https://pagure.io/rpkg-util, might have more questions :-) > > You have probably already noticed...the docs at > https://docs.pagure.org/rpkg-util/index.html > are for rpkg-util version that is currently in Fedora but it contains > some nice introduction, nevertheless. > > The (newer) git version from the pagure repository has some features > (e.g. git_release macro), > which are not yet documented there. > > man page of the git version ($ man rpkg) is up to date, however. > > Thank you for looking at it. > clime ...in case of any questions i am on this email or on #fedora-apps. Again, thanks for looking at it. Cheers. > > > > > Adam > > > > On Tue, Feb 25, 2020 at 12:20 AM clime wrote: > >> > >> Hello! > >> > >> On Mon, 24 Feb 2020 at 17:50, Pierre-Yves Chibon > >> wrote: > >> > > >> > Good Morning Everyone, > >> > > >> > This topic has already been discussed a few times over the past month, > >> > but Adam > >> > Saleh, Nils Philippsen and myself have had the opportunity to invest > >> > some time > >> > on it with the hope of making the packager's life simpler as well as > >> > making it > >> > easier to build automation around our package maintenance. > >> > > >> > We have investigated a few ideas, the full list with their pros and cons > >> > can be > >> > found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev > >> > If you have any questions about some of these, please ask them, we'll be > >> > happy > >> > to answer them and potentially complete this document. > >> > > >> > > >> > For both the release and the changelog fields we believe using RPM > >> > macros would > >> > satisfy the requirements we have for opting-in/out: > >> > - You can easily opt-in by using the macros > >> > - You can easily opt-out by removing the macros from your spec file > >> > > >> > > >> > For the changelog, we believe we have a potential good solution: > >> > - The changelog will be automatically generated using an external > >> > `changelog` > >> > file as well as the commit history > >> > - When you opt-in, you will simply move the existing changelog to an > >> > external > >> > file `changelog`, which is placed in the dist-git repo and add the > >> > appropriate macro. > >> > - Upon building, the macro will generate the changelog using all the > >> > commits > >> > of the repo up to the last commit touching the `changelog` file. Of > >> > all > >> > these commits it will only consider the commits following these > >> > rules: > >> > - There are generally two approaches regarding what to include by > >> > default: > >> > 1. commits touching only the `sources`, `.gitignore`, > >> > `dead.package` > >> > files, `tests` folder will be ignored by default, i.e. a > >> > blacklist > >> > 2. only commits touching the spec file or patches will be included > >> > by > >> > default, i.e. a whitelist > >> > ==> Which approach do you think is better/will work in most cases? > >> > - empty commits (not touching any files) will be included on the > >> > assumption > >> > that this is their purpose > >> > - commits containing "magic keyword" (#changelog_exclude, > >> > #changelog_include?) will be ignored or included as the case may be > >> > - Finally the content of the changelog file specified will be appended > >> > to the > >> > changelog generated from the git history > >> > > >> > > >> > If you wanted to edit the changelog, you would: > >> > - Generate the changelog locally via a command like: > >> > `fedpkg generate-changelog > changelog` > >> > - Edit `changelog` as desired > >> > - Commit and push the changes > >> > > >> > Since the macro will only consider the commits up to the first commit > >> > touching > >> > `changelog` (in other words, the macro will only consider the commits > >> > after this > >> > one) and include `changelog` file as is, your edits will appear in the > >> > RPM > >> > changelog as you want. > >> > > >> > One thing to note is that rebuilds from the same commit will have the > >> > same > >> > %changelog, they will not get a new entry. If you want a new changelog > >> > entry, > >> > you have to create a new commit with the desired changelog entry as > >> > commit > >> > message (the commit itself can be empty). > >> > > >> > > >> > > >> > However, for the release field, we are struggling a little bit more, two > >> > options > >> > are more appealing to us: > >> > > >> > A) The release field is automatically generated using two elements: > >> > - the number of commits at this version > >> > - the number of builds at this commit > >> > - the dist-tag being added
Re: Ideas and proposal for removing changelog and release fields from spec file
Hello Adam!
On Tue, 25 Feb 2020 at 08:58, Adam Saleh wrote:
>
> Nice, I have been trying to fight through the 'git context already missing'
> with pure lua rpm macros,
> and so far was hitting walls left and right :-)
>
> Will look at https://pagure.io/rpkg-util, might have more questions :-)
You have probably already noticed...the docs at
https://docs.pagure.org/rpkg-util/index.html
are for rpkg-util version that is currently in Fedora but it contains
some nice introduction, nevertheless.
The (newer) git version from the pagure repository has some features
(e.g. git_release macro),
which are not yet documented there.
man page of the git version ($ man rpkg) is up to date, however.
Thank you for looking at it.
clime
>
> Adam
>
> On Tue, Feb 25, 2020 at 12:20 AM clime wrote:
>>
>> Hello!
>>
>> On Mon, 24 Feb 2020 at 17:50, Pierre-Yves Chibon wrote:
>> >
>> > Good Morning Everyone,
>> >
>> > This topic has already been discussed a few times over the past month, but
>> > Adam
>> > Saleh, Nils Philippsen and myself have had the opportunity to invest some
>> > time
>> > on it with the hope of making the packager's life simpler as well as
>> > making it
>> > easier to build automation around our package maintenance.
>> >
>> > We have investigated a few ideas, the full list with their pros and cons
>> > can be
>> > found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
>> > If you have any questions about some of these, please ask them, we'll be
>> > happy
>> > to answer them and potentially complete this document.
>> >
>> >
>> > For both the release and the changelog fields we believe using RPM macros
>> > would
>> > satisfy the requirements we have for opting-in/out:
>> > - You can easily opt-in by using the macros
>> > - You can easily opt-out by removing the macros from your spec file
>> >
>> >
>> > For the changelog, we believe we have a potential good solution:
>> > - The changelog will be automatically generated using an external
>> > `changelog`
>> > file as well as the commit history
>> > - When you opt-in, you will simply move the existing changelog to an
>> > external
>> > file `changelog`, which is placed in the dist-git repo and add the
>> > appropriate macro.
>> > - Upon building, the macro will generate the changelog using all the
>> > commits
>> > of the repo up to the last commit touching the `changelog` file. Of all
>> > these commits it will only consider the commits following these rules:
>> > - There are generally two approaches regarding what to include by
>> > default:
>> > 1. commits touching only the `sources`, `.gitignore`, `dead.package`
>> > files, `tests` folder will be ignored by default, i.e. a blacklist
>> > 2. only commits touching the spec file or patches will be included by
>> > default, i.e. a whitelist
>> > ==> Which approach do you think is better/will work in most cases?
>> > - empty commits (not touching any files) will be included on the
>> > assumption
>> > that this is their purpose
>> > - commits containing "magic keyword" (#changelog_exclude,
>> > #changelog_include?) will be ignored or included as the case may be
>> > - Finally the content of the changelog file specified will be appended
>> > to the
>> > changelog generated from the git history
>> >
>> >
>> > If you wanted to edit the changelog, you would:
>> > - Generate the changelog locally via a command like:
>> > `fedpkg generate-changelog > changelog`
>> > - Edit `changelog` as desired
>> > - Commit and push the changes
>> >
>> > Since the macro will only consider the commits up to the first commit
>> > touching
>> > `changelog` (in other words, the macro will only consider the commits
>> > after this
>> > one) and include `changelog` file as is, your edits will appear in the RPM
>> > changelog as you want.
>> >
>> > One thing to note is that rebuilds from the same commit will have the same
>> > %changelog, they will not get a new entry. If you want a new changelog
>> > entry,
>> > you have to create a new commit with the desired changelog entry as commit
>> > message (the commit itself can be empty).
>> >
>> >
>> >
>> > However, for the release field, we are struggling a little bit more, two
>> > options
>> > are more appealing to us:
>> >
>> > A) The release field is automatically generated using two elements:
>> > - the number of commits at this version
>> > - the number of builds at this commit
>> > - the dist-tag being added after them
>> > The release field would thus look like:
>> > ``.%{dist}``
>> >
>> > So we could have: (A, B, C and D being different commits)
>> > A -- version: 0.9 -- release: ?
>> > |
>> > B -- version: 1.0 -- release: 1.0
>> > |
>> > C -- version: 1.0 -- release: 2.0
>> > |
>> > D -- version: 1.1 -- release: 1.0
>> >
>> > A rebuild of the commit D, would lead to a release 1.1 (assuming the first
>> > one
>> > succeeded)/
>> >
>>
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020 at 05:48:36PM +0100, Pierre-Yves Chibon wrote: > - commits containing "magic keyword" (#changelog_exclude, > #changelog_include?) will be ignored or included as the case may be Could we please use the usual git commit keyword syntax? I.e. the e-mail header format (Signed-off-by:). E.g.: RPM-Changelog: include RPM-Changelog: exclude > If you wanted to edit the changelog, you would: > - Generate the changelog locally via a command like: > `fedpkg generate-changelog > changelog` > - Edit `changelog` as desired > - Commit and push the changes > If the changelog file name is hard-coded, I recommend "fedpkg generate-changelog" command to write directly into the "changelog" file. -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Nice, I have been trying to fight through the 'git context already missing'
with pure lua rpm macros,
and so far was hitting walls left and right :-)
Will look at https://pagure.io/rpkg-util, might have more questions :-)
Adam
On Tue, Feb 25, 2020 at 12:20 AM clime wrote:
> Hello!
>
> On Mon, 24 Feb 2020 at 17:50, Pierre-Yves Chibon
> wrote:
> >
> > Good Morning Everyone,
> >
> > This topic has already been discussed a few times over the past month,
> but Adam
> > Saleh, Nils Philippsen and myself have had the opportunity to invest
> some time
> > on it with the hope of making the packager's life simpler as well as
> making it
> > easier to build automation around our package maintenance.
> >
> > We have investigated a few ideas, the full list with their pros and cons
> can be
> > found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
> > If you have any questions about some of these, please ask them, we'll be
> happy
> > to answer them and potentially complete this document.
> >
> >
> > For both the release and the changelog fields we believe using RPM
> macros would
> > satisfy the requirements we have for opting-in/out:
> > - You can easily opt-in by using the macros
> > - You can easily opt-out by removing the macros from your spec file
> >
> >
> > For the changelog, we believe we have a potential good solution:
> > - The changelog will be automatically generated using an external
> `changelog`
> > file as well as the commit history
> > - When you opt-in, you will simply move the existing changelog to an
> external
> > file `changelog`, which is placed in the dist-git repo and add the
> > appropriate macro.
> > - Upon building, the macro will generate the changelog using all the
> commits
> > of the repo up to the last commit touching the `changelog` file. Of
> all
> > these commits it will only consider the commits following these
> rules:
> > - There are generally two approaches regarding what to include by
> default:
> > 1. commits touching only the `sources`, `.gitignore`,
> `dead.package`
> > files, `tests` folder will be ignored by default, i.e. a
> blacklist
> > 2. only commits touching the spec file or patches will be included
> by
> > default, i.e. a whitelist
> > ==> Which approach do you think is better/will work in most cases?
> > - empty commits (not touching any files) will be included on the
> assumption
> > that this is their purpose
> > - commits containing "magic keyword" (#changelog_exclude,
> > #changelog_include?) will be ignored or included as the case may be
> > - Finally the content of the changelog file specified will be appended
> to the
> > changelog generated from the git history
> >
> >
> > If you wanted to edit the changelog, you would:
> > - Generate the changelog locally via a command like:
> > `fedpkg generate-changelog > changelog`
> > - Edit `changelog` as desired
> > - Commit and push the changes
> >
> > Since the macro will only consider the commits up to the first commit
> touching
> > `changelog` (in other words, the macro will only consider the commits
> after this
> > one) and include `changelog` file as is, your edits will appear in the
> RPM
> > changelog as you want.
> >
> > One thing to note is that rebuilds from the same commit will have the
> same
> > %changelog, they will not get a new entry. If you want a new changelog
> entry,
> > you have to create a new commit with the desired changelog entry as
> commit
> > message (the commit itself can be empty).
> >
> >
> >
> > However, for the release field, we are struggling a little bit more, two
> options
> > are more appealing to us:
> >
> > A) The release field is automatically generated using two elements:
> > - the number of commits at this version
> > - the number of builds at this commit
> > - the dist-tag being added after them
> > The release field would thus look like:
> > ``.%{dist}``
> >
> > So we could have: (A, B, C and D being different commits)
> > A -- version: 0.9 -- release: ?
> > |
> > B -- version: 1.0 -- release: 1.0
> > |
> > C -- version: 1.0 -- release: 2.0
> > |
> > D -- version: 1.1 -- release: 1.0
> >
> > A rebuild of the commit D, would lead to a release 1.1 (assuming the
> first one
> > succeeded)/
> >
> > Pros:
> > - Easy to replicate locally
> > - Every changelog entry can be mapped to a `version-release` (No
> changes to the
> >packaging guidelines)
> > - Allows triggering two builds from the same commit without any manual
> change
> >(simplifies mass-rebuilds)
> > - Easy to link a certain build with a specific commit
> > - Easy to “guess” the next release value before triggering the build
> >
> > Cons:
> > - Old packages which are no longer receiving upstream releases may need
> > special care (for example if they are at the release -48 but only
> had 45
> > commits leading to this)
> > - Relies on information from the
Re: Ideas and proposal for removing changelog and release fields from spec file
Hello!
On Mon, 24 Feb 2020 at 17:50, Pierre-Yves Chibon wrote:
>
> Good Morning Everyone,
>
> This topic has already been discussed a few times over the past month, but
> Adam
> Saleh, Nils Philippsen and myself have had the opportunity to invest some time
> on it with the hope of making the packager's life simpler as well as making it
> easier to build automation around our package maintenance.
>
> We have investigated a few ideas, the full list with their pros and cons can
> be
> found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
> If you have any questions about some of these, please ask them, we'll be happy
> to answer them and potentially complete this document.
>
>
> For both the release and the changelog fields we believe using RPM macros
> would
> satisfy the requirements we have for opting-in/out:
> - You can easily opt-in by using the macros
> - You can easily opt-out by removing the macros from your spec file
>
>
> For the changelog, we believe we have a potential good solution:
> - The changelog will be automatically generated using an external `changelog`
> file as well as the commit history
> - When you opt-in, you will simply move the existing changelog to an
> external
> file `changelog`, which is placed in the dist-git repo and add the
> appropriate macro.
> - Upon building, the macro will generate the changelog using all the commits
> of the repo up to the last commit touching the `changelog` file. Of all
> these commits it will only consider the commits following these rules:
> - There are generally two approaches regarding what to include by default:
> 1. commits touching only the `sources`, `.gitignore`, `dead.package`
> files, `tests` folder will be ignored by default, i.e. a blacklist
> 2. only commits touching the spec file or patches will be included by
> default, i.e. a whitelist
> ==> Which approach do you think is better/will work in most cases?
> - empty commits (not touching any files) will be included on the
> assumption
> that this is their purpose
> - commits containing "magic keyword" (#changelog_exclude,
> #changelog_include?) will be ignored or included as the case may be
> - Finally the content of the changelog file specified will be appended to
> the
> changelog generated from the git history
>
>
> If you wanted to edit the changelog, you would:
> - Generate the changelog locally via a command like:
> `fedpkg generate-changelog > changelog`
> - Edit `changelog` as desired
> - Commit and push the changes
>
> Since the macro will only consider the commits up to the first commit touching
> `changelog` (in other words, the macro will only consider the commits after
> this
> one) and include `changelog` file as is, your edits will appear in the RPM
> changelog as you want.
>
> One thing to note is that rebuilds from the same commit will have the same
> %changelog, they will not get a new entry. If you want a new changelog entry,
> you have to create a new commit with the desired changelog entry as commit
> message (the commit itself can be empty).
>
>
>
> However, for the release field, we are struggling a little bit more, two
> options
> are more appealing to us:
>
> A) The release field is automatically generated using two elements:
> - the number of commits at this version
> - the number of builds at this commit
> - the dist-tag being added after them
> The release field would thus look like:
> ``.%{dist}``
>
> So we could have: (A, B, C and D being different commits)
> A -- version: 0.9 -- release: ?
> |
> B -- version: 1.0 -- release: 1.0
> |
> C -- version: 1.0 -- release: 2.0
> |
> D -- version: 1.1 -- release: 1.0
>
> A rebuild of the commit D, would lead to a release 1.1 (assuming the first one
> succeeded)/
>
> Pros:
> - Easy to replicate locally
> - Every changelog entry can be mapped to a `version-release` (No changes to
> the
>packaging guidelines)
> - Allows triggering two builds from the same commit without any manual change
>(simplifies mass-rebuilds)
> - Easy to link a certain build with a specific commit
> - Easy to “guess” the next release value before triggering the build
>
> Cons:
> - Old packages which are no longer receiving upstream releases may need
> special care (for example if they are at the release -48 but only had 45
> commits leading to this)
> - Relies on information from the build system for the build number (but can
> be closely simulated locally since the info is only used for
> rebuilds)
> - Upgrade path may be problematic if Fn is upgraded to version X with one
> commit while Fn-1 is upgrade to version X with 2 commits (or more)
>
>
> B) The release field is automatically generated taking existing builds in all
> current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This means
> for
> builds of the same epoch:version, find a new release
Re: Ideas and proposal for removing changelog and release fields from spec file
Dne 24. 02. 20 v 18:13 Miro Hrončok napsal(a): > Can we please have a "git is the only source of truth" version of this? I.e. > "Compute the release field from the number > of commits since the last version change" in the document. It seem to only > have one con (breaks if two builds are > triggered from the same commit) which is the status quo. > +1 Or the combination of: Get the release field from the annotation of the git tag and Get the release field using the number of tags since the last version change If the regexp does not match, just count the tags (or the commits). And BTW I like the changelog proposal. -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 24. 02. 20 19:30, Nicolas Mailhot via devel wrote: Le lundi 24 février 2020 à 18:13 +0100, Miro Hrončok a écrit : On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: However, for the release field, we are struggling a little bit more, two options are more appealing to us: Can we please have a "git is the only source of truth" version of this? Packages have a life in review before they get a Fedora git (sometimes the life can be years in copr, OBS or elsewhere as our review process is none too fast) Yet even if we count builds, the package needs to be in git first. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
Le lundi 24 février 2020 à 18:13 +0100, Miro Hrončok a écrit : > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > > However, for the release field, we are struggling a little bit > > more, two options > > are more appealing to us: > > Can we please have a "git is the only source of truth" version of > this? Packages have a life in review before they get a Fedora git (sometimes the life can be years in copr, OBS or elsewhere as our review process is none too fast) Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On Mon, Feb 24, 2020, 18:38 Miro Hrončok wrote: > On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: > > However, for the release field, we are struggling a little bit more, two > options > > are more appealing to us: > > Can we please have a "git is the only source of truth" version of this? > I.e. > "Compute the release field from the number of commits since the last > version > change" in the document. It seem to only have one con (breaks if two > builds are > triggered from the same commit) which is the status quo. > > If you need to rebuild for a libpingouisawesome soname bump, you just do > an > empty commit with the explanation. > > If you merge that empty commit to a branch that did not need it, it would > have a > bogus changelog entry (status quo). If you care, you would not merge but > cherry-pick anything thta comes next (which is now much easier given the > benefit > of not having the %changelog and release). > > With the proposed solution that includes "successful build count" you > always > bump and build even if it is not needed and also you make the release > number > depend on a specific build system, which I think is kinda wrong. > > i.e. if you do two "fedpkg build" in a row without a commit, I think that > the > second one should still fail with "already been build" kind of message. > What if build environment has changed? > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ideas and proposal for removing changelog and release fields from spec file
On 24. 02. 20 17:48, Pierre-Yves Chibon wrote: However, for the release field, we are struggling a little bit more, two options are more appealing to us: Can we please have a "git is the only source of truth" version of this? I.e. "Compute the release field from the number of commits since the last version change" in the document. It seem to only have one con (breaks if two builds are triggered from the same commit) which is the status quo. If you need to rebuild for a libpingouisawesome soname bump, you just do an empty commit with the explanation. If you merge that empty commit to a branch that did not need it, it would have a bogus changelog entry (status quo). If you care, you would not merge but cherry-pick anything thta comes next (which is now much easier given the benefit of not having the %changelog and release). With the proposed solution that includes "successful build count" you always bump and build even if it is not needed and also you make the release number depend on a specific build system, which I think is kinda wrong. i.e. if you do two "fedpkg build" in a row without a commit, I think that the second one should still fail with "already been build" kind of message. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Ideas and proposal for removing changelog and release fields from spec file
Good Morning Everyone,
This topic has already been discussed a few times over the past month, but Adam
Saleh, Nils Philippsen and myself have had the opportunity to invest some time
on it with the hope of making the packager's life simpler as well as making it
easier to build automation around our package maintenance.
We have investigated a few ideas, the full list with their pros and cons can be
found in this document: https://hackmd.io/8pSwJidhTYel9euQqMEKpw?viev
If you have any questions about some of these, please ask them, we'll be happy
to answer them and potentially complete this document.
For both the release and the changelog fields we believe using RPM macros would
satisfy the requirements we have for opting-in/out:
- You can easily opt-in by using the macros
- You can easily opt-out by removing the macros from your spec file
For the changelog, we believe we have a potential good solution:
- The changelog will be automatically generated using an external `changelog`
file as well as the commit history
- When you opt-in, you will simply move the existing changelog to an external
file `changelog`, which is placed in the dist-git repo and add the
appropriate macro.
- Upon building, the macro will generate the changelog using all the commits
of the repo up to the last commit touching the `changelog` file. Of all
these commits it will only consider the commits following these rules:
- There are generally two approaches regarding what to include by default:
1. commits touching only the `sources`, `.gitignore`, `dead.package`
files, `tests` folder will be ignored by default, i.e. a blacklist
2. only commits touching the spec file or patches will be included by
default, i.e. a whitelist
==> Which approach do you think is better/will work in most cases?
- empty commits (not touching any files) will be included on the assumption
that this is their purpose
- commits containing "magic keyword" (#changelog_exclude,
#changelog_include?) will be ignored or included as the case may be
- Finally the content of the changelog file specified will be appended to the
changelog generated from the git history
If you wanted to edit the changelog, you would:
- Generate the changelog locally via a command like:
`fedpkg generate-changelog > changelog`
- Edit `changelog` as desired
- Commit and push the changes
Since the macro will only consider the commits up to the first commit touching
`changelog` (in other words, the macro will only consider the commits after this
one) and include `changelog` file as is, your edits will appear in the RPM
changelog as you want.
One thing to note is that rebuilds from the same commit will have the same
%changelog, they will not get a new entry. If you want a new changelog entry,
you have to create a new commit with the desired changelog entry as commit
message (the commit itself can be empty).
However, for the release field, we are struggling a little bit more, two options
are more appealing to us:
A) The release field is automatically generated using two elements:
- the number of commits at this version
- the number of builds at this commit
- the dist-tag being added after them
The release field would thus look like:
``.%{dist}``
So we could have: (A, B, C and D being different commits)
A -- version: 0.9 -- release: ?
|
B -- version: 1.0 -- release: 1.0
|
C -- version: 1.0 -- release: 2.0
|
D -- version: 1.1 -- release: 1.0
A rebuild of the commit D, would lead to a release 1.1 (assuming the first one
succeeded)/
Pros:
- Easy to replicate locally
- Every changelog entry can be mapped to a `version-release` (No changes to the
packaging guidelines)
- Allows triggering two builds from the same commit without any manual change
(simplifies mass-rebuilds)
- Easy to link a certain build with a specific commit
- Easy to “guess” the next release value before triggering the build
Cons:
- Old packages which are no longer receiving upstream releases may need
special care (for example if they are at the release -48 but only had 45
commits leading to this)
- Relies on information from the build system for the build number (but can
be closely simulated locally since the info is only used for
rebuilds)
- Upgrade path may be problematic if Fn is upgraded to version X with one
commit while Fn-1 is upgrade to version X with 2 commits (or more)
B) The release field is automatically generated taking existing builds in all
current Fedora releases (ie: rawhide, Fn, Fn-1...) into account. This means for
builds of the same epoch:version, find a new release that (if at all possible)
ensures upgradability from older Fedora versions to newer ones, adhering to the
structure of a release tag documented in the Versioning Guidelines[1]. Going
from the (RPM-wise) "latest build" that the new one should surpass, this can
mean bumping in the front
