Re: Proposed Fedora packaging guideline: More Go packaging

[Nicolas Mailhot]

Hi Jason,

> > "nm" == nicolas mailhot wrote:
> Jason L Tibbitts wrote:
> > nm> And the forge macros are now available since
> > nm> redhat-rpm-config-73-1.fc28 (I had missed the push due to
> > upstream
> > nm> renaming the file). Heartfelt thanks to Jason Tibbitts !
> > Please don't forget to let me know when it's time to start thinking
> > about pushing this down to F27.  And maybe F26.  And as far as I can
> > tell it should work with only minor modification in EPEL7 (via
> > epel-rpm-macros).  
> 
> I don't know about EPEL6, but we use it as-is in EL7 and it works just
> as well (except maybe for the %autosetup bits but IIRC that's
> autosetup which is broken in EL7). 
> 
> It's a good idea to let people play with it in fedora-devel maybe a
> month, in case I missed something, but from a technical POW I'm prety
> sure it could be merged up to EL7 now.

Now that the appropriate observation time has passed, and people have
not find new problems (in fact they found scenarii that worked I had not
thought about) can you please merge

https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/18

and push it to stable?

The Go macros need it and Fedora Go maintainers now want to fork
forgemeta if redhat-rpm-config does not make forgemeta available in the
Fedora stable versions they target, which would all make a huge mess. I
guess it's been too successful ;).

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "nicolas mailhot" 
À: "Jan Chaloupka" 
>> I mentioned a list of things that you did not answer fully. Most important 
>> to me:
>> - your macros do not generate build-time dependencies, which I see as 
>> one of the drawbacks.

> Generating BuildRequires dynamically needs changes in rpm tooling (mostly 
> mock, I think). So it can not be done today if I'm not wrong. If you have a 
> tool that outputs BuildRequires outside rpm, you can still use it. That's not 
> an
> absolute showstopper as the go compiler is pretty explicit on what's missing, 
> so BuildRequires are easily added during the first %build or %check run 
> (though that's tedious and annoying).

>> Do you plan to ignore them?

>Generating BuildRequires would be quite easy if rpm tooling provided an entry 
>point at the end of %prep. So, that's a mid-term objective and requires 
>sending an RFE mock-side first.

And that part is done:

https://github.com/rpm-software-management/mock/issues/160

Feel free to complete/correct/support the request so it has a change to be 
taken into account and we can start working on integrated Go BuildRequires

Regards

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jan Chaloupka" 

Hi Jan

Apologies for the delayed answer, I was beating the PR into shape to address 
review comments.

> Let's keep moving forward, 
> improving the infrastructure for other folks and pushing new ideas and 
> improvements into practical solutions.

Let's move forward, I'm sick on sitting on a private spec stash and I see 
people trying to package stuff already in my list with the old guidelines.


On 12/17/2017 08:11 AM, nicolas.mailhot wrote:

>> It builds on the hard work of the Go SIG and reuses the rpm automation 
>> ofhttps://fedoraproject.org/wiki/PackagingDrafts/Go  when it exists, and 
>> produces compatible packages.

> Can you describe what you mean by "compatible packages"?

That mostly means the resulting packages can use "old-style" packages and 
"old-style" packages can use "new-style" packages. The effective result is 
similar enough there is no need for a rupture. Now, as I wrote before 
"old-style" packages will often be too old or not rigorous enough to be useful 
to "new-style" packages.

> I mentioned a list of things that you did not answer fully. Most important to 
> me:
> - your macros do not generate build-time dependencies, which I see as 
> one of the drawbacks.

Generating BuildRequires dynamically needs changes in rpm tooling (mostly mock, 
I think). So it can not be done today if I'm not wrong. If you have a tool that 
outputs BuildRequires outside rpm, you can still use it. That's not an absolute 
showstopper as the go compiler is pretty explicit on what's missing, so 
BuildRequires are easily added during the first %build or %check run (though 
that's tedious and annoying).

> Do you plan to ignore them?

Generating BuildRequires would be quite easy if rpm tooling provided an entry 
point at the end of %prep. So, that's a mid-term objective and requires sending 
an RFE mock-side first.

Requires generation is automatic and complete, so a Go -devel package will pull 
in all it needs without manual intervention (except for version constrains, 
that needs some maturing go dep side first)

> - support for more subpackages. You describe how to specify which files 
> go to which packages 
> (https://fedoraproject.org/wiki/More_Go_packaging#Separate_code_packages) 
> but you don't say how list of provided packages and a list of 
> dependencies are generated for the subpackages.

It is fully automated and transparent, rpm does not care why you split your Go 
code, it will compute provides and requires for every directory installed and 
owned under %{gopath}/src/%{goipath}, and attribute the result to the 
(sub)package owning the the directory. So just choosing what to install in each 
%goinstall invocation is sufficient to compute the corresponding 
requires/provides and attach them to the correct subpackage. If you don't want 
your package to pull a particular dep, just don't install the code that calls 
it.

> - reproducible evaluation of macros. Either for running automatic 
> analysis over spec files or for debugging purposes. I would like the 
> macros to be built in top of binaries I can run separately.

That would be quite difficult to do as most macros depend on rpm variables and 
other context defined during the build run, and can themselves alter rpm state 
dynamically via variable setting and other calls. rpm is not a dumb archive 
format it includes all kinds of package building facilities that the macros 
take advantage of, it ships with an embedded lua engine, and so on.

For "simple" macros that only read a specific variable set, and only perform 
"simple" actions (or set a specific set of variables) you could certainly 
rewrite them as separate scripts or binaries (for example the naming code is 
pretty trivial to rewrite in any language, and takes only one input and one 
output). But the more extensive the integration with rpm is, the more rpm glue 
you'd need to interface the result, and in some cases the glue is likely to be 
a significant part of the existing code. Also, you'd lose quite a lot of 
flexibility, and complexify your build root requirements.

I'm not saying it can not be done, or it is not worthwhile to do, but it's a 
lot of analysis and rewriting work and I'm not sure it's worth it (I'm 
certainly not interested to do it myself).

Note that autodeps are already effectively separate scripts, you just need to 
pass them a few arguments and pipe them Go code directory names to see them 
working. It may feel weird but that's how the autodep rpm API is defined. Other 
ecosystems packaged as rpms sometimes use separate binaries there, sometimes 
with light shell gluing. You have my blessing to rewrite them in Go if you want 
to.


>> - automated package naming derived from the native identifier (import path). 
>> No more packages names without any relation with current upstream naming.

> Can you provide a list of package names that diverge from this convention?

For example, all the golang/x/foo stuff, which is a pity as it is 

Re: Proposed Fedora packaging guideline: More Go packaging

[Jan Chaloupka]

Hi Nicolas,

before I start arguing, I appreciate all your hard work on improving the 
Go guidelines, spending so much time explaining reasoning behind your 
decision. Also appreciating the effort on making the life easier for 
packagers, not just in the Go land, but throughout the distribution as 
well. There is a lot of great ideas in your Go proposal I would like to 
see implemented and widely used. The more transparent and easy-to-use 
spec files, the better for all maintainers. Let's keep moving forward, 
improving the infrastructure for other folks and pushing new ideas and 
improvements into practical solutions.



On 12/17/2017 08:11 AM, nicolas.mail...@laposte.net wrote:

Hi,

I am proposing for inclusion a set of rpm technical files aimed at automating 
the packaging of forge-hosted projects.

- Packaging draft:https://fedoraproject.org/wiki/More_Go_packaging
-https://pagure.io/packaging-committee/issue/734
- go-srpm-macros RFE with the technical 
files:https://bugzilla.redhat.com/show_bug.cgi?id=1526721

This proposal is integrated with and depends on 
thehttps://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation  
draft
It builds on the hard work of the Go SIG and reuses the rpm automation 
ofhttps://fedoraproject.org/wiki/PackagingDrafts/Go  when it exists, and 
produces compatible packages.


Can you describe what you mean by "compatible packages"? I mentioned a 
list of things that you did not answer fully. Most important to me:
- your macros do not generate build-time dependencies, which I see as 
one of the drawbacks. Do you plan to ignore them?
- support for more subpackages. You describe how to specify which files 
go to which packages 
(https://fedoraproject.org/wiki/More_Go_packaging#Separate_code_packages) 
but you don't say how list of provided packages and a list of 
dependencies are generated for the subpackages.
- reproducible evaluation of macros. Either for running automatic 
analysis over spec files or for debugging purposes. I would like the 
macros to be built in top of binaries I can run separately. With Jakub 
(jca...@redhat.com) we were discussing many times we could provide a 
minimal rpm built on top of gofedlib that would extract all the 
necessary pieces, including the naming convention so everyone can import 
provided library guided by the same rules as the guidelines enforce. I 
would like to see us synchronizing on this topic at some point.




What it does:

- drastically shorter spec files, up to 90% in some cases, often removing 
hundreds of lines per spec.


+1 here, the shorter the spec file the better as long as it is still 
clear and transparent

By better I mean:
- easier to read, easier to understand, easier to adopt
- less places to change and look if a spec file change is needed
By clear I mean:
- it is obvious what the spec file is actually declaring to do
- it is easy to customize various parts (e.g. list of tests, list of 
dependencies)



- simple, packager-friendly spec syntax


+1 as long as each macro provides a single functionality. E.g.
- generate a list of provided packages
- generate a list of tests
- generate a list of dependencies

+1 to the %goname, %gourl, %gosource, %goinstall, %__goprovides, 
%_gorequires and other usefull macros as long as they are simple and 
transparent


As long as the macros are optional and user can use any of them if it is 
suitable (suitable meant per a packager judgment).




- automated package naming derived from the native identifier (import path). No 
more packages names without any relation with current upstream naming.


Can you provide a list of package names that diverge from this convention?

For historical reasons there are some packages that does not fall into 
the naming schema. Either because a go project got migrated to a 
different repo or because it make/made sense to combine some projects 
together and ship them in a single rpm. I don't mention Kubernetes, 
Docker and other popular projects as you already mention exception for them.




- working Go autoprovides. No forgotten provides anymore.


Sometimes it make sense to skip some provided package. E.g. some 
packages provide Windows specific code

which has no use on Linux. So the claim is not completely valid.


- working Go autorequires. No forgotten requires anymore.


Depending on how you generate the list you can non-intentionally require 
more than is needed.

Which causes installation of unneeded trees of rpms.
E.g. for building purposes there is no need to install dependencies of tests
So the claim is not completely valid.

(Valid for both requires/provides): Sometimes you only need to 
install/provide a subset of a Go project.
There are not many projects that imports every package another project 
provides.
So to save time packaging dependencies of unimported packages (and their 
following maintenance),
it is beneficial to generate only partial list of required/provided 
packages.


Plus, in case CGO, there is no 1:1 

Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Tuesday, February 6, 2018 7:34:03 PM
> Subject: Re: Proposed Fedora packaging guideline:  More Go packaging
> 
> 
> 
> - Mail original -
> De: "Jakub Cajka"
> 
> Hi Jakub,
> 
> >> And I'm sure any
> >> attempt to strip the WIP bits from my side will be met with energetic
> >> protests.
> 
> > Have you tried that? What make you assume that? I'm sure that if you do it
> > in constructive way, they will be accepted.
> 
> My experience with humans is that they get attached to their bits of text
> even when they have no time to finish them. But yes, I haven't tried, I
> don't want to see a wiki page for a month after writing and formatting and
> revising two separate packaging drafts.
> 
> > I believe that technically exhausting document is needed as Go packaging is
> > far from trivial. Sure it would be great to have
> > (trivial) quick start guide. I think that your proposal fits that bill more
> > than full documentation.
> 
> IMHO that's exactly what FPC expects: a quick start document, not an in-depth
> encyclopædia. In-depth is too hardcore to be validated by domain laymen, is
> useless to onboard new packagers (which is the primary objective of Fedora
> guidelines), is never finished because the state of the art changes, and
> can't be applied to check whether a spec is good or not because in-depth
> concerns *complex* cases where the "right" choice is not obvious, depends on
> many factors that change over time, and usually requires asking the domain
> experts directly.
> 
> So you have the simple common cases, which are sufficient for most packagers,
> and can be addressed by a formal document FPC can review and enforce, and
> "other things" which are valuable but can not ever attain the formalness and
> strictness level expected of Fedora guidelines, require continuous freedom
> to edit, and are hosted in other Fedora wiki pages. At least that's how
> real-life law is written (law text + expert commentaries) and how other
> Fedora packaging SIGs function and got their guidelines approved.
> 
> > They are used primarily to minimize build root size, by reducing the deps
> > size and code size.
> 
> Stripping example material from gopath installation filelists achieves the
> same objective without all the complexity of managing packages that share
> directories Go considers indivisible.

??, I have been talking about tests. From Go perspective source files and test 
source file, testdata are two distinctive sets.
From my POV if your code depends for regular builds on tests code it means that 
there is something horribly wrong with your project(regardless of language 
used).

Docs and examples would be great too, it should be in guidelines that packager 
must use reasonable effort to create subpackages with them.

> 
> >> 7. even core Go people refuse to touch the subject with a 10 foot pole.
> >> Sam
> >> Boyer tried to demo a very small part of what would be necessary this week
> >> end at FOSDEM and this part of his demo *failed*. I don't have the level
> >> of
> >> Go understanding Sam Boyer Has.
> 
> > What has been demoed?? Unfortunately I could not make it there
> 
> Sam Boyer presented the current state of his go dep prototype, and explained
> his design choices (for example no nested projects which means the go import
> path root becomes a core concept and it's useless to invest in splitting
> projects as that usually requires nesting). He ended up stating go dep will
> ignore tests by default, tried to demo optional test handling, and that
> failed. He had another conference the next day I couldn't get to as I was
> stuck in another part of the campus.

Cool, there is recording for anyone interested 
https://fosdem.org/2018/schedule/event/dep/.

> 
> > Have you met with Jan there?
> 
> Unfortunately FOSDEM was its usual crazy stuff with multiple interesting
> conferences at any given time and no hope of being admitted in a room if
> you're late. So I've just been running from conf room to conf room without
> any hope of meeting anyone :(
> 
> > Your packaging proposal seems fairly monolithic and disregarding any prior
> > art, so extensions will be very painful.
> 
> Well, the creation of the proposal was a long suite of extending to cover
> m

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka" 

Hi Jakub,

>> And I'm sure any
>> attempt to strip the WIP bits from my side will be met with energetic
>> protests.

> Have you tried that? What make you assume that? I'm sure that if you do it in 
> constructive way, they will be accepted.

My experience with humans is that they get attached to their bits of text even 
when they have no time to finish them. But yes, I haven't tried, I don't want 
to see a wiki page for a month after writing and formatting and revising two 
separate packaging drafts.

> I believe that technically exhausting document is needed as Go packaging is 
> far from trivial. Sure it would be great to have
> (trivial) quick start guide. I think that your proposal fits that bill more 
> than full documentation.

IMHO that's exactly what FPC expects: a quick start document, not an in-depth 
encyclopædia. In-depth is too hardcore to be validated by domain laymen, is 
useless to onboard new packagers (which is the primary objective of Fedora 
guidelines), is never finished because the state of the art changes, and can't 
be applied to check whether a spec is good or not because in-depth concerns 
*complex* cases where the "right" choice is not obvious, depends on many 
factors that change over time, and usually requires asking the domain experts 
directly.

So you have the simple common cases, which are sufficient for most packagers, 
and can be addressed by a formal document FPC can review and enforce, and 
"other things" which are valuable but can not ever attain the formalness and 
strictness level expected of Fedora guidelines, require continuous freedom to 
edit, and are hosted in other Fedora wiki pages. At least that's how real-life 
law is written (law text + expert commentaries) and how other Fedora packaging 
SIGs function and got their guidelines approved.

> They are used primarily to minimize build root size, by reducing the deps 
> size and code size.

Stripping example material from gopath installation filelists achieves the same 
objective without all the complexity of managing packages that share 
directories Go considers indivisible.

>> 7. even core Go people refuse to touch the subject with a 10 foot pole. Sam
>> Boyer tried to demo a very small part of what would be necessary this week
>> end at FOSDEM and this part of his demo *failed*. I don't have the level of
>> Go understanding Sam Boyer Has.

> What has been demoed?? Unfortunately I could not make it there

Sam Boyer presented the current state of his go dep prototype, and explained 
his design choices (for example no nested projects which means the go import 
path root becomes a core concept and it's useless to invest in splitting 
projects as that usually requires nesting). He ended up stating go dep will 
ignore tests by default, tried to demo optional test handling, and that failed. 
He had another conference the next day I couldn't get to as I was stuck in 
another part of the campus.

> Have you met with Jan there?

Unfortunately FOSDEM was its usual crazy stuff with multiple interesting 
conferences at any given time and no hope of being admitted in a room if you're 
late. So I've just been running from conf room to conf room without any hope of 
meeting anyone :(

> Your packaging proposal seems fairly monolithic and disregarding any prior 
> art, so extensions will be very painful.

Well, the creation of the proposal was a long suite of extending to cover more 
cases (that's why some macros are rather long and convoluted), so I'm quite 
sure extension is possible. Of course I may miss something, if you identify a 
roadblock please point it out.

>> You want to test one of my packages, fine, just rebuild it. That will run
>> unit tests *and* build the project binaries (which are also a form of code
>> testing, and a very thorough one at that).

> I don't think that is great user experience nor packagers/maintainers 
> experience.

That's pretty much how rpm prefers it to be (use %check and Builddeps) and also 
how Go prefers it to be (no provision for splitting directories). Of course 
anyone interested can try to fight the design of both of them to achieve 
something else, and would try to help, but I'm not interested in being this 
person.

>> > and shipping pre-built shared libraries.
>> 
>> The pre-build shared libraries the old guidelines refuse to build or
>> document? Why exactly are you stating it's a major use case, Fedora is not
>> doing it today. 

> ?? I'm bit confused. Yes, it is not mentioned there yet as Jan have been 
> working on stabilizing APIs, so it will be feasible
> and added there.

My maybe naïve POW is that shared libraries are just another container for the 
elements currently installed in %gopath/src, so doing them is mostly adding a 
separate stage in %build and making %goinstall install the result instead of 
installing raw files. 

The hardest part is to get compatible code bases, defining what needs stripping 
in 

Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>
> Cc: gol...@lists.fedoraproject.org, "Discussion of RPM packaging standards 
> and practices for Fedora"
> <packag...@lists.fedoraproject.org>
> Sent: Monday, February 5, 2018 3:27:01 PM
> Subject: Re: [Fedora-packaging] Re:  Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Jakub Cajka"
> 
> > Our as Fedora or yours company/org? I believe that your contribution of
> > those in to Fedora will be much
> > appreciated.
> 
> Our was meaning the set of specs we are preparing for inclusion. Can't really
> share them before the macros they depend on are merged in rawhide.
> 
> You can grab most of those from
> https://copr.fedorainfracloud.org/coprs/nim/More_Go_Packaging
> 
> before I nuke it to launch a new build from scratch (if seems a bit of grpc
> bootstraping failed this run so that's back to build log analysis, add
> constrains, scratch, retry, wait 3 days for the result. I s hate the yum
> version in copr, every time there is a dumb decision to make it will make
> it, it manages to make yum in EL7 look smart)

You have to explicit about the specs, TBH I have had more fun with dnf, when it 
fist came out.

I believe that you can already open pull requests and review requests stating 
the dependency(so it doesn't get merger too soon) as it will take it some time 
to merge and details can get ironed out during merging/review. As IMHO your 
guidelines will require whole distro change/rebuild to make it compliant. I 
feel like you haven't been engaging with current maintainers much.

JC

> 
> > But IMHO even your changes will not change this, if you don't have few
> > dedicated packager around to
> > do all the bidding.
> 
> Sure, the changes will only remove some barriers to new Go packagers, they
> can't replace those packagers, or the people who take care of the baseline
> core.
> 
> Regards,
> 
> --
> Nicolas Mailhot
> ___
> golang mailing list -- gol...@lists.fedoraproject.org
> To unsubscribe send an email to golang-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Monday, February 5, 2018 4:48:31 PM
> Subject: Re: Proposed Fedora packaging guideline:  More Go packaging
> 
> 
> 
> - Mail original -
> De: "Jakub Cajka"
> 
> Hi Jakub
> 
> > I think that it would be best if Nicolas could fold his proposal in to the
> > original draft as
> > optional part for simple library/binary packages.
> 
> Frankly, that's a lot of work and churn, I don't want the new parts to be
> refused because of something in the old parts, and I certainly do not want
> to spend weeks replacing bits only to put them back in and so on while
> people made up their mind on the things they want replaced or not. The new
> parts are pretty much autonomous and complete, they are sufficient to create
> working Fedora Go packages, they are ready for FPC review.
> 
> If someone wants to extract the ready non discussion parts of the old page
> and get them approved I can work with him to merge both elements
> 
> I didn't want to write about the old page, but since you put it on the table.
> 
> It is full of digressions and elements of no evident applied value while
> packaging Go software. It's not an operational "how to do stuff" document
> it's a "here are various things you may like to know about Go that may or
> may not help you create Fedora Go packages, if they do not help you forget
> about it and run gofed". There are too many WIP non operational bits in the
> old pages to allow merging while in an approval process. And I'm sure any
> attempt to strip the WIP bits from my side will be met with energetic
> protests.

Have you tried that? What make you assume that? I'm sure that if you do it in 
constructive way, they will be accepted.

> 
> People, if you want that page to be ever approved by FPC make it more focused
> and accurate. Remove anything that does not explain to a Fedora Go packager
> how to write a Fedora Go spec file and what to ask upstream Go projects in a
> Fedora packager role. And make sure what remains is succinct, easy to read,
> and applicable without undocumented holes or gofed magic.

I believe that technically exhausting document is needed as Go packaging is far 
from trivial. Sure it would be great to have (trivial) quick start guide. I 
think that your proposal fits that bill more than full documentation.

> 
> I know that's a lot of non-fun work. I did this work on the new page. I don't
> particularly *like* writing documentation. For every line I put in the new
> page, I had to ask myself "is the value to the Fedora packager sufficient to
> justify the time spend writing the text, formatting it, linking in the right
> place, proofing the result". I didn't put in stuff I could not justify
> cleaning up myself. If it was too much work to document it was either of
> insufficient value or better automated rather than explained in a manual
> process.
> 
> Any part of the text that can not be finished or serves another role has no
> place in a guideline submitted to FPC. It should be nuked or moved to a
> separate wiki page. All the half-finished and non operational stuff in there
> is the reason the draft has been stuck in pre-approval stage for four years.
> Just put yourself in FPC's place, its mission is to confirm a guideline is
> ready to be used by the average Fedora packager, not to produce it from
> half-finished half-related messy notes of the domain experts.

I believe I can see myself in their shoes(sure not 100% accurately) and I don't 
see anything that you are describing from their POV, it would be good to have 
the guidelines sooner, but I will have, as you, rather something complete than 
something half baked and semi broken. Please note that they are WIP and 
everybody is welcomed to contribute in constructive way. I don't think that 
there is any request for FPC to create the guidelines on their own and honestly 
it would rather bad idea IMHO.

> 
> > As his proposal doesn't cover at least two major use cases, i.e. separate
> > packaging of tests(they
> > have no place in devel package as they artificially inflate build root
> > size)
> 
> At this point of time my mind is pretty much set. I won't do separate
> packaging of tests because:
> 1. that raises complex dependency handling questions
> 2. the average Go project test code is full of crufty junk
> 3. the average Go test depends on assets not tracked 

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka" 

Hi Jakub

> I think that it would be best if Nicolas could fold his proposal in to the 
> original draft as
> optional part for simple library/binary packages.

Frankly, that's a lot of work and churn, I don't want the new parts to be 
refused because of something in the old parts, and I certainly do not want to 
spend weeks replacing bits only to put them back in and so on while people made 
up their mind on the things they want replaced or not. The new parts are pretty 
much autonomous and complete, they are sufficient to create working Fedora Go 
packages, they are ready for FPC review.

If someone wants to extract the ready non discussion parts of the old page and 
get them approved I can work with him to merge both elements

I didn't want to write about the old page, but since you put it on the table.

It is full of digressions and elements of no evident applied value while 
packaging Go software. It's not an operational "how to do stuff" document it's 
a "here are various things you may like to know about Go that may or may not 
help you create Fedora Go packages, if they do not help you forget about it and 
run gofed". There are too many WIP non operational bits in the old pages to 
allow merging while in an approval process. And I'm sure any attempt to strip 
the WIP bits from my side will be met with energetic protests.

People, if you want that page to be ever approved by FPC make it more focused 
and accurate. Remove anything that does not explain to a Fedora Go packager how 
to write a Fedora Go spec file and what to ask upstream Go projects in a Fedora 
packager role. And make sure what remains is succinct, easy to read, and 
applicable without undocumented holes or gofed magic.

I know that's a lot of non-fun work. I did this work on the new page. I don't 
particularly *like* writing documentation. For every line I put in the new 
page, I had to ask myself "is the value to the Fedora packager sufficient to 
justify the time spend writing the text, formatting it, linking in the right 
place, proofing the result". I didn't put in stuff I could not justify cleaning 
up myself. If it was too much work to document it was either of insufficient 
value or better automated rather than explained in a manual process.

Any part of the text that can not be finished or serves another role has no 
place in a guideline submitted to FPC. It should be nuked or moved to a 
separate wiki page. All the half-finished and non operational stuff in there is 
the reason the draft has been stuck in pre-approval stage for four years. Just 
put yourself in FPC's place, its mission is to confirm a guideline is ready to 
be used by the average Fedora packager, not to produce it from half-finished 
half-related messy notes of the domain experts.

> As his proposal doesn't cover at least two major use cases, i.e. separate 
> packaging of tests(they
> have no place in devel package as they artificially inflate build root size)

At this point of time my mind is pretty much set. I won't do separate packaging 
of tests because:
1. that raises complex dependency handling questions 
2. the average Go project test code is full of crufty junk
3. the average Go test depends on assets not tracked within Go
4. Go is not designed to separate elements shipped in the same directory
5. no one could answer when I asked the *three* Fedora lists what they were 
using the existing test packages for
6. from what I've seen many of the existing test packages simply can not work 
because they fail to include elements they need
7. even core Go people refuse to touch the subject with a 10 foot pole. Sam 
Boyer tried to demo a very small part of what would be necessary this week end 
at FOSDEM and this part of his demo *failed*. I don't have the level of Go 
understanding Sam Boyer Has.

I am ready to work with people who want to make separate test packages and I 
tried very hard to make them possible in the future but I won't spend time 
trying to ship half-baked stuff that does not work and that no one has a need 
for.

The proposed patterns strip test code from devel packages so build root sizes 
are safe (except for the original package build root, since I *do* execute unit 
tests in check, and they pull their deps in the build root).

You want to test one of my packages, fine, just rebuild it. That will run unit 
tests *and* build the project binaries (which are also a form of code testing, 
and a very thorough one at that).

> and shipping pre-built shared libraries.

The pre-build shared libraries the old guidelines refuse to build or document? 
Why exactly are you stating it's a major use case, Fedora is not doing it today.

"By default, the standard golang compiler produces static libraries. There is 
little value in shipping these prebuilt, especially since these libraries are 
very specifically tied to the exact minor release of the golang compiler."

"Presently the shared object libraries 

Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka" 

> Our as Fedora or yours company/org? I believe that your contribution of those 
> in to Fedora will be much
> appreciated.

Our was meaning the set of specs we are preparing for inclusion. Can't really 
share them before the macros they depend on are merged in rawhide.

You can grab most of those from
https://copr.fedorainfracloud.org/coprs/nim/More_Go_Packaging

before I nuke it to launch a new build from scratch (if seems a bit of grpc 
bootstraping failed this run so that's back to build log analysis, add 
constrains, scratch, retry, wait 3 days for the result. I s hate the yum 
version in copr, every time there is a dumb decision to make it will make it, 
it manages to make yum in EL7 look smart) 

> But IMHO even your changes will not change this, if you don't have few 
> dedicated packager around to
> do all the bidding.

Sure, the changes will only remove some barriers to new Go packagers, they 
can't replace those packagers, or the people who take care of the baseline core.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Monday, February 5, 2018 12:16:14 PM
> Subject: [Fedora-packaging] Re:  Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Jakub Cajka"
> 
> > I think one of the main responsibilities of Fedora packager is to work with
> > upstreams, help them
> > mature and generally improve their projects.
> 
> Sure but expecting everything to be perfect and consistent before shipping
> anything just DOES NOT WORK. Reality is not black and white, it's messy with
> shades of gray
> 
> Even the C/C++ guys do not manage to ship a compat-free package ecosystem,
> and their projects had a few more decades than Go projects to mature, and
> rpm was pretty much built around C/C++ expectations.
> 
> Compat packages are a fact of life in any large software ecosystem, where you
> can't achieve perfect cohesion. Go is getting *very* large. It needs compat
> packages. That does not mean there will be hundreds of them because,
> creating packages is *work*, that does not need they will need maintaining
> for years, because the aim of each compat package is to get obsoleted as
> fast as possible, but there will always be some of them at any given time.
> We are not building a cathedral. Bazaars are messy when full of life.
> 
> > Do you have any evidence supporting this claim? From my point of view Jan
> > and other packages has
> > been spending lot of time on on boarding packagers and working on tooling.
> 
> No one (and certainly not me) is accusing you or Jan not to spend a crazy
> amount of time and energy trying to make things work. But you did not
> achieve a satisfactory Go state in Fedora, read again what Owen wrote, I
> didn't put any word in his mouth, even though I could have written pretty
> much the same message a few months ago.
> 
> Again, no one (and certainly not me) disputes your level of dedication to the
> "cause". You just made some choices in the past (trying to avoid working
> within rpm via godep, refusing to include different states of the same Go
> code in the distro when major Go apps *disagree* on the correct state to
> include) that didn't work out in practice, with the hindsight of several
> years of Fedora Go packaging and the Go package state achieved in Fedora
> after those years.
> 
> It's time to adjust those choices a bit.
> 
> > From my perspective distribution will end up with crazy amount of
> > bitrotting, backport, constant
> > attention requiring package crud..., IMHO basically same as now, apart of
> > it we will have few 1000s
> > of Go based packages with compat names nobody cares about, instead of
> > current 500 or so packages.
> 
> Fedora has lots of Go packages no one cares about because they do not permit
> building the apps people *do* care about.
> 
> Building the apps people care about requires a more aggressive update policy,
> and accepting people will work in parallel on apps, that demand different
> code states, of common dependencies. And there are not so many of those, I
> count 18 of them in our repo out of 476 Go packages, even though the work is
> not completely finished, and finalizing the build of complex tip apps is
> almost certain to increase the proportion a little. That's awfully *good*
> and *nice* given the "no Go API compatibility" scarecrows people like to
> invoke.
> 
> You won't *ever* attract a large pool of packagers, to work on a package
> baseline, that will eventually in some years permit building apps, but never
> this year, because upstream state is not conflict and compat-free yet.
> 
> Regards,
> 
> --
> Nicolas Mailhot
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
> 

Our as Fedora or yours company/org? I believe that your contribution of those 
in to Fedora will be much appreciated.
But IMHO even your changes will not change this, if you don't have few 
dedicated packager around to do all the bidding.

JC
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka" 

> I think one of the main responsibilities of Fedora packager is to work with 
> upstreams, help them
> mature and generally improve their projects.

Sure but expecting everything to be perfect and consistent before shipping 
anything just DOES NOT WORK. Reality is not black and white, it's messy with 
shades of gray

Even the C/C++ guys do not manage to ship a compat-free package ecosystem, and 
their projects had a few more decades than Go projects to mature, and rpm was 
pretty much built around C/C++ expectations.

Compat packages are a fact of life in any large software ecosystem, where you 
can't achieve perfect cohesion. Go is getting *very* large. It needs compat 
packages. That does not mean there will be hundreds of them because, creating 
packages is *work*, that does not need they will need maintaining for years, 
because the aim of each compat package is to get obsoleted as fast as possible, 
but there will always be some of them at any given time. We are not building a 
cathedral. Bazaars are messy when full of life.

> Do you have any evidence supporting this claim? From my point of view Jan and 
> other packages has
> been spending lot of time on on boarding packagers and working on tooling.

No one (and certainly not me) is accusing you or Jan not to spend a crazy 
amount of time and energy trying to make things work. But you did not achieve a 
satisfactory Go state in Fedora, read again what Owen wrote, I didn't put any 
word in his mouth, even though I could have written pretty much the same 
message a few months ago.

Again, no one (and certainly not me) disputes your level of dedication to the 
"cause". You just made some choices in the past (trying to avoid working within 
rpm via godep, refusing to include different states of the same Go code in the 
distro when major Go apps *disagree* on the correct state to include) that 
didn't work out in practice, with the hindsight of several years of Fedora Go 
packaging and the Go package state achieved in Fedora after those years.

It's time to adjust those choices a bit.

> From my perspective distribution will end up with crazy amount of bitrotting, 
> backport, constant
> attention requiring package crud..., IMHO basically same as now, apart of it 
> we will have few 1000s
> of Go based packages with compat names nobody cares about, instead of current 
> 500 or so packages.

Fedora has lots of Go packages no one cares about because they do not permit 
building the apps people *do* care about.

Building the apps people care about requires a more aggressive update policy, 
and accepting people will work in parallel on apps, that demand different code 
states, of common dependencies. And there are not so many of those, I count 18 
of them in our repo out of 476 Go packages, even though the work is not 
completely finished, and finalizing the build of complex tip apps is almost 
certain to increase the proportion a little. That's awfully *good* and *nice* 
given the "no Go API compatibility" scarecrows people like to invoke.

You won't *ever* attract a large pool of packagers, to work on a package 
baseline, that will eventually in some years permit building apps, but never 
this year, because upstream state is not conflict and compat-free yet.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Robert-André Mauchin" <zebo...@gmail.com>
> To: devel@lists.fedoraproject.org
> Cc: gol...@lists.fedoraproject.org, "Discussion of RPM packaging standards 
> and practices for Fedora"
> <packag...@lists.fedoraproject.org>
> Sent: Friday, February 2, 2018 4:54:10 PM
> Subject: Re: Proposed Fedora packaging guideline:  More Go packaging
> 
> On mardi 30 janvier 2018 16:11:49 CET nicolas.mail...@laposte.net wrote:
> > Hi,
> > 
> > Now the technical PR is submitted
> > https://src.fedoraproject.org/rpms/go-srpm-macros/pull-request/1
> > 
> > and waiting for action from the go-srpm-macros maintainers, I took (quite a
> > long) time to refresh and flesh out the corresponding packaging guidelines
> > proposal. It should be fairly complete now:
>  
> > https://fedoraproject.org/wiki/More_Go_packaging
> > 
> > I'd appreciate review to check if I have forgotten an important case, if
> > people understand the text, if they have enhancements or corrections to
> > propose, and so on.
>  
> > Then I will push it FPC side again.
> > 
> > Actual practice should be fairly simple and self-explanatory, the proposal
> > length can be scary but that's because it documents all kinds of corner
> > cases that required digging through specs and mailing lists to find
> > resolution examples before. The basic Go packaging skeleton will be
> > sufficient is most cases without requiring to read any documentation.
>  
> > Regards,
> > 
> > --
> > Nicolas Mailhot
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
> Hello Nicolas,
> 
> Could you add two full examples, one for binary package, one for library
> package, like in https://fedoraproject.org/wiki/PackagingDrafts/
> Go#Sample_RPM_Spec
> 
> Putting it all together (https://fedoraproject.org/wiki/
> More_Go_packaging#Putting_it_all_together) is nice but don't understand if it
> is for a binary or a library.
> 
> Also, is there any plan to update Gofed with these new guidelines? If it's
> not
> automated like today, we will lose time instead of gaining some in the end.
> 
> Best regards,
> 
> Robert-André
> 

I think that it would be best if Nicolas could fold his proposal in to the 
original draft as optional part for simple library/binary packages.

As his proposal doesn't cover at least two major use cases, i.e. separate 
packaging of tests(they have no place in devel package as they artificially 
inflate build root size) and shipping pre-built shared libraries.

Just my opinion,

JC

> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Saturday, February 3, 2018 4:27:36 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> Hi Jakub
> 
> > I'm strongly against general unrestricted practice of compat packages as
> > proposed. If you need compat package you
> > need to work with usptreams on stabilizing the API/project, fork it, or
> > just use COPR as your projects(or its
> > dependencies) are not yet mature enough for Fedora.
> 
> I appreciate the sentiment, and I quite hate compat packages, but the
> restrictions you want did not work in practice for Fedora.
> 
> Making compat package creation hard does not result in faster fixing to use
> new code. What it actually does is to block the packaging of all the
> projects that need a more recent package state, because packagers that need
> the old state for one of their packages, will drag their feet on updating
> common dependencies till the code of their packages is fixed upstream, or
> they have the time to fix it themselves.
> 

I think one of the main responsibilities of Fedora packager is to work with 
upstreams, help them mature and generally improve their projects.

> And blocking packaging of new part means you do not get the new packagers
> that would join because they're interested in the new parts, and would
> contribute to the maintenance of common deps (not all of which need
> compat-ing), and would relieve part of the pressure on the existing
> packagers, that prevents them from working as much a they'd like to on
> updating their packages. It's a death spiral. It results in a massively
> obsolete Go package baselines, full of holes, because all the energy is
> poured in making existing stuff work, at the expense of onboarding new
> packages and packagers.
> 
> Regards,
> 
> --
> Nicolas Mailhot
> ___
> golang mailing list -- gol...@lists.fedoraproject.org
> To unsubscribe send an email to golang-le...@lists.fedoraproject.org
> 

Do you have any evidence supporting this claim? From my point of view Jan and 
other packages has been spending lot of time on on boarding packagers and 
working on tooling.
From my perspective distribution will end up with crazy amount of bitrotting, 
backport, constant attention requiring package crud..., IMHO basically same as 
now, apart of it we will have few 1000s of Go based packages with compat names 
nobody cares about, instead of current 500 or so packages.

JC
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Nicolas Mailhot"

> It's a bit of a Lego guideline, you assemble the spec blocs you need, and 
> ignore those you don't need. The
> example was chosen to include as many blocks as possible, with the 
> walkthrough explaining their respective
> functions. All the blocks are very short and succinct, don't search for long 
> runs of shell code like in the old
> guidelines

That being said if people feel it's useful I can add a barebones lib-only 
example with none of the optional blocks in. However, I fear that the more ways 
the page explains the same stuff, the more confusing and difficult to maintain 
it will be.

So, can people chime in to say whether they feel a supplementary example is 
useful of counter productive?

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Nicolas Mailhot]

Hi Robert André

That's an interesting request

I guess you can't figure if the example is for building binaries or Go libs, 
because there is no hard frontier between both cases in the proposed guidelines

In Go, everything is effectively a code library that can be reused elsewhere. 
So the approach taken in the proposed guidelines, is that everything is a code 
library, some of which happen to build binaries, or setup compatibility 
symlinks, or need bootstrapping

It's a bit of a Lego guideline, you assemble the spec blocs you need, and 
ignore those you don't need. The example was chosen to include as many blocks 
as possible, with the walkthrough explaining their respective functions. All 
the blocks are very short and succinct, don't search for long runs of shell 
code like in the old guidelines
-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma 
brièveté.___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka" 

Hi Jakub

> I'm strongly against general unrestricted practice of compat packages as 
> proposed. If you need compat package you
> need to work with usptreams on stabilizing the API/project, fork it, or just 
> use COPR as your projects(or its
> dependencies) are not yet mature enough for Fedora.

I appreciate the sentiment, and I quite hate compat packages, but the 
restrictions you want did not work in practice for Fedora.

Making compat package creation hard does not result in faster fixing to use new 
code. What it actually does is to block the packaging of all the projects that 
need a more recent package state, because packagers that need the old state for 
one of their packages, will drag their feet on updating common dependencies 
till the code of their packages is fixed upstream, or they have the time to fix 
it themselves.

And blocking packaging of new part means you do not get the new packagers that 
would join because they're interested in the new parts, and would contribute to 
the maintenance of common deps (not all of which need compat-ing), and would 
relieve part of the pressure on the existing packagers, that prevents them from 
working as much a they'd like to on updating their packages. It's a death 
spiral. It results in a massively obsolete Go package baselines, full of holes, 
because all the energy is poured in making existing stuff work, at the expense 
of onboarding new packages and packagers. 

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Robert-André Mauchin]

On mardi 30 janvier 2018 16:11:49 CET nicolas.mail...@laposte.net wrote:
> Hi,
> 
> Now the technical PR is submitted
> https://src.fedoraproject.org/rpms/go-srpm-macros/pull-request/1
> 
> and waiting for action from the go-srpm-macros maintainers, I took (quite a
> long) time to refresh and flesh out the corresponding packaging guidelines
> proposal. It should be fairly complete now:
 
> https://fedoraproject.org/wiki/More_Go_packaging
> 
> I'd appreciate review to check if I have forgotten an important case, if
> people understand the text, if they have enhancements or corrections to
> propose, and so on.
 
> Then I will push it FPC side again.
> 
> Actual practice should be fairly simple and self-explanatory, the proposal
> length can be scary but that's because it documents all kinds of corner
> cases that required digging through specs and mailing lists to find
> resolution examples before. The basic Go packaging skeleton will be
> sufficient is most cases without requiring to read any documentation. 
 
> Regards,
> 
> -- 
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Hello Nicolas,

Could you add two full examples, one for binary package, one for library 
package, like in https://fedoraproject.org/wiki/PackagingDrafts/
Go#Sample_RPM_Spec

Putting it all together (https://fedoraproject.org/wiki/
More_Go_packaging#Putting_it_all_together) is nice but don't understand if it 
is for a binary or a library.

Also, is there any plan to update Gofed with these new guidelines? If it's not 
automated like today, we will lose time instead of gaining some in the end.

Best regards,

Robert-André

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 4:24:52 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> >> Filling upstream holes is pretty much the definition of an
> >> integrator/distributor role. In Go they are particularly numerous and
> >> deep,
> >> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
> >> is astonishingly free of the problems that plague many Go projects,
> >> proving
> >> it *is* possible to do good release engineering in Go).
> 
> > ??
> 
> Just sending some kuddos Kubernetes-side. All the Kubernetes parts we had to
> work with so far were astonishingly free of API breakage and dependency
> cycles because Kubernetes people chose their deps carefully and do version
> their code states. It was a vacation from packaging other Go code.

Yes kubernetes is kind of better than most projects, but IMHO still uses lots 
of not so good practices, although getting bit better over the years.

> 
> > > Pain will not be eased in any significant way as you will still have to
> > > carefully evaluate every
> > > change so you don't break any dependent package.
> > 
> >> Pain *will* be eased in a significant way because we will have the tooling
> >> to
> >> detect breakage automatically, and because fixes once they're done will be
> >> packaged and shared.
> 
> > Your proposal doesn't provide this kind of tooling, if I'm not mistaken.
> 
> Not directly. It does provide the means to easily rev a spec to a new code
> state (version tag or commit), and it makes deps systematic (so Fedora
> tooling can accurately detect what is likely to be impacted by a change).
> Together, those open the way to use generic Fedora testing tools to automate
> build tests.

It makes it bit easier in some cases, but I wouldn't call it and revolution(as 
you are trying to sell it) rather than evolution(and omission of some currently 
mentioned parts), building on top of current practice.

> 
> >> If it's minor let's do it. I'm quite sure you will be pleasantly surprised
> >> by
> >> the level of pain those minor changes remove.
> 
> > ?? I'm in no way opposing your proposal per se.
> 
> Thank you
> 
> >> Fedora can detect the breakage which is the first step to fix it. It does
> >> not
> >> take an army of Go experts, just unbundled packages that can be rebuilt
> >> automatically to identify incompatibilities. Go experts are better
> >> employed
> >> working on fixes than doing rebuild checks that can be automated.
> 
> > Rebuilds could be automated, analysis of them not easily.
> 
> Sure, but at that point
> 1. you know if the new API is compatible or not with other packages
> 2. you've saved human packagers the time and effort to do the same rebuild
> check manually
> 
> So that's still a net win, no ?

No really, or not much, until you determine why the builds have failed(I can 
rebuild of whole distro each day, still I don't have enough spare time to 
determine and fix all FTBFS issues). You still have to have someone who would 
analyze those failures.

> 
> >> And nothing stops you for continuing to use those. Except, the golden test
> >> will always be to actually try to build the code, which is something that
> >> can be automated reliably once the dependencies exported to Fedora tools
> >> are
> >> themselves complete and reliable, and Go packages are exported as rpm
> >> packages, not hidden in private vendor trees.
> 
> > ?? I bit don't understand you. If you can, please help Jan to improve them.
> 
> What I meant is that:
> 1. there are many ways to try to divine if a code state will build, but the
> final arbiter is the Go compiler
> 2. to use Fedora-wide QA tools, code objects need to be exported as packages.
> Every missing test dependency, which is not packaged as a separate Go rpm,
> is something Fedora QA won't be able to run.

I think that I'm still missing your point. Of course you need to package all 
your dependencies.

> 
> >> When you do not unbundle you are still shipping the vendored code, you are
> >> still responsible for its bugs (security  or not). Compat packages do not
> &

Re: Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Colin Walters"

> I appreciate the work you're doing here, 

Thank you

> but I think the right path for golang (indeed for most other language 
> ecosystems) is to autogenerate
> specs.

You're, of course, are entitled to your opinion, but I do not share it :) I 
think rpm has been an incredibly successful technology. It is used today to 
package software no one could imagine when rpm was designed, in languages that 
didn't even exist at that time, and yet one can open spec files from last 
millenium and still recognize most of the parts.

And during all that time, people have been writing one spec generator after 
another, that all failed to get serious traction, and make any serious impact.

I think that the reason for rpm success on one hand, and the serial flops of 
spec generators on the other, it that spec generators are inherently 
inflexible. They do not deal well with special cases. They do not deal well 
with mixed projects that require combining several packaging patterns. They are 
very slow to adapt to rpm enhancements. It's so easy to generate yet another 
line of code there is no incentive to refactor what's generated to remove parts 
no longer needed, to strive for succinctness, efficiency and maintainability.

By and large generators generate very verbose low-quality spec code, which can 
of course, be manually adapted, but is not designed to be adapted by humans, 
making adaptations hard and inefficient (that seems to be a common pitfall of 
code generators of any sort). And, when packaging any large pool of software, 
you will hit many special cases, because software is written by humans, humans 
are messy, and expecting every software project to follow conventions with the 
level of strictness expected by spec generators is, IMHO illusory.

spec file syntax, warts and all, seems to hit the sweet spot between the wish 
to automate as much as possible, and the need to add a few lines of custom code 
whenever a project needs a little dose of custom handling.

I could, quite easily I think, implement something similar to autospec for 
Fedora Go packages. All it would take would be to define a variable name for 
each parameter of the autospec manifest, then call a giant macro that generates 
the rest of the spec file. I didn't do it not because I don't know how to, nor 
because it wouldn't work as is for many Go projects, but because it would 
suffer from the usual generator inflexibility, and inability to handle special 
cases, and projects that deviate slightly from the norm. I *chose* to split the 
Go automation in separate rpm macros, that can be combined by packagers as 
needed in different ways, that can be mixed with the other macros implemented 
by Fedora, that can be completed by custom spec code when there is a need (that 
is BTW a balance I didn't get quite right when I automated fonts packaging a 
few years ago, with hindsight I wish I had designed something a little more 
adaptable, with more flexibility).

Basically I feel that providing powerful spec verbs, is a more powerful model 
than expecting to capture every need in rigid autogenerated templates, no 
matter the amount of switches bolted on later, to manage all the deviations not 
envisioned when defining the original "simple" generation.

That being said rpm is not perfect and will need to continue to evolve to stay 
relevant. For example, with the horizontal compilation model of Go, the number 
of BuildRequires in specs is getting out of hand. So rpm and mock will probably 
need to evolve towards a model, where BuildRequires are not frozen till the end 
of %prep. That would allow to infer at least some BuildRequires from what's in 
the project archive, instead of requiring packagers to declare every single 
BuildRequires manually, because the build root is setup before the archive is 
even opened. However I see such things more like a natural evolution of the rpm 
model, and certainly nothing that requires dumping rpm, or "fixing" it with a 
generation layer.

> Also having all of the golang -devel packages in a single git repository
> with subdirectories so it's easy to have a single PR affect multiple packages
> at once. 

I can't really disagree there, that's how we work internally. It has proved 
invaluable every time we hit "damn, that's the twentieth spec that needs this 
exception, maybe it's time to fold it in common macro code to simplify those 
specs and avoid writing the exception another time, let's refactors all this in 
a single commit (or commit series)" 

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Colin Walters]

On Thu, Feb 1, 2018, at 10:24 AM, nicolas.mail...@laposte.net wrote:
> 
> Not directly. It does provide the means to easily rev a spec to a new 
> code state (version tag or commit), and it makes deps systematic (so 
> Fedora tooling can accurately detect what is likely to be impacted by a 
> change). Together, those open the way to use generic Fedora testing 
> tools to automate build tests.

I appreciate the work you're doing here, but I think the right path
for golang (indeed for most other language ecosystems) is to autogenerate
specs.  Like this:
https://github.com/clearlinux/autospec
plus language-specific plugins basically.  

Also having all of the golang -devel packages in a single git repository
with subdirectories so it's easy to have a single PR affect multiple packages
at once.  (Of course, this model crosses other things beyond golang).

Reference: https://github.com/coreos/coreos-overlay for the latter.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Przemek Klosowski]

On 02/01/2018 05:49 AM, Jakub Cajka wrote:

On contrary Fedora is trying to fill the hole that upstream Go projects dug them selves in to. 
IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even deeper levels of 
"Go (vendor) hell".


This unfortunately became a trend: "the old packaging is dead, long live 
the new packaging!" (not just go, but also pip/conda/npm/...). I think 
Fedora is doing a good job getting it under control, leading upstream 
projects to get their act together---and it demonstrated that this 
works, and actually makes upstream better.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka"

>> Filling upstream holes is pretty much the definition of an
>> integrator/distributor role. In Go they are particularly numerous and deep,
>> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
>> is astonishingly free of the problems that plague many Go projects, proving
>> it *is* possible to do good release engineering in Go).

> ??

Just sending some kuddos Kubernetes-side. All the Kubernetes parts we had to 
work with so far were astonishingly free of API breakage and dependency cycles 
because Kubernetes people chose their deps carefully and do version their code 
states. It was a vacation from packaging other Go code.

> > Pain will not be eased in any significant way as you will still have to
> > carefully evaluate every
> > change so you don't break any dependent package.
> 
>> Pain *will* be eased in a significant way because we will have the tooling to
>> detect breakage automatically, and because fixes once they're done will be
>> packaged and shared.

> Your proposal doesn't provide this kind of tooling, if I'm not mistaken.

Not directly. It does provide the means to easily rev a spec to a new code 
state (version tag or commit), and it makes deps systematic (so Fedora tooling 
can accurately detect what is likely to be impacted by a change). Together, 
those open the way to use generic Fedora testing tools to automate build tests.

>> If it's minor let's do it. I'm quite sure you will be pleasantly surprised by
>> the level of pain those minor changes remove.

> ?? I'm in no way opposing your proposal per se.

Thank you

>> Fedora can detect the breakage which is the first step to fix it. It does not
>> take an army of Go experts, just unbundled packages that can be rebuilt
>> automatically to identify incompatibilities. Go experts are better employed
>> working on fixes than doing rebuild checks that can be automated.

> Rebuilds could be automated, analysis of them not easily.

Sure, but at that point
1. you know if the new API is compatible or not with other packages
2. you've saved human packagers the time and effort to do the same rebuild 
check manually

So that's still a net win, no ?

>> And nothing stops you for continuing to use those. Except, the golden test
>> will always be to actually try to build the code, which is something that
>> can be automated reliably once the dependencies exported to Fedora tools are
>> themselves complete and reliable, and Go packages are exported as rpm
>> packages, not hidden in private vendor trees.

> ?? I bit don't understand you. If you can, please help Jan to improve them.

What I meant is that:
1. there are many ways to try to divine if a code state will build, but the 
final arbiter is the Go compiler
2. to use Fedora-wide QA tools, code objects need to be exported as packages. 
Every missing test dependency, which is not packaged as a separate Go rpm, is 
something Fedora QA won't be able to run.

>> When you do not unbundle you are still shipping the vendored code, you are
>> still responsible for its bugs (security  or not). Compat packages do not
>> augment the code surface you have to care about, on the contrary they offer
>> you the possibility to *reduce* the number of code states in the
>> distribution.

> Please don't put words in to my mouth. I haven't been talking about not 
> un-bundling, just compat
> packages.

Sorry if I misunderstood you:(

However, I do think compat packages are a natural effect of unbundling. One 
does need something to fill in during the time upstream or Fedora work on the 
effects of an API break. Other Fedora language ecosystems do not manage to 
avoid them. You can strive to limit compat packages to the minimum amount, and 
the minimum period, but you can not fully avoid them, once an ecosystem has 
grown enough it is too big to be perfectly coordinated all the time.

So making compat creation easy and fast, is required to manage a huge amount of 
interdependent packages, which are not too careful about API stability. Even if 
one has little love for compat packages as such.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Owen Taylor"

> I'm embarrassed to admit that before I sent my mail I carefully read over
> ... the old PackageDrafts/Go :-( My only excuse is that it was in my
> browser history.

NP, that gave you some context on where Fedora is today.

> Having actually read the relevant parts of "More Go Packaging", the
> explanation of compat packages and notification procedures does make the
> intended operation clearer,

Thank you. Do not hesitate to improve the wording in the wiki, or post 
suggestions here, if you have ideas on how to make it clearer, simpler or more 
efficient

> though the social and technical barrier to a
> packager new to Fedora will still be high if packaging their target package
> requires creating a compat package and fixing multiple other packages.

Unfortunately, packaging something in Go is still going to be harder than 
packaging software in another language in the near term :( I'd love to find 
more magic bullets.

> I still worry that Fedora is not big enough to move the status-quo in the
> Go world - to get the point where Go programs require github.com/foo/bar >=
> .2.3 and actually have been tested with a multiple versions in that range,
> not exactly the one vendored version shipped with the program.

That's a legitimate worry, yes. However given container and cloud people are 
massively adopting Go, that critical cloud software is now mostly written in 
Go, I don't think Fedora can afford to pass on Go and still stay relevant 
server-side. That's even more true for Fedora downstreams and Fedora's main 
sponsor.

So I guess it all boils down to strategic choices for Fedora and Red Hat: 
invest in Go packaging, even if it *is* painful, or pass, and lose relevance 
server-side in the near future. Red Hat certainly has the assets, with 
Openshift and Coreos, to influence the Go world in a Fedora-positive way. I 
don't know if it will choose, or manage, to leverage them. It can not fail to 
notice the many Go projects that propose their software as Ubuntu Docker images 
today. It can not fail to notice that Jakub and Jan, with all their qualities, 
are a tad overworked and not really sufficient to pull Fedora Go forward the 
required amount. The Go ecosystem has just grown too big. Lastly, I understand 
the temptation to let Fedora and RHEL slip in favor of product lines more 
profitable in the near term.

On my employer's side we *do* wish to stay relevant in a Cloud world. My Go 
contributions are an attempt to partner with the Fedora Centos and RHEL 
communities on that. If the partnership does not bear fruits, and 
Fedora/Centos/RHEL do not move in a direction we can use, we'll have to invest 
elsewhere. I'd regret it but this proposal and the spec dump that will follow 
are just too much work to do on one's free time. And employer time requires 
results.

> I haven't had time to read through the entire proposal, but it certainly
> looks like a major step forward!

Please finish reading it and propose all the fixes and comments and 
improvements you want! Your experience is appreciated.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 2:51:13 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> Hi Jakub,
> 
> > It depends (as everything) on available manpower, if you are willing to own
> > your dependencies
> > you can package anything and everything debundled.
> 
> Sure, but available manpower depends on how high the bar you put for people
> wanting to join, and right now this bar is pretty high.
> 
> > On contrary, to this state you dislike(or seems to) Fedora got because of
> > effort of many to de
> > bundle Go based projects.
> 
> I'm *not* disparaging the past efforts of many. They did a lot of good.
> Unfortunately, the result still falls short of the mark. The baseline is not
> complete enough, the baseline is old and aging fast, would-be Go packagers
> like Owen are ready to give up when they start getting an idea of the Go
> state in Fedora.
> 
> It's time to invest in tooling and simplification so the few Go Fedora
> packagers there is, can do more with less, so casual Go packagers can
> contribute their little bit instead of being repulsed by complexity that can
> be avoided and factored out, so people already familiar with Fedora
> packaging can package the Go bits they need without needing a lengthy
> formation on Go specifics.
> 
> There *are* lots of people that would like to do Go packaging. Many Go
> software projects currently make headlines. That there are so few Go
> packagers in Fedora, at the time Go is in the limelight, is a pretty good
> indicator we are making it too hard.
> 
> > I don't see way how it makes it less painful(you have still the whole
> > distro issue). It makes specs
> > more abstract and streamlined when it works, but more opaque and hard to
> > read when its not(and you
> > have to debug some arcane srpm/lua macros).
> 
> It makes it less painful because there is no package-specific magic in each
> and every Go spec file.
> 
> You do not need to debug code in each spec because code that already just
> works for hundreds of  packages is unlikely to not work on the next spec
> file.
> 
> You do not need to comb each and every line of a Go spec file wondering if
> the line is cut and pasting of a common template (multiply by all past
> revisions of common templates) or if it is package-specific. The common
> parts are in common macros not spread right and left.
> 
> You do not need to worry wether common code is complete or not because
> factoring out common code removes the temptation to document 90% of what's
> needed, leaving others to fill in the missing 10%. 10% amounts to an
> astonishing level of work when you multiply it by the number of Go packagers
> and the number of Go packages we would ideally need.
> 
> Plus, when you do find a bug in common code, the fix is shared with the whole
> distribution, not hidden in a single spec file.
> 
> That's what successful factoring of common code earns you.
> 
> And given I have more than 470 Go spec files that just work with the proposed
> macros with no special package-specific adjustment needed, and most time I
> compare one of those to the Fedora equivalent I'm executing more unit tests
> and building more binaries, with drastically shorter spec code, I'm pretty
> sure the factoring out *is* successful.
> 
> >> This step is going to be painful I'm afraid, Fedora dug itself a
> >> deep hole, leaving it is going to be hard.
> 
> > On contrary Fedora is trying to fill the hole that upstream Go projects dug
> > them selves in to.
> 
> Filling upstream holes is pretty much the definition of an
> integrator/distributor role. In Go they are particularly numerous and deep,
> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
> is astonishingly free of the problems that plague many Go projects, proving
> it *is* possible to do good release engineering in Go).

??

> 
> Fedora did dug itself a hole by trying the bundling shortcut around EL7 time.
> Where are the EL7 Go packages? Why are Fedora Atomic people not spearheading
> Go packaging in Fedora? Waiting for bundling to be solved upstream-side made
> Fedora worse not better.

I'm not sure that Fedora. Go ask them don't ask me or other regular Fedora 
maintainers. 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka" 

Hi Jakub,

> It depends (as everything) on available manpower, if you are willing to own 
> your dependencies
> you can package anything and everything debundled.

Sure, but available manpower depends on how high the bar you put for people 
wanting to join, and right now this bar is pretty high.

> On contrary, to this state you dislike(or seems to) Fedora got because of 
> effort of many to de
> bundle Go based projects.

I'm *not* disparaging the past efforts of many. They did a lot of good. 
Unfortunately, the result still falls short of the mark. The baseline is not 
complete enough, the baseline is old and aging fast, would-be Go packagers like 
Owen are ready to give up when they start getting an idea of the Go state in 
Fedora.

It's time to invest in tooling and simplification so the few Go Fedora 
packagers there is, can do more with less, so casual Go packagers can 
contribute their little bit instead of being repulsed by complexity that can be 
avoided and factored out, so people already familiar with Fedora packaging can 
package the Go bits they need without needing a lengthy formation on Go 
specifics.

There *are* lots of people that would like to do Go packaging. Many Go software 
projects currently make headlines. That there are so few Go packagers in 
Fedora, at the time Go is in the limelight, is a pretty good indicator we are 
making it too hard.

> I don't see way how it makes it less painful(you have still the whole distro 
> issue). It makes specs 
> more abstract and streamlined when it works, but more opaque and hard to read 
> when its not(and you
> have to debug some arcane srpm/lua macros).

It makes it less painful because there is no package-specific magic in each and 
every Go spec file. 

You do not need to debug code in each spec because code that already just works 
for hundreds of  packages is unlikely to not work on the next spec file.

You do not need to comb each and every line of a Go spec file wondering if the 
line is cut and pasting of a common template (multiply by all past revisions of 
common templates) or if it is package-specific. The common parts are in common 
macros not spread right and left.

You do not need to worry wether common code is complete or not because 
factoring out common code removes the temptation to document 90% of what's 
needed, leaving others to fill in the missing 10%. 10% amounts to an 
astonishing level of work when you multiply it by the number of Go packagers 
and the number of Go packages we would ideally need.

Plus, when you do find a bug in common code, the fix is shared with the whole 
distribution, not hidden in a single spec file.

That's what successful factoring of common code earns you.

And given I have more than 470 Go spec files that just work with the proposed 
macros with no special package-specific adjustment needed, and most time I 
compare one of those to the Fedora equivalent I'm executing more unit tests and 
building more binaries, with drastically shorter spec code, I'm pretty sure the 
factoring out *is* successful.

>> This step is going to be painful I'm afraid, Fedora dug itself a
>> deep hole, leaving it is going to be hard.

> On contrary Fedora is trying to fill the hole that upstream Go projects dug 
> them selves in to.

Filling upstream holes is pretty much the definition of an 
integrator/distributor role. In Go they are particularly numerous and deep, but 
Fedora users do want their docker and kubernetes (and Kubernetes, BTW, is 
astonishingly free of the problems that plague many Go projects, proving it 
*is* possible to do good release engineering in Go).

Fedora did dug itself a hole by trying the bundling shortcut around EL7 time. 
Where are the EL7 Go packages? Why are Fedora Atomic people not spearheading Go 
packaging in Fedora? Waiting for bundling to be solved upstream-side made 
Fedora worse not better.

> IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even 
> deeper levels of "Go 
> vendor) hell".

I fully agree. What I meant to convey is that replicating upstream vendoring 
Fedora-side only makes the situation worse for users, for upstreams, and for 
Fedora.

> Pain will not be eased in any significant way as you will still have to 
> carefully evaluate every
> change so you don't break any dependent package.

Pain *will* be eased in a significant way because we will have the tooling to 
detect breakage automatically, and because fixes once they're done will be 
packaged and shared.

>> The guidelines and automation aim at making upgrading easy, and avoid one
>> package or packager blocking others

> This is not really and automation per se. It is minor re-factoring and spec 
> streamlining.
> It might or might not enable easier implementation of packaging automation.

If it's minor let's do it. I'm quite sure you will be pleasantly surprised by 
the level of pain those minor changes remove.

>> That would be expensive computing-side but a lot 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Owen Taylor]

Hi Nicolas,

I'm embarrassed to admit that before I sent my mail I carefully read over
... the old PackageDrafts/Go :-( My only excuse is that it was in my
browser history.

Having actually read the relevant parts of "More Go Packaging", the
explanation of compat packages and notification procedures does make the
intended operation clearer, though the social and technical barrier to a
packager new to Fedora will still be high if packaging their target package
requires creating a compat package and fixing multiple other packages. As
you say, once Fedora packaging catches up to the modern state, things will
be better, because the onus won't be on the new packager to do the  work of
updating old libraries.

I still worry that Fedora is not big enough to move the status-quo in the
Go world - to get the point where Go programs require github.com/foo/bar >=
1.2.3 and actually have been tested with a multiple versions in that range,
not exactly the one vendored version shipped with the program. Sp we might
end up with lots of Fedora-specific bugs that the upstream doesn't care
about. But if the Fedora Go community can get some size and momentum and
start pushing fixes upstream, that will certainly be a positive force for
reducing Go programs stuck on random old versions of dependencies!

I haven't had time to read through the entire proposal, but it certainly
looks like a major step forward!

Owen



On Thu, Feb 1, 2018 at 5:21 AM,  wrote:

>
>
> - Mail original -
> De: "Owen Taylor"
>
> Hi Owen,
>
> > Is there a guide for Fedora packagers about how to handle unbundling for
> > golang packages? The draft guidelines don't seem to go into any details.
>
> I don't think there is, nor that it is necessarily needed. The posted
> guidelines should be sufficient technically, there are no magic I know of I
> didn't document, the rest is just a lot of work (but see ↓)
>
> > I've looked at packaging a few golang packages unbundled, and have
> > immediately run into:
> >
> > A) lots of unpackaged dependencies
> > B) dependencies that are packaged in Fedora with different, often much
> > older versions
>
> Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't
> expect anyone to be able to package anything but the most trivial app in
> unbundled mode. Too many common parts are missing, when they are not
> missing they are too old, trying to update an existing Go package is an
> exercise in frustration (too much package-specific shell code, that is
> difficult to understand and does not really work with new code versions),
> and trying to update or add missing parts just reveals more breakage and
> work to be done.
>
> However, accepting to package complex Go apps in bundled mode is how
> Fedora got to this state in the first place. In plain speak, bundling
> (vendoring in Go linguo) just does not scale. You need an awful lot of
> manpower to audit the hundreds of other projects each app bundles, bundling
> removes all the package tooling that may help you to do so, and the result
> is not shared with any other package, or with other versions of the same
> package, so you get zero positive network effects. Worse, big bundled apps
> that do not actually try to work in unbundled mode, do not actually test
> the code they export (they are bundling, remember) so the result is toxic
> to small Go packages that try to work with them.
>
> So bundling parts is a direct road to bundling everything, bundling
> everything is a direct road to bundling everything blindly, bundling
> everything blindly is error-prone and dangerous (because upstreams are only
> human and do make lots of mistakes), and pretty much removes any value
> Fedora can add to end users, to other parts of Fedora that would like to
> integrate with Go software, or to the upstream projects themselves (no
> Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to
> stabilize parts when upstream is lost in tunnel effect mode and does not
> realize that it is wasting everyone's time starting with its own).
>
> Therefore, trying to get all this it a better state, requires the
> following steps IMHO:
>
> 1. completely refactor our Go packaging style it's less painful to update
> Go packages and they do not need a packager with deep knowledge of
> package-specific shell glue. That takes documentation, and factoring out
> common Go packaging functions in shared rpm code (macros and autodeps) →
> that's what I posted
>
> 2. using the new documentation and tooling to clean up years of Fedora
> technical debt, and create a new set of up-to-date Go packages that can
> serve as new baseline → I have hundreds of specs that I'm waiting for step
> 1 to complete to submit. They won't constitute a full baseline by
> themselves, they're not all 100% done, it's too much work to do alone but
> working with others requires the common macros and documentation to be
> merged and adopted. This step is going to be painful 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 11:21:59 AM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Owen Taylor"
> 
> Hi Owen,
> 
> > Is there a guide for Fedora packagers about how to handle unbundling for
> > golang packages? The draft guidelines don't seem to go into any details.
> 
> I don't think there is, nor that it is necessarily needed. The posted
> guidelines should be sufficient technically, there are no magic I know of I
> didn't document, the rest is just a lot of work (but see ↓)
> 
> > I've looked at packaging a few golang packages unbundled, and have
> > immediately run into:
> >
> > A) lots of unpackaged dependencies
> > B) dependencies that are packaged in Fedora with different, often much
> > older versions
> 
> Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't
> expect anyone to be able to package anything but the most trivial app in
> unbundled mode. Too many common parts are missing, when they are not missing
> they are too old, trying to update an existing Go package is an exercise in
> frustration (too much package-specific shell code, that is difficult to
> understand and does not really work with new code versions), and trying to
> update or add missing parts just reveals more breakage and work to be done.

It depends (as everything) on available manpower, if you are willing to own 
your dependencies you can package anything and everything debundled.

> 
> However, accepting to package complex Go apps in bundled mode is how Fedora
> got to this state in the first place. In plain speak, bundling (vendoring in
> Go linguo) just does not scale. You need an awful lot of manpower to audit
> the hundreds of other projects each app bundles, bundling removes all the
> package tooling that may help you to do so, and the result is not shared
> with any other package, or with other versions of the same package, so you
> get zero positive network effects. Worse, big bundled apps that do not
> actually try to work in unbundled mode, do not actually test the code they
> export (they are bundling, remember) so the result is toxic to small Go
> packages that try to work with them.

On contrary, to this state you dislike(or seems to) Fedora got because of 
effort of many to de-bundle Go based projects.

> 
> So bundling parts is a direct road to bundling everything, bundling
> everything is a direct road to bundling everything blindly, bundling
> everything blindly is error-prone and dangerous (because upstreams are only
> human and do make lots of mistakes), and pretty much removes any value
> Fedora can add to end users, to other parts of Fedora that would like to
> integrate with Go software, or to the upstream projects themselves (no
> Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to
> stabilize parts when upstream is lost in tunnel effect mode and does not
> realize that it is wasting everyone's time starting with its own).
> 
> Therefore, trying to get all this it a better state, requires the following
> steps IMHO:
> 
> 1. completely refactor our Go packaging style it's less painful to update Go
> packages and they do not need a packager with deep knowledge of
> package-specific shell glue. That takes documentation, and factoring out
> common Go packaging functions in shared rpm code (macros and autodeps) →
> that's what I posted

I don't see way how it makes it less painful(you have still the whole distro 
issue). It makes specs more abstract and streamlined when it works, but more 
opaque and hard to read when its not(and you have to debug some arcane srpm/lua 
macros).

> 
> 2. using the new documentation and tooling to clean up years of Fedora
> technical debt, and create a new set of up-to-date Go packages that can
> serve as new baseline → I have hundreds of specs that I'm waiting for step 1
> to complete to submit. They won't constitute a full baseline by themselves,
> they're not all 100% done, it's too much work to do alone but working with
> others requires the common macros and documentation to be merged and
> adopted. This step is going to be painful I'm afraid, Fedora dug itself a
> deep hole, leaving it is going to be hard.

On contrary Fedora is trying to fill the hole that upstream Go projects dug 
them selves in to. IM

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Owen Taylor"

Hi Owen,

> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.

I don't think there is, nor that it is necessarily needed. The posted 
guidelines should be sufficient technically, there are no magic I know of I 
didn't document, the rest is just a lot of work (but see ↓)

> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
>
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much
> older versions

Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't 
expect anyone to be able to package anything but the most trivial app in 
unbundled mode. Too many common parts are missing, when they are not missing 
they are too old, trying to update an existing Go package is an exercise in 
frustration (too much package-specific shell code, that is difficult to 
understand and does not really work with new code versions), and trying to 
update or add missing parts just reveals more breakage and work to be done.

However, accepting to package complex Go apps in bundled mode is how Fedora got 
to this state in the first place. In plain speak, bundling (vendoring in Go 
linguo) just does not scale. You need an awful lot of manpower to audit the 
hundreds of other projects each app bundles, bundling removes all the package 
tooling that may help you to do so, and the result is not shared with any other 
package, or with other versions of the same package, so you get zero positive 
network effects. Worse, big bundled apps that do not actually try to work in 
unbundled mode, do not actually test the code they export (they are bundling, 
remember) so the result is toxic to small Go packages that try to work with 
them.

So bundling parts is a direct road to bundling everything, bundling everything 
is a direct road to bundling everything blindly, bundling everything blindly is 
error-prone and dangerous (because upstreams are only human and do make lots of 
mistakes), and pretty much removes any value Fedora can add to end users, to 
other parts of Fedora that would like to integrate with Go software, or to the 
upstream projects themselves (no Fedora QA, no stream of Fedora-originated 
fixes, no Fedora pressure to stabilize parts when upstream is lost in tunnel 
effect mode and does not realize that it is wasting everyone's time starting 
with its own).

Therefore, trying to get all this it a better state, requires the following 
steps IMHO:

1. completely refactor our Go packaging style it's less painful to update Go 
packages and they do not need a packager with deep knowledge of 
package-specific shell glue. That takes documentation, and factoring out common 
Go packaging functions in shared rpm code (macros and autodeps) → that's what I 
posted 

2. using the new documentation and tooling to clean up years of Fedora 
technical debt, and create a new set of up-to-date Go packages that can serve 
as new baseline → I have hundreds of specs that I'm waiting for step 1 to 
complete to submit. They won't constitute a full baseline by themselves, 
they're not all 100% done, it's too much work to do alone but working with 
others requires the common macros and documentation to be merged and adopted. 
This step is going to be painful I'm afraid, Fedora dug itself a deep hole, 
leaving it is going to be hard.

3. hopefully, the result will be streamlined enough it won't be too painful to 
keep up to date, having an up to date baseline will help attract new Go 
packagers like you, everyone will benefit and be happy. We had to package some 
ostree bits for example to create this baseline, and I'm pretty sure ostree 
people within Fedora would prefer to maintain those themselves, if the rest of 
the Fedora go universe didn't make it too hard  

> B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version,

The guidelines and automation aim at making upgrading easy, and avoid one 
package or packager blocking others

> and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.

I hope that normalizing Fedora Go packages means it will be possible to 
automate QA tests such as "try to rebuild all the packages that depend on 
golang(foo) every time you see the package providing golang(foo) changing, and 
report what broke"

That would be expensive computing-side but a lot less expensive and long than 
expecting each Go packager to do it himself with his own means. And that is 
certainly not overkill, given how lax Go projects are about maintaining API 
stability.

And then in case of breakage, revert or create a compat package. That's why 
there is a long chapter dedicated to compat package creation in the proposed 
guidelines.

Regards,

-- 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Owen Taylor" <otay...@redhat.com>
> To: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>
> Cc: gol...@lists.fedoraproject.org, "Discussion of RPM packaging standards 
> and practices for Fedora"
> <packag...@lists.fedoraproject.org>
> Sent: Wednesday, January 31, 2018 6:50:21 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> Hi Nicolas,
> 
> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.
> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
> 
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much older
> versions
> 
> A) is a pretty known quantity for any type of package, but I found B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version, and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.
> 
> Some tutorial showing a practical example of packaging a golang package for
> the first time I think would be very helpful.
> 
> Owen
> 

Unfortunately this is how the situation stand thanks to the wide spread use of 
vendoring and generally no release and API stability in Go projects. You have 
to package all not packaged deps and work with other maintainers(which mostly 
means with Jan Chaloupka, cc'ed) to get other already packaged deps to commonly 
acceptable version/commit level, although it might not be even possible thanks 
to API breaks between commits. So you have to stick to bundling in some cases, 
if you really want to package that project.

JC

> 
> On Wed, Jan 31, 2018 at 5:30 AM, < nicolas.mail...@laposte.net > wrote:
> 
> 
> >De: "Neal Gompa"
> 
> > The only thing I see that might be missing is autogenerating
> > bundled(golang()) Provides when a vendor tree exists (with the
> > appropriate automatic filters on Requires).
> 
> I had though a little about doing it but first, as many Go elements,
> vendoring relies on conventions not standards. The nasty thing about
> conventions is that they are not applied 100% the same way by everyone,
> making automation a PITA. And second interactions with autodeps can be
> nasty: you can filter out provides, but do you filter out requires? What
> about all the junk code many projects ship as testing and examples and which
> is vendored with the rest?
> 
> I don't say it can't be done, or that it would be difficult to do once the
> rest is merged, but I'll live it to someone that absolutely want to ship a
> Go package with vendored parts.
> 
> Right now we sidestep the issue in our packages by rm -fr ing anything that
> looks like vendored code in %prep. Unbundling takes time but it has a
> positive cumulative effect: the more you unbundle the less you need to worry
> about in other packages with the same requirements. And unbundling reveals
> code/legal problems that it would take about as much work to solve by
> auditing the vendored code manually.
> 
> Regards,
> 
> --
> Nicolas mAilhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
> 
> ___
> golang mailing list -- gol...@lists.fedoraproject.org
> To unsubscribe send an email to golang-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Owen Taylor]

Hi Nicolas,

Is there a guide for Fedora packagers about how to handle unbundling for
golang packages? The draft guidelines don't seem to go into any details.
I've looked at packaging a few golang packages unbundled, and have
immediately run into:

 A) lots of unpackaged dependencies
 B) dependencies that are packaged in Fedora with different, often much
older versions

A) is a pretty known quantity for any type of package, but I found B) more
intimidating. It seems like to package the new package, I have to get the
maintainer of the library to upgrade the version, and someone needs to
rebuild everything that depends on the rebuilt library and test that the
rebuilt packages work.

Some tutorial showing a practical example of packaging a golang package for
the first time I think would be very helpful.

Owen


On Wed, Jan 31, 2018 at 5:30 AM,  wrote:

> >De: "Neal Gompa"
>
> > The only thing I see that might be missing is autogenerating
> > bundled(golang()) Provides when a vendor tree exists (with the
> > appropriate automatic filters on Requires).
>
> I had though a little about doing it but first, as many Go elements,
> vendoring relies on conventions not standards. The nasty thing about
> conventions is that they are not applied 100% the same way by everyone,
> making automation a PITA. And second interactions with autodeps can be
> nasty: you can filter out provides, but do you filter out requires? What
> about all the junk code many projects ship as testing and examples and
> which is vendored with the rest?
>
> I don't say it can't be done, or that it would be difficult to do once the
> rest is merged, but I'll live it to someone that absolutely want to ship a
> Go package with vendored parts.
>
> Right now we sidestep the issue in our packages by rm -fr ing anything
> that looks like vendored code in %prep. Unbundling takes time but it has a
> positive cumulative effect: the more you unbundle the less you need to
> worry about in other packages with the same requirements. And unbundling
> reveals code/legal problems that it would take about as much work to solve
> by auditing the vendored code manually.
>
> Regards,
>
> --
> Nicolas mAilhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

>De: "Neal Gompa" 

> The only thing I see that might be missing is autogenerating
> bundled(golang()) Provides when a vendor tree exists (with the
> appropriate automatic filters on Requires).

I had though a little about doing it but first, as many Go elements, vendoring 
relies on conventions not standards. The nasty thing about conventions is that 
they are not applied 100% the same way by everyone, making automation a PITA. 
And second interactions with autodeps can be nasty: you can filter out 
provides, but do you filter out requires? What about all the junk code many 
projects ship as testing and examples and which is vendored with the rest?

I don't say it can't be done, or that it would be difficult to do once the rest 
is merged, but I'll live it to someone that absolutely want to ship a Go 
package with vendored parts.

Right now we sidestep the issue in our packages by rm -fr ing anything that 
looks like vendored code in %prep. Unbundling takes time but it has a positive 
cumulative effect: the more you unbundle the less you need to worry about in 
other packages with the same requirements. And unbundling reveals code/legal 
problems that it would take about as much work to solve by auditing the 
vendored code manually.

Regards,

-- 
Nicolas mAilhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Tue, Jan 30, 2018 at 10:11 AM,   wrote:
> Hi,
>
> Now the technical PR is submitted
> https://src.fedoraproject.org/rpms/go-srpm-macros/pull-request/1
>
> and waiting for action from the go-srpm-macros maintainers, I took (quite a 
> long) time to refresh and flesh out the corresponding packaging guidelines 
> proposal. It should be fairly complete now:
>
> https://fedoraproject.org/wiki/More_Go_packaging
>
> I'd appreciate review to check if I have forgotten an important case, if 
> people understand the text, if they have enhancements or corrections to 
> propose, and so on.
>
> Then I will push it FPC side again.
>
> Actual practice should be fairly simple and self-explanatory, the proposal 
> length can be scary but that's because it documents all kinds of corner cases 
> that required digging through specs and mailing lists to find resolution 
> examples before. The basic Go packaging skeleton will be sufficient is most 
> cases without requiring to read any documentation.
>

The only thing I see that might be missing is autogenerating
bundled(golang()) Provides when a vendor tree exists (with the
appropriate automatic filters on Requires).



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

Hi,

Now the technical PR is submitted
https://src.fedoraproject.org/rpms/go-srpm-macros/pull-request/1

and waiting for action from the go-srpm-macros maintainers, I took (quite a 
long) time to refresh and flesh out the corresponding packaging guidelines 
proposal. It should be fairly complete now:

https://fedoraproject.org/wiki/More_Go_packaging

I'd appreciate review to check if I have forgotten an important case, if people 
understand the text, if they have enhancements or corrections to propose, and 
so on.

Then I will push it FPC side again.

Actual practice should be fairly simple and self-explanatory, the proposal 
length can be scary but that's because it documents all kinds of corner cases 
that required digging through specs and mailing lists to find resolution 
examples before. The basic Go packaging skeleton will be sufficient is most 
cases without requiring to read any documentation. 

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka" 

> Very nice list, it would be nice to have it as sub-wiki page of guidelines. I 
> have took liberty to add
> few points.

Ok, I put it here so people have a place to work on it 
https://fedoraproject.org/wiki/More_Go_packaging#Go_developer_guidance:_making_your_software_third-party-friendly

It can be moved later wherever you feel is more appropriate

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka"

Hi Jakub,

>> For my part I doubt I'll ever use it in EL6 since I did it
>> for Go and the EL6 Go stack is really too old for a merge to be interesting.
>> Anyway I'll certainly let you know when I feel the time is right (but do not
>> block on me!)

> If we are talking about EPEL6 stack, it is fairly fresh(1.9.2) and stable(it 
> will be on 1.9 for whole of its 
> upstream support), although Go packaging macros are missing.

Yes the core golang package is there but first the other Go macros are not 
available in EL6 and second the software level of many common Go packages is 
very old in EL6.

So, assuming the forge macros just work in EL6, because the little bit of rpm 
lua they need works the same as in EL7 and devel, updating EL6 to the level 
needed for a Go spec to be shared mostly unchanged between EL6/EL7 and devel 
would require:

— making the other Go macros available in EL6 (and I'm far from sure they would 
work unchanged in EL6, the forge part is easy since it's 100% lua but the other 
parts exercise the shell and rpm and other stuff that may behave in a slightly 
different way in such an ancient codebase. Not necessarily a show-stopper but 
definitely something that requires testing and adjusting time by someone)

— updating all the common Go software packages to the same level as in devel. I 
don't say it can't be done (after all I'm doing it for EL7 on hundreds of 
packages) but that's a *lot* more work than just updating a few macro files. 
Especially if it hits bootstraping issues not existing in EL7 and devel.

Therefore, it requires someone with time and motivation, and should really not 
be attempted before EL7 and devel are done and things have settled a bit, and 
by that point will there still be enough interest in upgrading the Go state in 
EL6 for it to be worth the pain?

Of course one could limit oneself to making the macros available (which still 
requires some testing and eventually some code porting), that would enable 
using the same Go spec coding style in EL6 EL7 and devel, if not sharing the 
spec themselves, but the interest of autodeps is severely limited, if you do 
not have a large baseline of packages providing the deps (and the dep version) 
other packages need to build.

That's why I wrote the forge part could be made available in EL6, it depends on 
little except the lua built in rpm, and can be useful as-is, while the Go part 
is something else entirely, as its utility is directly linked to the quantity 
and freshness of Go software packages in the distro.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>
> Sent: Tuesday, January 23, 2018 7:45:06 PM
> Subject: Re:  Re: Proposed Fedora packaging guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Mátyás Selmeci"
> 
> Hi,
> 
> > This looks pretty cool!
> 
> Thanks for the feedback!
> 
> > One thing I notice in the limitations section of
> > your draft is a lot of "we can't do XXX due to lack of release
> > discipline..."
> 
> > Do you have any recommendations for Go programmers on how to structure
> > their software so that it is easy to package?
> 
> If there is an interest I can add a section on how to make a Go project
> easier to package, sure.
> 
> It won't be earth-shattering, just the Go declination of basic common sense
> rules that are needed in any coding language, that many Go projects already
> apply (unfortunately not all of them):
> 
> — do not change your import path every other month
> — do not make your code accessible through multiple import paths
> – only use smallcaps in your import path (I know some systems are case
> insensitive. Many others are NOT)
> – communicate clearly the canonical import path of the project at the top of
> your README.md
> — if you absolutely need to change your import path fix your code to use the
> new import path do not rely on http redirections
> – that includes testing and example code
> — do not add a .git suffix to your import path
> 
> — use _testdata for all the material needed by unit test

- make sure that all tests and exclusive test dependencies are really only in 
_test.go files

> – put your example code in _examples (with subdirectories if you ship several
> examples). Do not use creative unusual names such as tutorial.
> – do not pepper your subdirectories with .md files. Keep documentation in the
> project root or in a docs root subdirectory if there is too much of it
> — add a one-line summary and a least a § describing what your project does at
> the top of your README.md
> 
> — choose licenses already vetted by Fedora or Debian
> https://fedoraproject.org/wiki/Licensing:Main?rd=Licensing#Good_Licenses
> – add a licensing file named LICENSE
> — use unmodified plaintext canonical licensing texts, that state the LICENSE
> name at the top of the file
> — if you absolutely want to add an extension to make Windows happy, use
> LICENSE.txt
> — if you absolutely want to name your licensing file something else, please
> do not use something.md
> — do not hide your licensing terms in README.md, do not refer to a license by
> name without providing its text
> 
> – do releases
> — do releases, even for minor fixes. If you haven't felt the need to touch a
> project for months its latest state should be released!
> — do releases, even for projects that can only be called through another
> project. Do not rely on this other project to set code state through
> vendoring (that's easy to do, just propagate the tip project version to the
> ancillary projects at release time, like kubernetes does)
> – use semver for your releases https://semver.org/ Distributors and godep
> will thank you
> – if your project is in git, use a different branch for every major version
> of your project
> — if your project is in git, tag your release x.y.z as x.y.z, or vx.y.z,
> never as vx.y.zprettybeta. Versions and version tags are not the right place
> to document a project maturity.
> — numbers are cheap, never reuse the same number for a pre-release and a
> final release, increase the minor version!
> – please version your import paths with each major release (gopkg.in is good
> for that)

- ideally do major releases in separate branches, so you can do minor 
fixes/releases easily even for older major releases(look at GC) 
- do release whenever you alter your API, extending it, modifying it, removing 
some and document it in the release notes

> 
> – use releases of the projects you depend on
> — never depend on a project that already depends on you (otherwise software
> dependencies stop being a nice directed acyclic graph and you get into
> dependency loop hell. That's a nasic software engineering golden rule, not
> respecting it will bite you sooner or later with or without linux distros,
> despite vendoring)
> – if for some reason, one of the projects you depend on does not release,
> please ask nicely it to do so
> – if for some reason you need a code state in tip which is not in a release,
> please inform the origin you'd like it to do a release, and switch to this
> release as soon as it i

Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Jason L Tibbitts III" <ti...@math.uh.edu>
> To: "nicolas mailhot" <nicolas.mail...@laposte.net>
> Cc: gol...@lists.fedoraproject.org, "Development discussions related to 
> Fedora" <devel@lists.fedoraproject.org>,
> "Discussion of RPM packaging standards and practices for Fedora" 
> <packag...@lists.fedoraproject.org>
> Sent: Tuesday, January 23, 2018 6:00:24 PM
> Subject: Re: Proposed Fedora packaging guideline: More Go packaging
> 
> I wish this message wasn't crossposted everywhere, but I don't want to
> lose any discussion by trimming the CC list.  Sorry if replies generate
> bounces for some.
> 
> >>>>> "nm" == nicolas mailhot <nicolas.mail...@laposte.net> writes:
> 
> nm> And the forge macros are now available since
> nm> redhat-rpm-config-73-1.fc28 (I had missed the push due to upstream
> nm> renaming the file). Heartfelt thanks to Jason Tibbitts !
> 
> Please don't forget to let me know when it's time to start thinking
> about pushing this down to F27.  And maybe F26.  And as far as I can
> tell it should work with only minor modification in EPEL7 (via
> epel-rpm-macros).  I don't know about EPEL6, but we really should look
> at it given some of the other discussions about specfile compatibility.
> Some packagers wouldn't ever use it if it doesn't work everywhere.
> 

I think that it would be great to land it also in the EPEL6/7.

JC


> Finally, we should also talk about whether there is any integration or
> automation possible between fedpkg and specfiles configured with these
> macros.
> 
>  - J<
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: "Jason L Tibbitts III" <ti...@math.uh.edu>
> Cc: gol...@lists.fedoraproject.org, "Development discussions related to 
> Fedora" <devel@lists.fedoraproject.org>,
> "Discussion of RPM packaging standards and practices for Fedora" 
> <packag...@lists.fedoraproject.org>
> Sent: Tuesday, January 23, 2018 9:28:15 PM
> Subject: Re: Proposed Fedora packaging guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Jason L Tibbitts III"
> 
> >>>>> "nm" == nicolas mailhot <nicolas.mail...@laposte.net> writes:
> 
> >nm> And the forge macros are now available since
> >nm> redhat-rpm-config-73-1.fc28 (I had missed the push due to upstream
> >nm> renaming the file). Heartfelt thanks to Jason Tibbitts !
> 
> > Please don't forget to let me know when it's time to start thinking
> > about pushing this down to F27.  And maybe F26.  And as far as I can
> > tell it should work with only minor modification in EPEL7 (via
> > epel-rpm-macros).
> 
> I don't know about EPEL6, but we use it as-is in EL7 and it works just as
> well (except maybe for the %autosetup bits but IIRC that's autosetup which
> is broken in EL7). In fact it has probably been used more heavily in EL7
> than in fedora-devel so far. Maybe it also works in EPEL 6 but I've never
> tried it. I guess it depends mostly on the level of lua support in EL6 rpm
> and rpm-related tools now the forge macro code is lua only. I'm pretty sure
> many of the problems in the early versions of the macro were due to non-lua
> code and its interactions with lua code once the lua-ification started.
> 
> It's a good idea to let people play with it in fedora-devel maybe a month, in
> case I missed something, but from a technical POW I'm prety sure it could be
> merged up to EL7 now. I'll submit fedora-devel specific tweaks later (just
> like I submitted bitkeeper.org support today), right now the code is
> distro-agnostic. For my part I doubt I'll ever use it in EL6 since I did it
> for Go and the EL6 Go stack is really too old for a merge to be interesting.
> Anyway I'll certainly let you know when I feel the time is right (but do not
> block on me!)

If we are talking about EPEL6 stack, it is fairly fresh(1.9.2) and stable(it 
will be on 1.9 for whole of its upstream support), although Go packaging macros 
are missing.

JC

> 
> > Finally, we should also talk about whether there is any integration or
> > automation possible between fedpkg and specfiles configured with these
> > macros.
> 
> I'm afraid my knowledge of recent fedpkg enhancements is too sparse to be of
> any use there. Though I'm not opposed to the idea at all.
> 
> Thanks again!
> 
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Jason L Tibbitts III]

> "nm" == nicolas mailhot  writes:

nm> I don't know about EPEL6, but we use it as-is in EL7 and it works
nm> just as well (except maybe for the %autosetup bits but IIRC that's
nm> autosetup which is broken in EL7).

I had ported autosetup to EPEL6 and then at the next release the macros
showed up (without any discussion) in base RHEL6.  So they should be
there.  I'm not entirely sure what's actually broken about %autosetup in
EL7, though; I hadn't heard about breakage before this.  epel-rpm-macros
could conceivably carry %epel_autosetup or something which contains
fixes.  I mostly understand the internals of %autosetup so maybe there's
something I can do.

nm> Maybe it also works in EPEL 6 but I've never tried it. I guess it
nm> depends mostly on the level of lua support in EL6 rpm and
nm> rpm-related tools now the forge macro code is lua only.

I think it would be worth a try.  In my experience not all that much has
changed in the Lua interfaces since RHEL6 and even RHEL5.  (EL5 just
didn't have sources and patches in the Lua namespace.)

nm> For my part I doubt I'll ever use it in EL6 since I did it for Go
nm> and the EL6 Go stack is really too old for a merge to be
nm> interesting.

Well, sure, Go on EL6 is probably out but these macros were at least
presented as being far more general, and I'm sure there are plenty of
EPEL6 packages which could benefit.  Potential anything that packages a
git snapshot of something.

nm> I'm afraid my knowledge of recent fedpkg enhancements is too sparse
nm> to be of any use there. Though I'm not opposed to the idea at all.

It's just worth a brainstorm, I think.  I can imagine that loads of
people would love it if fedpkg could just auto-bump the bits necessary
to update a package to today's git head or some specific commit or
something like that.  Before we didn't really have a good standard way
to format a spec when you're using a checkout.  Now

Doesn't necessarily have to start in fedpkg, either.  A standalone
utility for doing a couple of things like that could be useful as a
prototype.

 - J<
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jason L Tibbitts III" 

> "nm" == nicolas mailhot  writes:

>nm> And the forge macros are now available since
>nm> redhat-rpm-config-73-1.fc28 (I had missed the push due to upstream
>nm> renaming the file). Heartfelt thanks to Jason Tibbitts !

> Please don't forget to let me know when it's time to start thinking
> about pushing this down to F27.  And maybe F26.  And as far as I can
> tell it should work with only minor modification in EPEL7 (via
> epel-rpm-macros).  

I don't know about EPEL6, but we use it as-is in EL7 and it works just as well 
(except maybe for the %autosetup bits but IIRC that's autosetup which is broken 
in EL7). In fact it has probably been used more heavily in EL7 than in 
fedora-devel so far. Maybe it also works in EPEL 6 but I've never tried it. I 
guess it depends mostly on the level of lua support in EL6 rpm and rpm-related 
tools now the forge macro code is lua only. I'm pretty sure many of the 
problems in the early versions of the macro were due to non-lua code and its 
interactions with lua code once the lua-ification started.

It's a good idea to let people play with it in fedora-devel maybe a month, in 
case I missed something, but from a technical POW I'm prety sure it could be 
merged up to EL7 now. I'll submit fedora-devel specific tweaks later (just like 
I submitted bitkeeper.org support today), right now the code is 
distro-agnostic. For my part I doubt I'll ever use it in EL6 since I did it for 
Go and the EL6 Go stack is really too old for a merge to be interesting. Anyway 
I'll certainly let you know when I feel the time is right (but do not block on 
me!)

> Finally, we should also talk about whether there is any integration or
> automation possible between fedpkg and specfiles configured with these
> macros.

I'm afraid my knowledge of recent fedpkg enhancements is too sparse to be of 
any use there. Though I'm not opposed to the idea at all.

Thanks again!

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Mátyás Selmeci" 

Hi,

> This looks pretty cool!

Thanks for the feedback!

> One thing I notice in the limitations section of 
> your draft is a lot of "we can't do XXX due to lack of release 
> discipline..."

> Do you have any recommendations for Go programmers on how to structure 
> their software so that it is easy to package?

If there is an interest I can add a section on how to make a Go project easier 
to package, sure.

It won't be earth-shattering, just the Go declination of basic common sense 
rules that are needed in any coding language, that many Go projects already 
apply (unfortunately not all of them):

— do not change your import path every other month
— do not make your code accessible through multiple import paths
– only use smallcaps in your import path (I know some systems are case 
insensitive. Many others are NOT)
– communicate clearly the canonical import path of the project at the top of 
your README.md
— if you absolutely need to change your import path fix your code to use the 
new import path do not rely on http redirections
– that includes testing and example code
— do not add a .git suffix to your import path

— use _testdata for all the material needed by unit test
– put your example code in _examples (with subdirectories if you ship several 
examples). Do not use creative unusual names such as tutorial.
– do not pepper your subdirectories with .md files. Keep documentation in the 
project root or in a docs root subdirectory if there is too much of it
— add a one-line summary and a least a § describing what your project does at 
the top of your README.md

— choose licenses already vetted by Fedora or Debian 
https://fedoraproject.org/wiki/Licensing:Main?rd=Licensing#Good_Licenses
– add a licensing file named LICENSE
— use unmodified plaintext canonical licensing texts, that state the LICENSE 
name at the top of the file
— if you absolutely want to add an extension to make Windows happy, use 
LICENSE.txt
— if you absolutely want to name your licensing file something else, please do 
not use something.md
— do not hide your licensing terms in README.md, do not refer to a license by 
name without providing its text

– do releases
— do releases, even for minor fixes. If you haven't felt the need to touch a 
project for months its latest state should be released!
— do releases, even for projects that can only be called through another 
project. Do not rely on this other project to set code state through vendoring 
(that's easy to do, just propagate the tip project version to the ancillary 
projects at release time, like kubernetes does)
– use semver for your releases https://semver.org/ Distributors and godep will 
thank you
– if your project is in git, use a different branch for every major version of 
your project
— if your project is in git, tag your release x.y.z as x.y.z, or vx.y.z, never 
as vx.y.zprettybeta. Versions and version tags are not the right place to 
document a project maturity.
— numbers are cheap, never reuse the same number for a pre-release and a final 
release, increase the minor version!
– please version your import paths with each major release (gopkg.in is good 
for that)

– use releases of the projects you depend on
— never depend on a project that already depends on you (otherwise software 
dependencies stop being a nice directed acyclic graph and you get into 
dependency loop hell. That's a nasic software engineering golden rule, not 
respecting it will bite you sooner or later with or without linux distros, 
despite vendoring)
– if for some reason, one of the projects you depend on does not release, 
please ask nicely it to do so
– if for some reason you need a code state in tip which is not in a release, 
please inform the origin you'd like it to do a release, and switch to this 
release as soon as it is available
– never depend on a commit state somewhere between two releases
— document the major versions of the stuff you depend on somewhere easy to 
find. If a major version is only usable past as specific minor version, 
document it
– add a unit test that detects if the project you depend on is missing the part 
that requires being after this minor version

– if your project provides wrappers, connexion glue or anything similar to many 
many many other projects keep the code for each other project in a separate 
subdirectory (Go package) so it can be desactivated without impacting the rest 
of your code if the other project ode has a problem.

— never add changes to the projects you reuse, submit the changes to those 
projects
— if you absolutely need to change a project you reuse, fork it cleanly with a 
new import path so distributors do not accidentally reinject the original 
project
— don't forget to rebase your code on the original project if you don't have 
the energy to maintain your own fork
– rebase to the latest minor version of every project you depend on at release 
time. Do not let changes accumulate 

Re: Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "nicolas mailhot" 
> Now that the non-Go part in redhat-rpm-macros is merged in devel I'll try to 
> do a clean PR on go-srpm-macros. 

> Then once Jan or Jakub accepts it it will be possible to play with the 
> automation in devel and I'll be able to share my specs somewhere (not that 
> the work is finished, but some parts *are* finished and  I'd rather have the 
> people who
> use those packages to review my conversions rather than redo part of them 
> without sharing existing work)

And the PR is done and sent:

https://src.fedoraproject.org/rpms/go-srpm-macros/pull-request/1

Now I only need to revisit
https://fedoraproject.org/wiki/More_Go_packaging

and enhance it with the packaging patterns that emerged after applying the 
proposal to the last few hundreds of Go spec files.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Neal Gompa"


> For snipping, use "[...]" notation to indicate skipped stuff. It's
> hard to tell otherwise.

Ok, that was easy to fix :)

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Jason L Tibbitts III]

I wish this message wasn't crossposted everywhere, but I don't want to
lose any discussion by trimming the CC list.  Sorry if replies generate
bounces for some.

> "nm" == nicolas mailhot  writes:

nm> And the forge macros are now available since
nm> redhat-rpm-config-73-1.fc28 (I had missed the push due to upstream
nm> renaming the file). Heartfelt thanks to Jason Tibbitts !

Please don't forget to let me know when it's time to start thinking
about pushing this down to F27.  And maybe F26.  And as far as I can
tell it should work with only minor modification in EPEL7 (via
epel-rpm-macros).  I don't know about EPEL6, but we really should look
at it given some of the other discussions about specfile compatibility.
Some packagers wouldn't ever use it if it doesn't work everywhere.

Finally, we should also talk about whether there is any integration or
automation possible between fedpkg and specfiles configured with these
macros.

 - J<
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Mátyás Selmeci]

On 12/17/2017 01:11 AM, nicolas.mail...@laposte.net wrote:

Hi,

I am proposing for inclusion a set of rpm technical files aimed at automating 
the packaging of forge-hosted projects.

- Packaging draft: https://fedoraproject.org/wiki/More_Go_packaging
- https://pagure.io/packaging-committee/issue/734
- go-srpm-macros RFE with the technical files: 
https://bugzilla.redhat.com/show_bug.cgi?id=1526721


Hi,

This looks pretty cool! One thing I notice in the limitations section of 
your draft is a lot of "we can't do XXX due to lack of release 
discipline..."


Do you have any recommendations for Go programmers on how to structure 
their software so that it is easy to package?


Thanks,
-Mat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Tue, Jan 23, 2018 at 9:40 AM,   wrote:
>
>> - Mail original -
>> De: "Neal Gompa"
>
>>> I'm curious, what are you missing in the preamble ? As far as I can see 
>>> it's all there (even though some values
>>> set to variables %gometa precomputes). I had it's right autogenerated some 
>>> parts of it in the past but it's all
>>> converted to variable use now to avoid packager surprise and permit 
>>> customization.
>
>> Actually, this is fine with me. This is what disturbed me a bit:
>> https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation#Packaging_examples
>
> Ah, yes the examples snip the lines that do not need specific changes for 
> readability
> Is the … not clear enough?
> I fear that putting a full preamble would be even more confusing for some 
> readers. Though if people disagree, I'l try to find some time to add the 
> skipped lines back in
>

For snipping, use "[...]" notation to indicate skipped stuff. It's
hard to tell otherwise.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: [Fedora-packaging] Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

> - Mail original -
> De: "Neal Gompa"

>> I'm curious, what are you missing in the preamble ? As far as I can see it's 
>> all there (even though some values
>> set to variables %gometa precomputes). I had it's right autogenerated some 
>> parts of it in the past but it's all
>> converted to variable use now to avoid packager surprise and permit 
>> customization.

> Actually, this is fine with me. This is what disturbed me a bit:
> https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation#Packaging_examples

Ah, yes the examples snip the lines that do not need specific changes for 
readability
Is the … not clear enough?
I fear that putting a full preamble would be even more confusing for some 
readers. Though if people disagree, I'l try to find some time to add the 
skipped lines back in

-- 
Nicolas Mailhot



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Fabio Valentini" 

> So, if I understand correctly, both the forge stuff and the new macros for
> go packaging are completely opt-in?
> If that's correct, this looks like the best solution to me - as old
> packages can then be converted one at a time (which I am looking forward to
> doing for my go packages, btw.).

It is fully opt-in. You can also use some parts and ignore others (though it's 
probably more complex than just using all of it unless you understand the new 
parts really well).

However I should warn that packaging "new-style" something is likely to force 
conversion to "new-style" at least some of its deps."Old-style" manual 
declaration of provides often forgets elements and autodeps have no mercy on 
missing provides. But, some of the "new-style" specs I did depend on existing 
Fedora packages I didn't have to touch.

Now that the non-Go part in redhat-rpm-macros is merged in devel I'll try to do 
a clean PR on go-srpm-macros. 

Then once Jan or Jakub accepts it it will be possible to play with the 
automation in devel and I'll be able to share my specs somewhere (not that the 
work is finished, but some parts *are* finished and  I'd rather have the people 
who use those packages to review my conversions rather than redo part of them 
without sharing existing work)

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Tue, Jan 23, 2018 at 9:00 AM,   wrote:
>
>
> - Mail original -
> De: "Neal Gompa"
>
>> As long as I can do Obsoletes/Provides for the old name for the devel,
>> unit-test,
>
> BTW is anyone using the unit-test packages? Right now I do not generate them, 
> I don't need them, and making them work with autodeps would be hairy 
> (deploying without autodeps should be trivial however)
>
> To be honest, given all the parts current packages fail to install, I'd 
> expect many of the current unit test packages to fail in mysterious ways, so 
> I'm curious: what use has been found for them?
>

No idea, but I just checked, and I don't even have the unit-test
subpackage enabled on snapd, so I don't particularly care too much
there...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Neal Gompa" 

> As long as I can do Obsoletes/Provides for the old name for the devel,
> unit-test, 

BTW is anyone using the unit-test packages? Right now I do not generate them, I 
don't need them, and making them work with autodeps would be hairy (deploying 
without autodeps should be trivial however)

To be honest, given all the parts current packages fail to install, I'd expect 
many of the current unit test packages to fail in mysterious ways, so I'm 
curious: what use has been found for them?

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Tue, Jan 23, 2018 at 8:54 AM,   wrote:
>
>
> - Mail original -
> De: "Neal Gompa"
>
>>>  2. if your concern is that the *forge* macros are defective somewhere I'd 
>>> be curious where as you'd be the
>>> first to report an actual technical problem. I've used them intensively in 
>>> rawhide and el7 with many different
>>>rpm tools and they are rock solid for me
>>
>
>> I don't consider them defective. They make me a bit squeamish because
>> I don't see the spec preamble and such because they're autogenerated,
>> but you've mentioned that it's possible to selectively use these
>> things with Go and forge macros.
>
> I'm curious, what are you missing in the preamble ? As far as I can see it's 
> all there (even though some values set to variables %gometa precomputes). I 
> had it's right autogenerated some parts of it in the past but it's all 
> converted to variable use now to avoid packager surprise and permit 
> customization.
>
> For example, what are you missing there?
>
> %<---
> %global goipathgithub.com/hashicorp/consul
> Version:   1.0.2
>
> %gometa
>
> %global common_description %{expand:
> Consul is a tool for service discovery […]}
>
> Name:consul
> Release: 5%{?dist}
> Summary: A tool for easy service discovery, monitoring and configuration
> License: MPLv2.0
> URL: https://www.consul.io/
> Source:  %{gosource}
> %<---
>
> Or here ?
>
> %<---
> %global goipathgithub.com/hashicorp/go-sockaddr
> %global commit 9b4c5fa5b10a683339a270d664474b9f4aee62fc
>
> %gometa
>
> %global common_description %{expand:
> Socket address convenience functions for Go. go-sockaddr is a convenience
> library…}
>
> Name:%{goname}
> Version: 0
> Release: 0.14%{?dist}
> Summary: A Go convenience library to manipulate IP Addresses and UNIX sockets
> License: MPLv2.0
> URL: %{gourl}
> Source:  %{gosource}
> %<---
>

Actually, this is fine with me. This is what disturbed me a bit:
https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation#Packaging_examples



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Neal Gompa" 

>>  2. if your concern is that the *forge* macros are defective somewhere I'd 
>> be curious where as you'd be the
>> first to report an actual technical problem. I've used them intensively in 
>> rawhide and el7 with many different
>>rpm tools and they are rock solid for me
>

> I don't consider them defective. They make me a bit squeamish because
> I don't see the spec preamble and such because they're autogenerated,
> but you've mentioned that it's possible to selectively use these
> things with Go and forge macros.

I'm curious, what are you missing in the preamble ? As far as I can see it's 
all there (even though some values set to variables %gometa precomputes). I had 
it's right autogenerated some parts of it in the past but it's all converted to 
variable use now to avoid packager surprise and permit customization.

For example, what are you missing there?

%<---
%global goipathgithub.com/hashicorp/consul
Version:   1.0.2

%gometa

%global common_description %{expand:
Consul is a tool for service discovery […]}

Name:consul
Release: 5%{?dist}
Summary: A tool for easy service discovery, monitoring and configuration
License: MPLv2.0
URL: https://www.consul.io/
Source:  %{gosource}
%<---

Or here ?

%<---
%global goipathgithub.com/hashicorp/go-sockaddr
%global commit 9b4c5fa5b10a683339a270d664474b9f4aee62fc

%gometa

%global common_description %{expand:
Socket address convenience functions for Go. go-sockaddr is a convenience
library…}

Name:%{goname}
Version: 0
Release: 0.14%{?dist}
Summary: A Go convenience library to manipulate IP Addresses and UNIX sockets
License: MPLv2.0
URL: %{gourl}
Source:  %{gosource}
%<---

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Tue, Jan 23, 2018 at 5:45 AM,   wrote:
>
>
> - Mail original -
> De: "Neal Gompa"
>
>>On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
>>
 I really do like this. There are only two issues I have with it:

 1. This seems to mandate that all packages must be named by their
 import path. My golang package (snapd) is not, intentionally so. I
 don't want to change this.

 2. Mandating a forge is going to be tricky for self-hosted stuff, or
 people who release Go code as tarballs (it's rare, but it happens).
 How do you deal with that?
>>>
>>> By not using the macros for packages not fitting the model?
>>>
>
>> The issue is that the new Go macros are tightly wound into the forge
>> macros. I just want to be sure that we can leverage things like the
>> dependency generators without all the other stuff.
>
> Hi Neal,
>
> I should probably not let this pass without clarifying:
>
>  1.  if your concern is that the *forge* macros will block packaging of 
> projects not hosted on known forges, they won't:
>   — they include code to disable transparently and without exploding the 
> associated bits if the hosting site is unknown
>   — the packager can pre-define the variables that would have been computed 
> if the site was known and all the rest of the automation will just work
>   — baring that the packager can ignore the site-related computations and use 
> whatever he wants instead in Source: and *setup, which are the only parts 
> that depend on hosting structure knowledge,
>

This was my main concern. There are plenty of people who don't like
any of the known forges. And Go ecosystem incompetence for mandating
URI-style deps and build paths doesn't really stop people from wanting
to do that.

>  2. if your concern is that the *forge* macros are defective somewhere I'd be 
> curious where as you'd be the first to report an actual technical problem. 
> I've used them intensively in rawhide and el7 with many different rpm tools 
> and they are rock solid for me
>

I don't consider them defective. They make me a bit squeamish because
I don't see the spec preamble and such because they're autogenerated,
but you've mentioned that it's possible to selectively use these
things with Go and forge macros.

SUSE does something similar with Python for their "singlespec" part,
and it's fantastic[1][2]. But I'm slightly uncomfortable with the idea
that there's no preamble *at all* in the spec using the Go macros and
it's generated fully by macros.

[1]: https://github.com/openSUSE/python-rpm-macros
[2]: 
https://build.opensuse.org/package/view_file/openSUSE:Factory/python-iniparse/python-iniparse.spec?expand=1

>  3. if your concern is that they won't ever be merged due to some people 
> non-constructive obstruction, well I share it to some point but I won't spend 
> more time arguing unicorns and what-if-maybe 
> I-don't-like-commits-let's-make-them-awful-to-use. The people blocking there 
> are free to take up the maintenance of my ~400 specs and redo them as they 
> want, we'll see how they love to waste tens of minutes per spec to position 
> setup args manually when they have to do it for hundreds of them, every time 
> there is a package update or a forge relayout.
>

Technically, no one has to "accept" anything for the macros to be
available. It's whether the macros would be broadly used. I could, in
fact, tomorrow, make various extra useful macros that I use available
in a package for people to pull in. It doesn't mean anyone has to use
them.

> The main difficulty with the proposal is not the *forge part, it's that 
> automated autodeps are well, automated. As every automation they are strict 
> and unyielding and do not take approximations well. They will fail 
> spectacularly in the following cases:
>
>   1. the packager does not package all the deps of its code → the resulting 
> package is uninstallable because autodeps add requires on missing bits
>
>   2. the packager packages code with garbage imports (very common in example 
> or test code) → the resulting package is uninstallable because autodeps add 
> require
>
>   3. the packager does not own properly the directories where its Go code is 
> installed → given that Go deps depend on directories, Go autodeps also depend 
> on proper directory ownership. Positioning autodeps on Go files themselves 
> would make package build times increase many many times as rpm would try to 
> compute the same autodeps for every single .go file in a directory. (and Go 
> autodeps are already slw as snails because invoking the go command is 
> slow)
>
> All those things are mitigated by the use of %goinstall that filters usual 
> example/test/vendor dirs so they don't trigger autodeps, and tries very hard 
> to own all relevant directories.
>
> For those reasons I don't propose to activate autodeps in old-style golang 
> packages. They need conversion  (and review by a human to check no problem 
> 

Re: Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Neal Gompa"


>> The issue is that the new Go macros are tightly wound into the forge
>> macros. I just want to be sure that we can leverage things like the
>> dependency generators without all the other stuff.

Hi Neal,

>I should probably not let this pass without clarifying:
> 3. if your concern is that they won't ever be merged 

And the forge macros are now available since redhat-rpm-config-73-1.fc28 (I had 
missed the push due to upstream renaming the file). Heartfelt thanks to  Jason 
Tibbitts !

I'll focus on pushing the Go parts to go-srpm-macros now so everyone can play 
with them

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[Fabio Valentini]

On Jan 23, 2018 12:41,  wrote:



- Mail original -
De: "Neal Gompa"

>On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
>
>>> I really do like this. There are only two issues I have with it:
>>>
>>> 1. This seems to mandate that all packages must be named by their
>>> import path. My golang package (snapd) is not, intentionally so. I
>>> don't want to change this.
>>>
>>> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
>>> people who release Go code as tarballs (it's rare, but it happens).
>>> How do you deal with that?
>>
>> By not using the macros for packages not fitting the model?
>>

> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.

Hi Neal,

I should probably not let this pass without clarifying:

 1.  if your concern is that the *forge* macros will block packaging of
projects not hosted on known forges, they won't:
  — they include code to disable transparently and without exploding the
associated bits if the hosting site is unknown
  — the packager can pre-define the variables that would have been computed
if the site was known and all the rest of the automation will just work
  — baring that the packager can ignore the site-related computations and
use whatever he wants instead in Source: and *setup, which are the only
parts that depend on hosting structure knowledge,

 2. if your concern is that the *forge* macros are defective somewhere I'd
be curious where as you'd be the first to report an actual technical
problem. I've used them intensively in rawhide and el7 with many different
rpm tools and they are rock solid for me

 3. if your concern is that they won't ever be merged due to some people
non-constructive obstruction, well I share it to some point but I won't
spend more time arguing unicorns and what-if-maybe
I-don't-like-commits-let's-make-them-awful-to-use. The people blocking
there are free to take up the maintenance of my ~400 specs and redo them as
they want, we'll see how they love to waste tens of minutes per spec to
position setup args manually when they have to do it for hundreds of them,
every time there is a package update or a forge relayout.

The main difficulty with the proposal is not the *forge part, it's that
automated autodeps are well, automated. As every automation they are strict
and unyielding and do not take approximations well. They will fail
spectacularly in the following cases:

  1. the packager does not package all the deps of its code → the resulting
package is uninstallable because autodeps add requires on missing bits

  2. the packager packages code with garbage imports (very common in
example or test code) → the resulting package is uninstallable because
autodeps add require

  3. the packager does not own properly the directories where its Go code
is installed → given that Go deps depend on directories, Go autodeps also
depend on proper directory ownership. Positioning autodeps on Go files
themselves would make package build times increase many many times as rpm
would try to compute the same autodeps for every single .go file in a
directory. (and Go autodeps are already slw as snails because invoking
the go command is slow)

All those things are mitigated by the use of %goinstall that filters usual
example/test/vendor dirs so they don't trigger autodeps, and tries very
hard to own all relevant directories.

For those reasons I don't propose to activate autodeps in old-style golang
packages. They need conversion  (and review by a human to check no problem
code is deployed) first.


So, if I understand correctly, both the forge stuff and the new macros for
go packaging are completely opt-in?
If that's correct, this looks like the best solution to me - as old
packages can then be converted one at a time (which I am looking forward to
doing for my go packages, btw.).

Fabio

Regards,

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Neal Gompa"

>On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
>
>>> I really do like this. There are only two issues I have with it:
>>>
>>> 1. This seems to mandate that all packages must be named by their
>>> import path. My golang package (snapd) is not, intentionally so. I
>>> don't want to change this.
>>>
>>> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
>>> people who release Go code as tarballs (it's rare, but it happens).
>>> How do you deal with that?
>>
>> By not using the macros for packages not fitting the model?
>>

> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.

Hi Neal,

I should probably not let this pass without clarifying:

 1.  if your concern is that the *forge* macros will block packaging of 
projects not hosted on known forges, they won't:
  — they include code to disable transparently and without exploding the 
associated bits if the hosting site is unknown
  — the packager can pre-define the variables that would have been computed if 
the site was known and all the rest of the automation will just work
  — baring that the packager can ignore the site-related computations and use 
whatever he wants instead in Source: and *setup, which are the only parts that 
depend on hosting structure knowledge, 

 2. if your concern is that the *forge* macros are defective somewhere I'd be 
curious where as you'd be the first to report an actual technical problem. I've 
used them intensively in rawhide and el7 with many different rpm tools and they 
are rock solid for me

 3. if your concern is that they won't ever be merged due to some people 
non-constructive obstruction, well I share it to some point but I won't spend 
more time arguing unicorns and what-if-maybe 
I-don't-like-commits-let's-make-them-awful-to-use. The people blocking there 
are free to take up the maintenance of my ~400 specs and redo them as they 
want, we'll see how they love to waste tens of minutes per spec to position 
setup args manually when they have to do it for hundreds of them, every time 
there is a package update or a forge relayout.

The main difficulty with the proposal is not the *forge part, it's that 
automated autodeps are well, automated. As every automation they are strict and 
unyielding and do not take approximations well. They will fail spectacularly in 
the following cases:

  1. the packager does not package all the deps of its code → the resulting 
package is uninstallable because autodeps add requires on missing bits

  2. the packager packages code with garbage imports (very common in example or 
test code) → the resulting package is uninstallable because autodeps add require

  3. the packager does not own properly the directories where its Go code is 
installed → given that Go deps depend on directories, Go autodeps also depend 
on proper directory ownership. Positioning autodeps on Go files themselves 
would make package build times increase many many times as rpm would try to 
compute the same autodeps for every single .go file in a directory. (and Go 
autodeps are already slw as snails because invoking the go command is slow)

All those things are mitigated by the use of %goinstall that filters usual 
example/test/vendor dirs so they don't trigger autodeps, and tries very hard to 
own all relevant directories.

For those reasons I don't propose to activate autodeps in old-style golang 
packages. They need conversion  (and review by a human to check no problem code 
is deployed) first.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Neal Gompa" 

Hi,

Thanks for the review !

> I really do like this. There are only two issues I have with it:

> 1. This seems to mandate that all packages must be named by their
> import path. My golang package (snapd) is not, intentionally so. I
> don't want to change this.

Well that was not the intent, I probably need to clarify things a bit more. The 
*devel* package must absolutely be named after the import path (for sanity 
sake). The core package can be named something else (usually because it's the 
packaging of an app)

For example in my etcd spec

Name:etcd ← well-known app name
%package   -n %{goname}-devel ← go naming for go devel stuff

> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> people who release Go code as tarballs (it's rare, but it happens).
> How do you deal with that?

The proposal automates integration with forges, it does not mandate it

You can declare manually archivename archiveurl and forgesetup before calling 
the %gometa macro and 
you should benefit from the rest of the automation even on an hosting site it 
does not know anything about. The whole code tries very hard to let the 
packager pre-define everything it may guess wrong or not know about to avoid 
giving up on all the automation just because a little part is wrong.

And even after that you can still declare Source manually and pass arguments to 
setup the old way it that's what you want

Not calling %gometa at all will kill stuff like goname which is kind of 
mandatory for consistency.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Marcin Dulak" <marcin.du...@gmail.com>
> To: "Discussion of RPM packaging standards and practices for Fedora" 
> <packag...@lists.fedoraproject.org>
> Cc: gol...@lists.fedoraproject.org, "Development discussions related to 
> Fedora" <devel@lists.fedoraproject.org>
> Sent: Monday, January 22, 2018 4:04:19 PM
> Subject: Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More 
> Go packaging
> 
> 
> 
> On Mon, Jan 22, 2018 at 2:45 PM, Neal Gompa < ngomp...@gmail.com > wrote:
> 
> 
> On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
> < dridi.boukelmo...@gmail.com > wrote:
> >> I really do like this. There are only two issues I have with it:
> >> 
> >> 1. This seems to mandate that all packages must be named by their
> >> import path. My golang package (snapd) is not, intentionally so. I
> >> don't want to change this.
> >> 
> >> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> >> people who release Go code as tarballs (it's rare, but it happens).
> >> How do you deal with that?
> > 
> > By not using the macros for packages not fitting the model?
> > 
> 
> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.
> 
> > I think this is very helpful especially when it's the common practice,
> > and I certainly won't blame anyone doing proper releases and not
> > just a git tag with github releases notes ;)
> > 
> > Regarding naming, I think python packages must be prefixed with
> > python[23]- and can Provides: the upstream project name.
> 
> I'm not sure if this matters in this discussion but an example Python3 part
> of a spec file https://fedoraproject.org/wiki/Packaging:Python
> to accommodate also EPEL (which on CentOS7 prefixes Python3 packages with
> python34 and not python3) would look like:
> 
> %package -n python%{python3_pkgversion}-%{pname}
> %{?python_provide:%python_provide python%{python3_pkgversion}-%{pname}}
> 
> Macin
> 

Hopefully something like this will never happen as generally I'm strongly 
against shipping multiple versions(of one implementation) of Go concurrently.

JC

> 
> On the
> > other hand we have packages like docker that are clearly named
> > after upstream's name, so I don't think that would be a problem for
> > snapd. (and maybe an exception needs to be granted?)
> > 
> 
> This rule only applies to Python packages that have modules that are
> designed to be imported by other Python code. Otherwise, this is not
> necessary.
> 
> 
> 
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
> 
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Marcin Dulak]

On Mon, Jan 22, 2018 at 2:45 PM, Neal Gompa  wrote:

> On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
>  wrote:
> >> I really do like this. There are only two issues I have with it:
> >>
> >> 1. This seems to mandate that all packages must be named by their
> >> import path. My golang package (snapd) is not, intentionally so. I
> >> don't want to change this.
> >>
> >> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> >> people who release Go code as tarballs (it's rare, but it happens).
> >> How do you deal with that?
> >
> > By not using the macros for packages not fitting the model?
> >
>
> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.
>
> > I think this is very helpful especially when it's the common practice,
> > and I certainly won't blame anyone doing proper releases and not
> > just a git tag with github releases notes ;)
> >
> > Regarding naming, I think python packages must be prefixed with
> > python[23]- and can Provides: the upstream project name.


I'm not sure if this matters in this discussion but an example Python3 part
of a spec file https://fedoraproject.org/wiki/Packaging:Python
to accommodate also EPEL (which on CentOS7 prefixes Python3 packages with
python34 and not python3) would look like:

%package -n python%{python3_pkgversion}-%{pname}
%{?python_provide:%python_provide python%{python3_pkgversion}-%{pname}}

Macin


> On the
> > other hand we have packages like docker that are clearly named
> > after upstream's name, so I don't think that would be a problem for
> > snapd. (and maybe an exception needs to be granted?)
> >
>
> This rule only applies to Python packages that have modules that are
> designed to be imported by other Python code. Otherwise, this is not
> necessary.
>
>
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
 wrote:
>> I really do like this. There are only two issues I have with it:
>>
>> 1. This seems to mandate that all packages must be named by their
>> import path. My golang package (snapd) is not, intentionally so. I
>> don't want to change this.
>>
>> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
>> people who release Go code as tarballs (it's rare, but it happens).
>> How do you deal with that?
>
> By not using the macros for packages not fitting the model?
>

The issue is that the new Go macros are tightly wound into the forge
macros. I just want to be sure that we can leverage things like the
dependency generators without all the other stuff.

> I think this is very helpful especially when it's the common practice,
> and I certainly won't blame anyone doing proper releases and not
> just a git tag with github releases notes ;)
>
> Regarding naming, I think python packages must be prefixed with
> python[23]- and can Provides: the upstream project name. On the
> other hand we have packages like docker that are clearly named
> after upstream's name, so I don't think that would be a problem for
> snapd. (and maybe an exception needs to be granted?)
>

This rule only applies to Python packages that have modules that are
designed to be imported by other Python code. Otherwise, this is not
necessary.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Dridi Boukelmoune]

> I really do like this. There are only two issues I have with it:
>
> 1. This seems to mandate that all packages must be named by their
> import path. My golang package (snapd) is not, intentionally so. I
> don't want to change this.
>
> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> people who release Go code as tarballs (it's rare, but it happens).
> How do you deal with that?

By not using the macros for packages not fitting the model?

I think this is very helpful especially when it's the common practice,
and I certainly won't blame anyone doing proper releases and not
just a git tag with github releases notes ;)

Regarding naming, I think python packages must be prefixed with
python[23]- and can Provides: the upstream project name. On the
other hand we have packages like docker that are clearly named
after upstream's name, so I don't think that would be a problem for
snapd. (and maybe an exception needs to be granted?)

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Proposed Fedora packaging guideline: More Go packaging

[Neal Gompa]

On Sun, Dec 17, 2017 at 2:11 AM,   wrote:
> Hi,
>
> I am proposing for inclusion a set of rpm technical files aimed at automating 
> the packaging of forge-hosted projects.
>
> - Packaging draft: https://fedoraproject.org/wiki/More_Go_packaging
> - https://pagure.io/packaging-committee/issue/734
> - go-srpm-macros RFE with the technical files: 
> https://bugzilla.redhat.com/show_bug.cgi?id=1526721
>
> This proposal is integrated with and depends on the 
> https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation 
> draft
> It builds on the hard work of the Go SIG and reuses the rpm automation of 
> https://fedoraproject.org/wiki/PackagingDrafts/Go when it exists, and 
> produces compatible packages.
>
> What it does:
>
> - drastically shorter spec files, up to 90% in some cases, often removing 
> hundreds of lines per spec.
> - simple, packager-friendly spec syntax
> - automated package naming derived from the native identifier (import path). 
> No more packages names without any relation with current upstream naming.
> - working Go autoprovides. No forgotten provides anymore.
> - working Go autorequires. No forgotten requires anymore.
> - strict automated directory ownership (used by autorequires and 
> autoprovides).
> - centralized computation of source URLs (via Forge-hosted projects packaging 
> automation). No more packages lacking guidelines. No more broken guidelines 
> no one notices.
> - easy switch between commits, tags and releases (via Forge-hosted projects 
> packaging automation). No more packages stuck on commits when upstream starts 
> releasing.
> - guidelines-compliant automated snapshot naming, including snapshot 
> timestamps (via Forge-hosted projects packaging automation). No more packages 
> stuck in 2014.
> - guidelines-compliant bootstraping.
> - systematic use of the Go switches defined by the Go maintainer. Easy to do 
> changes followed by a mass rebuild.
> - flexibility, do the right thing transparently by default, leave room for 
> special cases and overrides.
> - no bundling (a.k.a. vendoring) due to the pain of packaging one more Go 
> dependency.
> - centralized Go macros that can be audited and enhanced over time.
> - aggressive leverage of upstream unit tests to detect quickly broken code.
>
> Please consult packaging draft for full information.
>
> The proposal has been tested in Rawhide and EL7 over a set of ~ 140 Go 
> packages. This set is a mix of current Fedora packages, bumped to a more 
> recent version, rewrites of Fedora packages, and completely new packages.
>
> I hope posting the second part of the automation will answer some questions 
> people had on the 
> https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation 
> draft
>

I really do like this. There are only two issues I have with it:

1. This seems to mandate that all packages must be named by their
import path. My golang package (snapd) is not, intentionally so. I
don't want to change this.

2. Mandating a forge is going to be tricky for self-hosted stuff, or
people who release Go code as tarballs (it's rare, but it happens).
How do you deal with that?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org