Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Jakub Cajka" 

> I think one of the main responsibilities of Fedora packager is to work with 
> upstreams, help them
> mature and generally improve their projects.

Sure but expecting everything to be perfect and consistent before shipping 
anything just DOES NOT WORK. Reality is not black and white, it's messy with 
shades of gray

Even the C/C++ guys do not manage to ship a compat-free package ecosystem, and 
their projects had a few more decades than Go projects to mature, and rpm was 
pretty much built around C/C++ expectations.

Compat packages are a fact of life in any large software ecosystem, where you 
can't achieve perfect cohesion. Go is getting *very* large. It needs compat 
packages. That does not mean there will be hundreds of them because, creating 
packages is *work*, that does not need they will need maintaining for years, 
because the aim of each compat package is to get obsoleted as fast as possible, 
but there will always be some of them at any given time. We are not building a 
cathedral. Bazaars are messy when full of life.

> Do you have any evidence supporting this claim? From my point of view Jan and 
> other packages has
> been spending lot of time on on boarding packagers and working on tooling.

No one (and certainly not me) is accusing you or Jan not to spend a crazy 
amount of time and energy trying to make things work. But you did not achieve a 
satisfactory Go state in Fedora, read again what Owen wrote, I didn't put any 
word in his mouth, even though I could have written pretty much the same 
message a few months ago.

Again, no one (and certainly not me) disputes your level of dedication to the 
"cause". You just made some choices in the past (trying to avoid working within 
rpm via godep, refusing to include different states of the same Go code in the 
distro when major Go apps *disagree* on the correct state to include) that 
didn't work out in practice, with the hindsight of several years of Fedora Go 
packaging and the Go package state achieved in Fedora after those years.

It's time to adjust those choices a bit.

> From my perspective distribution will end up with crazy amount of bitrotting, 
> backport, constant
> attention requiring package crud..., IMHO basically same as now, apart of it 
> we will have few 1000s
> of Go based packages with compat names nobody cares about, instead of current 
> 500 or so packages.

Fedora has lots of Go packages no one cares about because they do not permit 
building the apps people *do* care about.

Building the apps people care about requires a more aggressive update policy, 
and accepting people will work in parallel on apps, that demand different code 
states, of common dependencies. And there are not so many of those, I count 18 
of them in our repo out of 476 Go packages, even though the work is not 
completely finished, and finalizing the build of complex tip apps is almost 
certain to increase the proportion a little. That's awfully *good* and *nice* 
given the "no Go API compatibility" scarecrows people like to invoke.

You won't *ever* attract a large pool of packagers, to work on a package 
baseline, that will eventually in some years permit building apps, but never 
this year, because upstream state is not conflict and compat-free yet.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Saturday, February 3, 2018 4:27:36 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> Hi Jakub
> 
> > I'm strongly against general unrestricted practice of compat packages as
> > proposed. If you need compat package you
> > need to work with usptreams on stabilizing the API/project, fork it, or
> > just use COPR as your projects(or its
> > dependencies) are not yet mature enough for Fedora.
> 
> I appreciate the sentiment, and I quite hate compat packages, but the
> restrictions you want did not work in practice for Fedora.
> 
> Making compat package creation hard does not result in faster fixing to use
> new code. What it actually does is to block the packaging of all the
> projects that need a more recent package state, because packagers that need
> the old state for one of their packages, will drag their feet on updating
> common dependencies till the code of their packages is fixed upstream, or
> they have the time to fix it themselves.
> 

I think one of the main responsibilities of Fedora packager is to work with 
upstreams, help them mature and generally improve their projects.

> And blocking packaging of new part means you do not get the new packagers
> that would join because they're interested in the new parts, and would
> contribute to the maintenance of common deps (not all of which need
> compat-ing), and would relieve part of the pressure on the existing
> packagers, that prevents them from working as much a they'd like to on
> updating their packages. It's a death spiral. It results in a massively
> obsolete Go package baselines, full of holes, because all the energy is
> poured in making existing stuff work, at the expense of onboarding new
> packages and packagers.
> 
> Regards,
> 
> --
> Nicolas Mailhot
> ___
> golang mailing list -- gol...@lists.fedoraproject.org
> To unsubscribe send an email to golang-le...@lists.fedoraproject.org
> 

Do you have any evidence supporting this claim? From my point of view Jan and 
other packages has been spending lot of time on on boarding packagers and 
working on tooling.
From my perspective distribution will end up with crazy amount of bitrotting, 
backport, constant attention requiring package crud..., IMHO basically same as 
now, apart of it we will have few 1000s of Go based packages with compat names 
nobody cares about, instead of current 500 or so packages.

JC
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Nicolas Mailhot"

> It's a bit of a Lego guideline, you assemble the spec blocs you need, and 
> ignore those you don't need. The
> example was chosen to include as many blocks as possible, with the 
> walkthrough explaining their respective
> functions. All the blocks are very short and succinct, don't search for long 
> runs of shell code like in the old
> guidelines

That being said if people feel it's useful I can add a barebones lib-only 
example with none of the optional blocks in. However, I fear that the more ways 
the page explains the same stuff, the more confusing and difficult to maintain 
it will be.

So, can people chime in to say whether they feel a supplementary example is 
useful of counter productive?

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka" 

Hi Jakub

> I'm strongly against general unrestricted practice of compat packages as 
> proposed. If you need compat package you
> need to work with usptreams on stabilizing the API/project, fork it, or just 
> use COPR as your projects(or its
> dependencies) are not yet mature enough for Fedora.

I appreciate the sentiment, and I quite hate compat packages, but the 
restrictions you want did not work in practice for Fedora.

Making compat package creation hard does not result in faster fixing to use new 
code. What it actually does is to block the packaging of all the projects that 
need a more recent package state, because packagers that need the old state for 
one of their packages, will drag their feet on updating common dependencies 
till the code of their packages is fixed upstream, or they have the time to fix 
it themselves.

And blocking packaging of new part means you do not get the new packagers that 
would join because they're interested in the new parts, and would contribute to 
the maintenance of common deps (not all of which need compat-ing), and would 
relieve part of the pressure on the existing packagers, that prevents them from 
working as much a they'd like to on updating their packages. It's a death 
spiral. It results in a massively obsolete Go package baselines, full of holes, 
because all the energy is poured in making existing stuff work, at the expense 
of onboarding new packages and packagers. 

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 4:24:52 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> >> Filling upstream holes is pretty much the definition of an
> >> integrator/distributor role. In Go they are particularly numerous and
> >> deep,
> >> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
> >> is astonishingly free of the problems that plague many Go projects,
> >> proving
> >> it *is* possible to do good release engineering in Go).
> 
> > ??
> 
> Just sending some kuddos Kubernetes-side. All the Kubernetes parts we had to
> work with so far were astonishingly free of API breakage and dependency
> cycles because Kubernetes people chose their deps carefully and do version
> their code states. It was a vacation from packaging other Go code.

Yes kubernetes is kind of better than most projects, but IMHO still uses lots 
of not so good practices, although getting bit better over the years.

> 
> > > Pain will not be eased in any significant way as you will still have to
> > > carefully evaluate every
> > > change so you don't break any dependent package.
> > 
> >> Pain *will* be eased in a significant way because we will have the tooling
> >> to
> >> detect breakage automatically, and because fixes once they're done will be
> >> packaged and shared.
> 
> > Your proposal doesn't provide this kind of tooling, if I'm not mistaken.
> 
> Not directly. It does provide the means to easily rev a spec to a new code
> state (version tag or commit), and it makes deps systematic (so Fedora
> tooling can accurately detect what is likely to be impacted by a change).
> Together, those open the way to use generic Fedora testing tools to automate
> build tests.

It makes it bit easier in some cases, but I wouldn't call it and revolution(as 
you are trying to sell it) rather than evolution(and omission of some currently 
mentioned parts), building on top of current practice.

> 
> >> If it's minor let's do it. I'm quite sure you will be pleasantly surprised
> >> by
> >> the level of pain those minor changes remove.
> 
> > ?? I'm in no way opposing your proposal per se.
> 
> Thank you
> 
> >> Fedora can detect the breakage which is the first step to fix it. It does
> >> not
> >> take an army of Go experts, just unbundled packages that can be rebuilt
> >> automatically to identify incompatibilities. Go experts are better
> >> employed
> >> working on fixes than doing rebuild checks that can be automated.
> 
> > Rebuilds could be automated, analysis of them not easily.
> 
> Sure, but at that point
> 1. you know if the new API is compatible or not with other packages
> 2. you've saved human packagers the time and effort to do the same rebuild
> check manually
> 
> So that's still a net win, no ?

No really, or not much, until you determine why the builds have failed(I can 
rebuild of whole distro each day, still I don't have enough spare time to 
determine and fix all FTBFS issues). You still have to have someone who would 
analyze those failures.

> 
> >> And nothing stops you for continuing to use those. Except, the golden test
> >> will always be to actually try to build the code, which is something that
> >> can be automated reliably once the dependencies exported to Fedora tools
> >> are
> >> themselves complete and reliable, and Go packages are exported as rpm
> >> packages, not hidden in private vendor trees.
> 
> > ?? I bit don't understand you. If you can, please help Jan to improve them.
> 
> What I meant is that:
> 1. there are many ways to try to divine if a code state will build, but the
> final arbiter is the Go compiler
> 2. to use Fedora-wide QA tools, code objects need to be exported as packages.
> Every missing test dependency, which is not packaged as a separate Go rpm,
> is something Fedora QA won't be able to run.

I think that I'm still missing your point. Of course you need to package all 
your dependencies.

> 
> >> When you do not unbundle you are still shipping the vendored code, you are
> >> still responsible for its bugs (security  or not). Compat packages do not
> &

Re: Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Colin Walters"

> I appreciate the work you're doing here, 

Thank you

> but I think the right path for golang (indeed for most other language 
> ecosystems) is to autogenerate
> specs.

You're, of course, are entitled to your opinion, but I do not share it :) I 
think rpm has been an incredibly successful technology. It is used today to 
package software no one could imagine when rpm was designed, in languages that 
didn't even exist at that time, and yet one can open spec files from last 
millenium and still recognize most of the parts.

And during all that time, people have been writing one spec generator after 
another, that all failed to get serious traction, and make any serious impact.

I think that the reason for rpm success on one hand, and the serial flops of 
spec generators on the other, it that spec generators are inherently 
inflexible. They do not deal well with special cases. They do not deal well 
with mixed projects that require combining several packaging patterns. They are 
very slow to adapt to rpm enhancements. It's so easy to generate yet another 
line of code there is no incentive to refactor what's generated to remove parts 
no longer needed, to strive for succinctness, efficiency and maintainability.

By and large generators generate very verbose low-quality spec code, which can 
of course, be manually adapted, but is not designed to be adapted by humans, 
making adaptations hard and inefficient (that seems to be a common pitfall of 
code generators of any sort). And, when packaging any large pool of software, 
you will hit many special cases, because software is written by humans, humans 
are messy, and expecting every software project to follow conventions with the 
level of strictness expected by spec generators is, IMHO illusory.

spec file syntax, warts and all, seems to hit the sweet spot between the wish 
to automate as much as possible, and the need to add a few lines of custom code 
whenever a project needs a little dose of custom handling.

I could, quite easily I think, implement something similar to autospec for 
Fedora Go packages. All it would take would be to define a variable name for 
each parameter of the autospec manifest, then call a giant macro that generates 
the rest of the spec file. I didn't do it not because I don't know how to, nor 
because it wouldn't work as is for many Go projects, but because it would 
suffer from the usual generator inflexibility, and inability to handle special 
cases, and projects that deviate slightly from the norm. I *chose* to split the 
Go automation in separate rpm macros, that can be combined by packagers as 
needed in different ways, that can be mixed with the other macros implemented 
by Fedora, that can be completed by custom spec code when there is a need (that 
is BTW a balance I didn't get quite right when I automated fonts packaging a 
few years ago, with hindsight I wish I had designed something a little more 
adaptable, with more flexibility).

Basically I feel that providing powerful spec verbs, is a more powerful model 
than expecting to capture every need in rigid autogenerated templates, no 
matter the amount of switches bolted on later, to manage all the deviations not 
envisioned when defining the original "simple" generation.

That being said rpm is not perfect and will need to continue to evolve to stay 
relevant. For example, with the horizontal compilation model of Go, the number 
of BuildRequires in specs is getting out of hand. So rpm and mock will probably 
need to evolve towards a model, where BuildRequires are not frozen till the end 
of %prep. That would allow to infer at least some BuildRequires from what's in 
the project archive, instead of requiring packagers to declare every single 
BuildRequires manually, because the build root is setup before the archive is 
even opened. However I see such things more like a natural evolution of the rpm 
model, and certainly nothing that requires dumping rpm, or "fixing" it with a 
generation layer.

> Also having all of the golang -devel packages in a single git repository
> with subdirectories so it's easy to have a single PR affect multiple packages
> at once. 

I can't really disagree there, that's how we work internally. It has proved 
invaluable every time we hit "damn, that's the twentieth spec that needs this 
exception, maybe it's time to fold it in common macro code to simplify those 
specs and avoid writing the exception another time, let's refactors all this in 
a single commit (or commit series)" 

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Colin Walters]

On Thu, Feb 1, 2018, at 10:24 AM, nicolas.mail...@laposte.net wrote:
> 
> Not directly. It does provide the means to easily rev a spec to a new 
> code state (version tag or commit), and it makes deps systematic (so 
> Fedora tooling can accurately detect what is likely to be impacted by a 
> change). Together, those open the way to use generic Fedora testing 
> tools to automate build tests.

I appreciate the work you're doing here, but I think the right path
for golang (indeed for most other language ecosystems) is to autogenerate
specs.  Like this:
https://github.com/clearlinux/autospec
plus language-specific plugins basically.  

Also having all of the golang -devel packages in a single git repository
with subdirectories so it's easy to have a single PR affect multiple packages
at once.  (Of course, this model crosses other things beyond golang).

Reference: https://github.com/coreos/coreos-overlay for the latter.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Przemek Klosowski]

On 02/01/2018 05:49 AM, Jakub Cajka wrote:

On contrary Fedora is trying to fill the hole that upstream Go projects dug them selves in to. 
IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even deeper levels of 
"Go (vendor) hell".


This unfortunately became a trend: "the old packaging is dead, long live 
the new packaging!" (not just go, but also pip/conda/npm/...). I think 
Fedora is doing a good job getting it under control, leading upstream 
projects to get their act together---and it demonstrated that this 
works, and actually makes upstream better.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka"

>> Filling upstream holes is pretty much the definition of an
>> integrator/distributor role. In Go they are particularly numerous and deep,
>> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
>> is astonishingly free of the problems that plague many Go projects, proving
>> it *is* possible to do good release engineering in Go).

> ??

Just sending some kuddos Kubernetes-side. All the Kubernetes parts we had to 
work with so far were astonishingly free of API breakage and dependency cycles 
because Kubernetes people chose their deps carefully and do version their code 
states. It was a vacation from packaging other Go code.

> > Pain will not be eased in any significant way as you will still have to
> > carefully evaluate every
> > change so you don't break any dependent package.
> 
>> Pain *will* be eased in a significant way because we will have the tooling to
>> detect breakage automatically, and because fixes once they're done will be
>> packaged and shared.

> Your proposal doesn't provide this kind of tooling, if I'm not mistaken.

Not directly. It does provide the means to easily rev a spec to a new code 
state (version tag or commit), and it makes deps systematic (so Fedora tooling 
can accurately detect what is likely to be impacted by a change). Together, 
those open the way to use generic Fedora testing tools to automate build tests.

>> If it's minor let's do it. I'm quite sure you will be pleasantly surprised by
>> the level of pain those minor changes remove.

> ?? I'm in no way opposing your proposal per se.

Thank you

>> Fedora can detect the breakage which is the first step to fix it. It does not
>> take an army of Go experts, just unbundled packages that can be rebuilt
>> automatically to identify incompatibilities. Go experts are better employed
>> working on fixes than doing rebuild checks that can be automated.

> Rebuilds could be automated, analysis of them not easily.

Sure, but at that point
1. you know if the new API is compatible or not with other packages
2. you've saved human packagers the time and effort to do the same rebuild 
check manually

So that's still a net win, no ?

>> And nothing stops you for continuing to use those. Except, the golden test
>> will always be to actually try to build the code, which is something that
>> can be automated reliably once the dependencies exported to Fedora tools are
>> themselves complete and reliable, and Go packages are exported as rpm
>> packages, not hidden in private vendor trees.

> ?? I bit don't understand you. If you can, please help Jan to improve them.

What I meant is that:
1. there are many ways to try to divine if a code state will build, but the 
final arbiter is the Go compiler
2. to use Fedora-wide QA tools, code objects need to be exported as packages. 
Every missing test dependency, which is not packaged as a separate Go rpm, is 
something Fedora QA won't be able to run.

>> When you do not unbundle you are still shipping the vendored code, you are
>> still responsible for its bugs (security  or not). Compat packages do not
>> augment the code surface you have to care about, on the contrary they offer
>> you the possibility to *reduce* the number of code states in the
>> distribution.

> Please don't put words in to my mouth. I haven't been talking about not 
> un-bundling, just compat
> packages.

Sorry if I misunderstood you:(

However, I do think compat packages are a natural effect of unbundling. One 
does need something to fill in during the time upstream or Fedora work on the 
effects of an API break. Other Fedora language ecosystems do not manage to 
avoid them. You can strive to limit compat packages to the minimum amount, and 
the minimum period, but you can not fully avoid them, once an ecosystem has 
grown enough it is too big to be perfectly coordinated all the time.

So making compat creation easy and fast, is required to manage a huge amount of 
interdependent packages, which are not too careful about API stability. Even if 
one has little love for compat packages as such.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Owen Taylor"

> I'm embarrassed to admit that before I sent my mail I carefully read over
> ... the old PackageDrafts/Go :-( My only excuse is that it was in my
> browser history.

NP, that gave you some context on where Fedora is today.

> Having actually read the relevant parts of "More Go Packaging", the
> explanation of compat packages and notification procedures does make the
> intended operation clearer,

Thank you. Do not hesitate to improve the wording in the wiki, or post 
suggestions here, if you have ideas on how to make it clearer, simpler or more 
efficient

> though the social and technical barrier to a
> packager new to Fedora will still be high if packaging their target package
> requires creating a compat package and fixing multiple other packages.

Unfortunately, packaging something in Go is still going to be harder than 
packaging software in another language in the near term :( I'd love to find 
more magic bullets.

> I still worry that Fedora is not big enough to move the status-quo in the
> Go world - to get the point where Go programs require github.com/foo/bar >=
> .2.3 and actually have been tested with a multiple versions in that range,
> not exactly the one vendored version shipped with the program.

That's a legitimate worry, yes. However given container and cloud people are 
massively adopting Go, that critical cloud software is now mostly written in 
Go, I don't think Fedora can afford to pass on Go and still stay relevant 
server-side. That's even more true for Fedora downstreams and Fedora's main 
sponsor.

So I guess it all boils down to strategic choices for Fedora and Red Hat: 
invest in Go packaging, even if it *is* painful, or pass, and lose relevance 
server-side in the near future. Red Hat certainly has the assets, with 
Openshift and Coreos, to influence the Go world in a Fedora-positive way. I 
don't know if it will choose, or manage, to leverage them. It can not fail to 
notice the many Go projects that propose their software as Ubuntu Docker images 
today. It can not fail to notice that Jakub and Jan, with all their qualities, 
are a tad overworked and not really sufficient to pull Fedora Go forward the 
required amount. The Go ecosystem has just grown too big. Lastly, I understand 
the temptation to let Fedora and RHEL slip in favor of product lines more 
profitable in the near term.

On my employer's side we *do* wish to stay relevant in a Cloud world. My Go 
contributions are an attempt to partner with the Fedora Centos and RHEL 
communities on that. If the partnership does not bear fruits, and 
Fedora/Centos/RHEL do not move in a direction we can use, we'll have to invest 
elsewhere. I'd regret it but this proposal and the spec dump that will follow 
are just too much work to do on one's free time. And employer time requires 
results.

> I haven't had time to read through the entire proposal, but it certainly
> looks like a major step forward!

Please finish reading it and propose all the fixes and comments and 
improvements you want! Your experience is appreciated.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 2:51:13 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> De: "Jakub Cajka"
> 
> Hi Jakub,
> 
> > It depends (as everything) on available manpower, if you are willing to own
> > your dependencies
> > you can package anything and everything debundled.
> 
> Sure, but available manpower depends on how high the bar you put for people
> wanting to join, and right now this bar is pretty high.
> 
> > On contrary, to this state you dislike(or seems to) Fedora got because of
> > effort of many to de
> > bundle Go based projects.
> 
> I'm *not* disparaging the past efforts of many. They did a lot of good.
> Unfortunately, the result still falls short of the mark. The baseline is not
> complete enough, the baseline is old and aging fast, would-be Go packagers
> like Owen are ready to give up when they start getting an idea of the Go
> state in Fedora.
> 
> It's time to invest in tooling and simplification so the few Go Fedora
> packagers there is, can do more with less, so casual Go packagers can
> contribute their little bit instead of being repulsed by complexity that can
> be avoided and factored out, so people already familiar with Fedora
> packaging can package the Go bits they need without needing a lengthy
> formation on Go specifics.
> 
> There *are* lots of people that would like to do Go packaging. Many Go
> software projects currently make headlines. That there are so few Go
> packagers in Fedora, at the time Go is in the limelight, is a pretty good
> indicator we are making it too hard.
> 
> > I don't see way how it makes it less painful(you have still the whole
> > distro issue). It makes specs
> > more abstract and streamlined when it works, but more opaque and hard to
> > read when its not(and you
> > have to debug some arcane srpm/lua macros).
> 
> It makes it less painful because there is no package-specific magic in each
> and every Go spec file.
> 
> You do not need to debug code in each spec because code that already just
> works for hundreds of  packages is unlikely to not work on the next spec
> file.
> 
> You do not need to comb each and every line of a Go spec file wondering if
> the line is cut and pasting of a common template (multiply by all past
> revisions of common templates) or if it is package-specific. The common
> parts are in common macros not spread right and left.
> 
> You do not need to worry wether common code is complete or not because
> factoring out common code removes the temptation to document 90% of what's
> needed, leaving others to fill in the missing 10%. 10% amounts to an
> astonishing level of work when you multiply it by the number of Go packagers
> and the number of Go packages we would ideally need.
> 
> Plus, when you do find a bug in common code, the fix is shared with the whole
> distribution, not hidden in a single spec file.
> 
> That's what successful factoring of common code earns you.
> 
> And given I have more than 470 Go spec files that just work with the proposed
> macros with no special package-specific adjustment needed, and most time I
> compare one of those to the Fedora equivalent I'm executing more unit tests
> and building more binaries, with drastically shorter spec code, I'm pretty
> sure the factoring out *is* successful.
> 
> >> This step is going to be painful I'm afraid, Fedora dug itself a
> >> deep hole, leaving it is going to be hard.
> 
> > On contrary Fedora is trying to fill the hole that upstream Go projects dug
> > them selves in to.
> 
> Filling upstream holes is pretty much the definition of an
> integrator/distributor role. In Go they are particularly numerous and deep,
> but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
> is astonishingly free of the problems that plague many Go projects, proving
> it *is* possible to do good release engineering in Go).

??

> 
> Fedora did dug itself a hole by trying the bundling shortcut around EL7 time.
> Where are the EL7 Go packages? Why are Fedora Atomic people not spearheading
> Go packaging in Fedora? Waiting for bundling to be solved upstream-side made
> Fedora worse not better.

I'm not sure that Fedora. Go ask them don't ask me or other regular Fedora 
maintainers. 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

De: "Jakub Cajka" 

Hi Jakub,

> It depends (as everything) on available manpower, if you are willing to own 
> your dependencies
> you can package anything and everything debundled.

Sure, but available manpower depends on how high the bar you put for people 
wanting to join, and right now this bar is pretty high.

> On contrary, to this state you dislike(or seems to) Fedora got because of 
> effort of many to de
> bundle Go based projects.

I'm *not* disparaging the past efforts of many. They did a lot of good. 
Unfortunately, the result still falls short of the mark. The baseline is not 
complete enough, the baseline is old and aging fast, would-be Go packagers like 
Owen are ready to give up when they start getting an idea of the Go state in 
Fedora.

It's time to invest in tooling and simplification so the few Go Fedora 
packagers there is, can do more with less, so casual Go packagers can 
contribute their little bit instead of being repulsed by complexity that can be 
avoided and factored out, so people already familiar with Fedora packaging can 
package the Go bits they need without needing a lengthy formation on Go 
specifics.

There *are* lots of people that would like to do Go packaging. Many Go software 
projects currently make headlines. That there are so few Go packagers in 
Fedora, at the time Go is in the limelight, is a pretty good indicator we are 
making it too hard.

> I don't see way how it makes it less painful(you have still the whole distro 
> issue). It makes specs 
> more abstract and streamlined when it works, but more opaque and hard to read 
> when its not(and you
> have to debug some arcane srpm/lua macros).

It makes it less painful because there is no package-specific magic in each and 
every Go spec file. 

You do not need to debug code in each spec because code that already just works 
for hundreds of  packages is unlikely to not work on the next spec file.

You do not need to comb each and every line of a Go spec file wondering if the 
line is cut and pasting of a common template (multiply by all past revisions of 
common templates) or if it is package-specific. The common parts are in common 
macros not spread right and left.

You do not need to worry wether common code is complete or not because 
factoring out common code removes the temptation to document 90% of what's 
needed, leaving others to fill in the missing 10%. 10% amounts to an 
astonishing level of work when you multiply it by the number of Go packagers 
and the number of Go packages we would ideally need.

Plus, when you do find a bug in common code, the fix is shared with the whole 
distribution, not hidden in a single spec file.

That's what successful factoring of common code earns you.

And given I have more than 470 Go spec files that just work with the proposed 
macros with no special package-specific adjustment needed, and most time I 
compare one of those to the Fedora equivalent I'm executing more unit tests and 
building more binaries, with drastically shorter spec code, I'm pretty sure the 
factoring out *is* successful.

>> This step is going to be painful I'm afraid, Fedora dug itself a
>> deep hole, leaving it is going to be hard.

> On contrary Fedora is trying to fill the hole that upstream Go projects dug 
> them selves in to.

Filling upstream holes is pretty much the definition of an 
integrator/distributor role. In Go they are particularly numerous and deep, but 
Fedora users do want their docker and kubernetes (and Kubernetes, BTW, is 
astonishingly free of the problems that plague many Go projects, proving it 
*is* possible to do good release engineering in Go).

Fedora did dug itself a hole by trying the bundling shortcut around EL7 time. 
Where are the EL7 Go packages? Why are Fedora Atomic people not spearheading Go 
packaging in Fedora? Waiting for bundling to be solved upstream-side made 
Fedora worse not better.

> IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even 
> deeper levels of "Go 
> vendor) hell".

I fully agree. What I meant to convey is that replicating upstream vendoring 
Fedora-side only makes the situation worse for users, for upstreams, and for 
Fedora.

> Pain will not be eased in any significant way as you will still have to 
> carefully evaluate every
> change so you don't break any dependent package.

Pain *will* be eased in a significant way because we will have the tooling to 
detect breakage automatically, and because fixes once they're done will be 
packaged and shared.

>> The guidelines and automation aim at making upgrading easy, and avoid one
>> package or packager blocking others

> This is not really and automation per se. It is minor re-factoring and spec 
> streamlining.
> It might or might not enable easier implementation of packaging automation.

If it's minor let's do it. I'm quite sure you will be pleasantly surprised by 
the level of pain those minor changes remove.

>> That would be expensive computing-side but a lot 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Owen Taylor]

Hi Nicolas,

I'm embarrassed to admit that before I sent my mail I carefully read over
... the old PackageDrafts/Go :-( My only excuse is that it was in my
browser history.

Having actually read the relevant parts of "More Go Packaging", the
explanation of compat packages and notification procedures does make the
intended operation clearer, though the social and technical barrier to a
packager new to Fedora will still be high if packaging their target package
requires creating a compat package and fixing multiple other packages. As
you say, once Fedora packaging catches up to the modern state, things will
be better, because the onus won't be on the new packager to do the  work of
updating old libraries.

I still worry that Fedora is not big enough to move the status-quo in the
Go world - to get the point where Go programs require github.com/foo/bar >=
1.2.3 and actually have been tested with a multiple versions in that range,
not exactly the one vendored version shipped with the program. Sp we might
end up with lots of Fedora-specific bugs that the upstream doesn't care
about. But if the Fedora Go community can get some size and momentum and
start pushing fixes upstream, that will certainly be a positive force for
reducing Go programs stuck on random old versions of dependencies!

I haven't had time to read through the entire proposal, but it certainly
looks like a major step forward!

Owen



On Thu, Feb 1, 2018 at 5:21 AM,  wrote:

>
>
> - Mail original -
> De: "Owen Taylor"
>
> Hi Owen,
>
> > Is there a guide for Fedora packagers about how to handle unbundling for
> > golang packages? The draft guidelines don't seem to go into any details.
>
> I don't think there is, nor that it is necessarily needed. The posted
> guidelines should be sufficient technically, there are no magic I know of I
> didn't document, the rest is just a lot of work (but see ↓)
>
> > I've looked at packaging a few golang packages unbundled, and have
> > immediately run into:
> >
> > A) lots of unpackaged dependencies
> > B) dependencies that are packaged in Fedora with different, often much
> > older versions
>
> Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't
> expect anyone to be able to package anything but the most trivial app in
> unbundled mode. Too many common parts are missing, when they are not
> missing they are too old, trying to update an existing Go package is an
> exercise in frustration (too much package-specific shell code, that is
> difficult to understand and does not really work with new code versions),
> and trying to update or add missing parts just reveals more breakage and
> work to be done.
>
> However, accepting to package complex Go apps in bundled mode is how
> Fedora got to this state in the first place. In plain speak, bundling
> (vendoring in Go linguo) just does not scale. You need an awful lot of
> manpower to audit the hundreds of other projects each app bundles, bundling
> removes all the package tooling that may help you to do so, and the result
> is not shared with any other package, or with other versions of the same
> package, so you get zero positive network effects. Worse, big bundled apps
> that do not actually try to work in unbundled mode, do not actually test
> the code they export (they are bundling, remember) so the result is toxic
> to small Go packages that try to work with them.
>
> So bundling parts is a direct road to bundling everything, bundling
> everything is a direct road to bundling everything blindly, bundling
> everything blindly is error-prone and dangerous (because upstreams are only
> human and do make lots of mistakes), and pretty much removes any value
> Fedora can add to end users, to other parts of Fedora that would like to
> integrate with Go software, or to the upstream projects themselves (no
> Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to
> stabilize parts when upstream is lost in tunnel effect mode and does not
> realize that it is wasting everyone's time starting with its own).
>
> Therefore, trying to get all this it a better state, requires the
> following steps IMHO:
>
> 1. completely refactor our Go packaging style it's less painful to update
> Go packages and they do not need a packager with deep knowledge of
> package-specific shell glue. That takes documentation, and factoring out
> common Go packaging functions in shared rpm code (macros and autodeps) →
> that's what I posted
>
> 2. using the new documentation and tooling to clean up years of Fedora
> technical debt, and create a new set of up-to-date Go packages that can
> serve as new baseline → I have hundreds of specs that I'm waiting for step
> 1 to complete to submit. They won't constitute a full baseline by
> themselves, they're not all 100% done, it's too much work to do alone but
> working with others requires the common macros and documentation to be
> merged and adopted. This step is going to be painful 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "nicolas mailhot" <nicolas.mail...@laposte.net>
> To: gol...@lists.fedoraproject.org
> Cc: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>, "Discussion of RPM packaging
> standards and practices for Fedora" <packag...@lists.fedoraproject.org>
> Sent: Thursday, February 1, 2018 11:21:59 AM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> 
> 
> - Mail original -
> De: "Owen Taylor"
> 
> Hi Owen,
> 
> > Is there a guide for Fedora packagers about how to handle unbundling for
> > golang packages? The draft guidelines don't seem to go into any details.
> 
> I don't think there is, nor that it is necessarily needed. The posted
> guidelines should be sufficient technically, there are no magic I know of I
> didn't document, the rest is just a lot of work (but see ↓)
> 
> > I've looked at packaging a few golang packages unbundled, and have
> > immediately run into:
> >
> > A) lots of unpackaged dependencies
> > B) dependencies that are packaged in Fedora with different, often much
> > older versions
> 
> Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't
> expect anyone to be able to package anything but the most trivial app in
> unbundled mode. Too many common parts are missing, when they are not missing
> they are too old, trying to update an existing Go package is an exercise in
> frustration (too much package-specific shell code, that is difficult to
> understand and does not really work with new code versions), and trying to
> update or add missing parts just reveals more breakage and work to be done.

It depends (as everything) on available manpower, if you are willing to own 
your dependencies you can package anything and everything debundled.

> 
> However, accepting to package complex Go apps in bundled mode is how Fedora
> got to this state in the first place. In plain speak, bundling (vendoring in
> Go linguo) just does not scale. You need an awful lot of manpower to audit
> the hundreds of other projects each app bundles, bundling removes all the
> package tooling that may help you to do so, and the result is not shared
> with any other package, or with other versions of the same package, so you
> get zero positive network effects. Worse, big bundled apps that do not
> actually try to work in unbundled mode, do not actually test the code they
> export (they are bundling, remember) so the result is toxic to small Go
> packages that try to work with them.

On contrary, to this state you dislike(or seems to) Fedora got because of 
effort of many to de-bundle Go based projects.

> 
> So bundling parts is a direct road to bundling everything, bundling
> everything is a direct road to bundling everything blindly, bundling
> everything blindly is error-prone and dangerous (because upstreams are only
> human and do make lots of mistakes), and pretty much removes any value
> Fedora can add to end users, to other parts of Fedora that would like to
> integrate with Go software, or to the upstream projects themselves (no
> Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to
> stabilize parts when upstream is lost in tunnel effect mode and does not
> realize that it is wasting everyone's time starting with its own).
> 
> Therefore, trying to get all this it a better state, requires the following
> steps IMHO:
> 
> 1. completely refactor our Go packaging style it's less painful to update Go
> packages and they do not need a packager with deep knowledge of
> package-specific shell glue. That takes documentation, and factoring out
> common Go packaging functions in shared rpm code (macros and autodeps) →
> that's what I posted

I don't see way how it makes it less painful(you have still the whole distro 
issue). It makes specs more abstract and streamlined when it works, but more 
opaque and hard to read when its not(and you have to debug some arcane srpm/lua 
macros).

> 
> 2. using the new documentation and tooling to clean up years of Fedora
> technical debt, and create a new set of up-to-date Go packages that can
> serve as new baseline → I have hundreds of specs that I'm waiting for step 1
> to complete to submit. They won't constitute a full baseline by themselves,
> they're not all 100% done, it's too much work to do alone but working with
> others requires the common macros and documentation to be merged and
> adopted. This step is going to be painful I'm afraid, Fedora dug itself a
> deep hole, leaving it is going to be hard.

On contrary Fedora is trying to fill the hole that upstream Go projects dug 
them selves in to. IM

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

- Mail original -
De: "Owen Taylor"

Hi Owen,

> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.

I don't think there is, nor that it is necessarily needed. The posted 
guidelines should be sufficient technically, there are no magic I know of I 
didn't document, the rest is just a lot of work (but see ↓)

> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
>
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much
> older versions

Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't 
expect anyone to be able to package anything but the most trivial app in 
unbundled mode. Too many common parts are missing, when they are not missing 
they are too old, trying to update an existing Go package is an exercise in 
frustration (too much package-specific shell code, that is difficult to 
understand and does not really work with new code versions), and trying to 
update or add missing parts just reveals more breakage and work to be done.

However, accepting to package complex Go apps in bundled mode is how Fedora got 
to this state in the first place. In plain speak, bundling (vendoring in Go 
linguo) just does not scale. You need an awful lot of manpower to audit the 
hundreds of other projects each app bundles, bundling removes all the package 
tooling that may help you to do so, and the result is not shared with any other 
package, or with other versions of the same package, so you get zero positive 
network effects. Worse, big bundled apps that do not actually try to work in 
unbundled mode, do not actually test the code they export (they are bundling, 
remember) so the result is toxic to small Go packages that try to work with 
them.

So bundling parts is a direct road to bundling everything, bundling everything 
is a direct road to bundling everything blindly, bundling everything blindly is 
error-prone and dangerous (because upstreams are only human and do make lots of 
mistakes), and pretty much removes any value Fedora can add to end users, to 
other parts of Fedora that would like to integrate with Go software, or to the 
upstream projects themselves (no Fedora QA, no stream of Fedora-originated 
fixes, no Fedora pressure to stabilize parts when upstream is lost in tunnel 
effect mode and does not realize that it is wasting everyone's time starting 
with its own).

Therefore, trying to get all this it a better state, requires the following 
steps IMHO:

1. completely refactor our Go packaging style it's less painful to update Go 
packages and they do not need a packager with deep knowledge of 
package-specific shell glue. That takes documentation, and factoring out common 
Go packaging functions in shared rpm code (macros and autodeps) → that's what I 
posted 

2. using the new documentation and tooling to clean up years of Fedora 
technical debt, and create a new set of up-to-date Go packages that can serve 
as new baseline → I have hundreds of specs that I'm waiting for step 1 to 
complete to submit. They won't constitute a full baseline by themselves, 
they're not all 100% done, it's too much work to do alone but working with 
others requires the common macros and documentation to be merged and adopted. 
This step is going to be painful I'm afraid, Fedora dug itself a deep hole, 
leaving it is going to be hard.

3. hopefully, the result will be streamlined enough it won't be too painful to 
keep up to date, having an up to date baseline will help attract new Go 
packagers like you, everyone will benefit and be happy. We had to package some 
ostree bits for example to create this baseline, and I'm pretty sure ostree 
people within Fedora would prefer to maintain those themselves, if the rest of 
the Fedora go universe didn't make it too hard  

> B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version,

The guidelines and automation aim at making upgrading easy, and avoid one 
package or packager blocking others

> and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.

I hope that normalizing Fedora Go packages means it will be possible to 
automate QA tests such as "try to rebuild all the packages that depend on 
golang(foo) every time you see the package providing golang(foo) changing, and 
report what broke"

That would be expensive computing-side but a lot less expensive and long than 
expecting each Go packager to do it himself with his own means. And that is 
certainly not overkill, given how lax Go projects are about maintaining API 
stability.

And then in case of breakage, revert or create a compat package. That's why 
there is a long chapter dedicated to compat package creation in the proposed 
guidelines.

Regards,

-- 

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Owen Taylor" <otay...@redhat.com>
> To: "Development discussions related to Fedora" 
> <devel@lists.fedoraproject.org>
> Cc: gol...@lists.fedoraproject.org, "Discussion of RPM packaging standards 
> and practices for Fedora"
> <packag...@lists.fedoraproject.org>
> Sent: Wednesday, January 31, 2018 6:50:21 PM
> Subject: Re:  [Fedora-packaging] Re: Proposed Fedora packaging 
> guideline: More Go packaging
> 
> Hi Nicolas,
> 
> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.
> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
> 
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much older
> versions
> 
> A) is a pretty known quantity for any type of package, but I found B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version, and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.
> 
> Some tutorial showing a practical example of packaging a golang package for
> the first time I think would be very helpful.
> 
> Owen
> 

Unfortunately this is how the situation stand thanks to the wide spread use of 
vendoring and generally no release and API stability in Go projects. You have 
to package all not packaged deps and work with other maintainers(which mostly 
means with Jan Chaloupka, cc'ed) to get other already packaged deps to commonly 
acceptable version/commit level, although it might not be even possible thanks 
to API breaks between commits. So you have to stick to bundling in some cases, 
if you really want to package that project.

JC

> 
> On Wed, Jan 31, 2018 at 5:30 AM, < nicolas.mail...@laposte.net > wrote:
> 
> 
> >De: "Neal Gompa"
> 
> > The only thing I see that might be missing is autogenerating
> > bundled(golang()) Provides when a vendor tree exists (with the
> > appropriate automatic filters on Requires).
> 
> I had though a little about doing it but first, as many Go elements,
> vendoring relies on conventions not standards. The nasty thing about
> conventions is that they are not applied 100% the same way by everyone,
> making automation a PITA. And second interactions with autodeps can be
> nasty: you can filter out provides, but do you filter out requires? What
> about all the junk code many projects ship as testing and examples and which
> is vendored with the rest?
> 
> I don't say it can't be done, or that it would be difficult to do once the
> rest is merged, but I'll live it to someone that absolutely want to ship a
> Go package with vendored parts.
> 
> Right now we sidestep the issue in our packages by rm -fr ing anything that
> looks like vendored code in %prep. Unbundling takes time but it has a
> positive cumulative effect: the more you unbundle the less you need to worry
> about in other packages with the same requirements. And unbundling reveals
> code/legal problems that it would take about as much work to solve by
> auditing the vendored code manually.
> 
> Regards,
> 
> --
> Nicolas mAilhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
> 
> ___
> golang mailing list -- gol...@lists.fedoraproject.org
> To unsubscribe send an email to golang-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Owen Taylor]

Hi Nicolas,

Is there a guide for Fedora packagers about how to handle unbundling for
golang packages? The draft guidelines don't seem to go into any details.
I've looked at packaging a few golang packages unbundled, and have
immediately run into:

 A) lots of unpackaged dependencies
 B) dependencies that are packaged in Fedora with different, often much
older versions

A) is a pretty known quantity for any type of package, but I found B) more
intimidating. It seems like to package the new package, I have to get the
maintainer of the library to upgrade the version, and someone needs to
rebuild everything that depends on the rebuilt library and test that the
rebuilt packages work.

Some tutorial showing a practical example of packaging a golang package for
the first time I think would be very helpful.

Owen


On Wed, Jan 31, 2018 at 5:30 AM,  wrote:

> >De: "Neal Gompa"
>
> > The only thing I see that might be missing is autogenerating
> > bundled(golang()) Provides when a vendor tree exists (with the
> > appropriate automatic filters on Requires).
>
> I had though a little about doing it but first, as many Go elements,
> vendoring relies on conventions not standards. The nasty thing about
> conventions is that they are not applied 100% the same way by everyone,
> making automation a PITA. And second interactions with autodeps can be
> nasty: you can filter out provides, but do you filter out requires? What
> about all the junk code many projects ship as testing and examples and
> which is vendored with the rest?
>
> I don't say it can't be done, or that it would be difficult to do once the
> rest is merged, but I'll live it to someone that absolutely want to ship a
> Go package with vendored parts.
>
> Right now we sidestep the issue in our packages by rm -fr ing anything
> that looks like vendored code in %prep. Unbundling takes time but it has a
> positive cumulative effect: the more you unbundle the less you need to
> worry about in other packages with the same requirements. And unbundling
> reveals code/legal problems that it would take about as much work to solve
> by auditing the vendored code manually.
>
> Regards,
>
> --
> Nicolas mAilhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[nicolas . mailhot]

>De: "Neal Gompa" 

> The only thing I see that might be missing is autogenerating
> bundled(golang()) Provides when a vendor tree exists (with the
> appropriate automatic filters on Requires).

I had though a little about doing it but first, as many Go elements, vendoring 
relies on conventions not standards. The nasty thing about conventions is that 
they are not applied 100% the same way by everyone, making automation a PITA. 
And second interactions with autodeps can be nasty: you can filter out 
provides, but do you filter out requires? What about all the junk code many 
projects ship as testing and examples and which is vendored with the rest?

I don't say it can't be done, or that it would be difficult to do once the rest 
is merged, but I'll live it to someone that absolutely want to ship a Go 
package with vendored parts.

Right now we sidestep the issue in our packages by rm -fr ing anything that 
looks like vendored code in %prep. Unbundling takes time but it has a positive 
cumulative effect: the more you unbundle the less you need to worry about in 
other packages with the same requirements. And unbundling reveals code/legal 
problems that it would take about as much work to solve by auditing the 
vendored code manually.

Regards,

-- 
Nicolas mAilhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Jakub Cajka]

- Original Message -
> From: "Marcin Dulak" <marcin.du...@gmail.com>
> To: "Discussion of RPM packaging standards and practices for Fedora" 
> <packag...@lists.fedoraproject.org>
> Cc: gol...@lists.fedoraproject.org, "Development discussions related to 
> Fedora" <devel@lists.fedoraproject.org>
> Sent: Monday, January 22, 2018 4:04:19 PM
> Subject: Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More 
> Go packaging
> 
> 
> 
> On Mon, Jan 22, 2018 at 2:45 PM, Neal Gompa < ngomp...@gmail.com > wrote:
> 
> 
> On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
> < dridi.boukelmo...@gmail.com > wrote:
> >> I really do like this. There are only two issues I have with it:
> >> 
> >> 1. This seems to mandate that all packages must be named by their
> >> import path. My golang package (snapd) is not, intentionally so. I
> >> don't want to change this.
> >> 
> >> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> >> people who release Go code as tarballs (it's rare, but it happens).
> >> How do you deal with that?
> > 
> > By not using the macros for packages not fitting the model?
> > 
> 
> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.
> 
> > I think this is very helpful especially when it's the common practice,
> > and I certainly won't blame anyone doing proper releases and not
> > just a git tag with github releases notes ;)
> > 
> > Regarding naming, I think python packages must be prefixed with
> > python[23]- and can Provides: the upstream project name.
> 
> I'm not sure if this matters in this discussion but an example Python3 part
> of a spec file https://fedoraproject.org/wiki/Packaging:Python
> to accommodate also EPEL (which on CentOS7 prefixes Python3 packages with
> python34 and not python3) would look like:
> 
> %package -n python%{python3_pkgversion}-%{pname}
> %{?python_provide:%python_provide python%{python3_pkgversion}-%{pname}}
> 
> Macin
> 

Hopefully something like this will never happen as generally I'm strongly 
against shipping multiple versions(of one implementation) of Go concurrently.

JC

> 
> On the
> > other hand we have packages like docker that are clearly named
> > after upstream's name, so I don't think that would be a problem for
> > snapd. (and maybe an exception needs to be granted?)
> > 
> 
> This rule only applies to Python packages that have modules that are
> designed to be imported by other Python code. Otherwise, this is not
> necessary.
> 
> 
> 
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
> 
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging

[Marcin Dulak]

On Mon, Jan 22, 2018 at 2:45 PM, Neal Gompa  wrote:

> On Mon, Jan 22, 2018 at 8:33 AM, Dridi Boukelmoune
>  wrote:
> >> I really do like this. There are only two issues I have with it:
> >>
> >> 1. This seems to mandate that all packages must be named by their
> >> import path. My golang package (snapd) is not, intentionally so. I
> >> don't want to change this.
> >>
> >> 2. Mandating a forge is going to be tricky for self-hosted stuff, or
> >> people who release Go code as tarballs (it's rare, but it happens).
> >> How do you deal with that?
> >
> > By not using the macros for packages not fitting the model?
> >
>
> The issue is that the new Go macros are tightly wound into the forge
> macros. I just want to be sure that we can leverage things like the
> dependency generators without all the other stuff.
>
> > I think this is very helpful especially when it's the common practice,
> > and I certainly won't blame anyone doing proper releases and not
> > just a git tag with github releases notes ;)
> >
> > Regarding naming, I think python packages must be prefixed with
> > python[23]- and can Provides: the upstream project name.


I'm not sure if this matters in this discussion but an example Python3 part
of a spec file https://fedoraproject.org/wiki/Packaging:Python
to accommodate also EPEL (which on CentOS7 prefixes Python3 packages with
python34 and not python3) would look like:

%package -n python%{python3_pkgversion}-%{pname}
%{?python_provide:%python_provide python%{python3_pkgversion}-%{pname}}

Macin


> On the
> > other hand we have packages like docker that are clearly named
> > after upstream's name, so I don't think that would be a problem for
> > snapd. (and maybe an exception needs to be granted?)
> >
>
> This rule only applies to Python packages that have modules that are
> designed to be imported by other Python code. Otherwise, this is not
> necessary.
>
>
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> packaging mailing list -- packag...@lists.fedoraproject.org
> To unsubscribe send an email to packaging-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org