Re: introducing curl-minimal and libcurl-minimal RPM packages

[Neal Gompa]

On Wed, Mar 16, 2016 at 5:44 PM, Jason L Tibbitts III  wrote:
>> "JLT" == Jason L Tibbitts  writes:
>
> JLT> Using a different soname and patching python-pycurl to use the
> JLT> maximal module if present and the minimal module otherwise would be
> JLT> another possibility, assuming it's even doable.
>
> Or, even easier, having python-curl-minimal which requires and is linked
> against a curl-minimal package which has a different soname, and then
> dnf can just try to import the big one before importing the small one.
> Boolean dependencies in the dnf package should take care of the rest.
>

We can't do that at all right now, because we still have stuff in the
build and compose toolchain that use yum and die horribly when we use
it. Unless something is done about mash, we are effectively locked out
most of the most useful cases to use rich dependencies.



-- 
真実はいつも一つ！/ Always, there's only one truth!
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Dennis Gilmore]

On Thursday, March 17, 2016 07:12:01 AM Kamil Dudka wrote:
> On Wednesday, March 16, 2016 17:07:39 Dennis Gilmore wrote:
> > On Wednesday, March 16, 2016 09:52:36 PM Kamil Dudka wrote:
> > > On Wednesday, March 16, 2016 15:48:54 Dennis Gilmore wrote:
> > > > On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> > > > > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > > > > Kamil Dudka (kdu...@redhat.com) said:
> > > > > > > Are you reading it from the specfile?
> > > > > > > 
> > > > > > > It is just an implementation detail of the packaging (the
> > > > > > > RemovePathPostfixes feature of rpm).  The string you mentioned
> > > > > > > neither
> > > > > > > appears in the SONAME, nor in any file installed by the RPMs in
> > > > > > > question.
> > > > > > 
> > > > > > ... which means if the SONAME is the same, you either are dealing
> > > > > > with
> > > > > 
> > > > > > Conflicts:
> > > > > Exactly.  libcurl conflicts with libcurl-minimal, which means that
> > > > > exactly
> > > > > one of them will be installed on any Fedora system at a time.  On a
> > > > > regular
> > > > > system (server, desktop, etc.) it will always be libcurl.
> > > > > 
> > > > > On the other hand, if you need to create a minimal installation of
> > > > > Fedora
> > > > > (e.g. a base image for Docker), you will pick libcurl-minimal
> > > > > instead
> > > > > of
> > > > > libcurl, to make the set of installed packages really minimal.
> > > > > 
> > > > > > , or relying on the behavior of ldconfig to figure out which
> > > > > > library you happen to get at a particular time.
> > > > > 
> > > > > ldconfig will always pick /usr/lib64/libcurl.so.4 because there can
> > > > > be
> > > > > only
> > > > > one file with that name installed at a time.
> > > > 
> > > > And how do you transition when you need to add the full featured
> > > > version?
> > > 
> > > # dnf install --allowerasing curl
> > 
> > And you are going to be able to do that in the tools taht build layered
> > images?
> 
> Sure.  I see no reason why it would not work (or cause any problems).
> 
> Kamil

You have more faith than me then. I would want to see it working first. 

Dennis

signature.asc
Description: This is a digitally signed message part.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Ville Skyttä]

On Wed, Mar 16, 2016 at 6:36 PM, Kamil Dudka  wrote:
> The curl and libcurl packages, which are both required by dnf,

Hm, does dnf really require curl? On my F-23 box:

$ rpm -e --test curl
error: Failed dependencies:
curl is needed by (installed) rpmdevtools-8.6-2.fc23.noarch
curl is needed by (installed) pyrpkg-1.36-1.fc23.noarch
curl is needed by (installed) dracut-live-043-63.git20151211.fc23.x86_64
curl is needed by (installed) libreport-plugin-kerneloops-2.6.4-1.fc23.x86_64
curl is needed by (installed) abrt-addon-kerneloops-2.8.0-3.fc23.x86_64
curl is needed by (installed) abrt-addon-xorg-2.8.0-3.fc23.x86_64
curl is needed by (installed) rpm-4.13.0-0.rc1.12.fc23.x86_64
curl is needed by (installed) fedora-packager-0.5.10.7-1.fc23.noarch

Ah, rpm requires it (through %__urlhelpercmd). So I suppose there's
the use case for curl-minimal I was wondering about.

> 
> http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-libcurl-minimal

libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
because it stuffs "minimal" into the version part, but the minimalism
here is not really related to the version. Did you consider e.g.
libcurl-minimal.so.x.y.z instead?
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Chris Adams]

Once upon a time, Paul Howarth  said:
> The installed soname is exactly the same as the regular libcurl
> version, making them interchangeable. The .minimal suffix is removed by
> rpm at packaging time using RemovePathPostfixes:

That seems like a terrible idea, to have to shared libraries in the
distribution with the same soname but different functionality.  Seems
like it would be much better to have a different soname, and have rpm
just use the minimalized version or change from linking against it to
dlopen()ing it (and being able to choose the minimal or regular).

-- 
Chris Adams 
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday 17 March 2016 13:21:42 Przemek Klosowski wrote:
> On 03/17/2016 12:54 PM, Kamil Dudka wrote:
> > I am also open to discuss other solution to the above problem.  The other
> > proposals I have captured in this thread are:
> > 
> > - use dlopen() -- already proven wrong (see RHBZ and upstream ML)
> 
> I looked and didn't see any discussion of dlopen() in 1305701; are you
> talking about a different RHBZ case?
> Could you summarize the argument against something like
> 
> libcurl_version=REGULAR;
> if(!(lp=dlopen("/usr/lib64/libcurl.so",RTLD_NOW))  {
>   lp=dlopen("/usr/lib64/libcurl-minimal.so",RTLD_NOW);
>   libcurl_version=MINIMAL;
> }
> if (!lp) abort("Can't load libcurl because ",dlerror());

I was (by mistake) speaking about loading libcurl's run-time dependencies
by dlopen(), which is implemented for OpenLDAP in RHEL-5.  It used to cause 
problems and was removed from upstream curl long time ago.

Nevertheless, most of the reasons are valid for loading libcurl itself, too:

- Your example would require libcurl-devel to be installed because it provides
  the /usr/lib64/libcurl.so symlink.  This would be yet another *run-time*
  dependency of the package you patched.  See the following bug for example:

https://bugzilla.redhat.com/215928

- This approach is not compatible with the dependency scanner of rmp-build.

- You would need to patch this way every single package to be able to load
  libcurl-minimal.  It is very likely that maintainers of upstream projects
  would reject such Fedora-specific patches.

- Loading libcurl by dlopen() changes the order of global initialization
  of system libraries (such as OpenSSL, NSS, and OpenLDAP), which itself
  causes bugs from time to time.

There was a similar proposal for libcurl to load TLS libraries by dlopen().  
You can read the response of the upstream curl maintainer:

https://lists.mayfirst.org/pipermail/vtls/2015-February/20.html

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 17:20:38 Przemek Klosowski wrote:
> On 03/16/2016 04:39 PM, Kamil Dudka wrote:
> > ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> > only one file with that name installed at a time.
> 
> OK, but how do you determine which one you have on a particular system
> someone just handed to you?

libcurl supports run-time introspection of its features where it makes sense:

https://curl.haxx.se/libcurl/c/curl_version_info.html

But in most cases, it should be sufficient to handle error codes that are 
returned out of libcurl API in case a requested feature is not supported, 
usually CURLE_NOT_BUILT_IN:

https://curl.haxx.se/libcurl/c/libcurl-errors.html#CURLENOTBUILTIN

... which defensively written code should do anyway.

Kamil

> It seems to me that you are using the 'alternatives' pattern without
> actually having the 'alternatives' infrastructure for it.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Dennis Gilmore]

On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > Kamil Dudka (kdu...@redhat.com) said:
> > > Are you reading it from the specfile?
> > > 
> > > It is just an implementation detail of the packaging (the
> > > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > > appears in the SONAME, nor in any file installed by the RPMs in
> > > question.
> > 
> > ... which means if the SONAME is the same, you either are dealing with
> 
> > Conflicts:
> Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly
> one of them will be installed on any Fedora system at a time.  On a regular
> system (server, desktop, etc.) it will always be libcurl.
> 
> On the other hand, if you need to create a minimal installation of Fedora
> (e.g. a base image for Docker), you will pick libcurl-minimal instead of
> libcurl, to make the set of installed packages really minimal.
> 
> > , or relying on the behavior of ldconfig to figure out which
> > library you happen to get at a particular time.
> 
> ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be only
> one file with that name installed at a time.

And how do you transition when you need to add the full featured version?

Dennis

signature.asc
Description: This is a digitally signed message part.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Przemek Klosowski]

On 03/16/2016 04:39 PM, Kamil Dudka wrote:
ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be 
only one file with that name installed at a time.
OK, but how do you determine which one you have on a particular system 
someone just handed to you?


It seems to me that you are using the 'alternatives' pattern without 
actually having the 'alternatives' infrastructure for it.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Dennis Gilmore]

On Wednesday, March 16, 2016 09:52:36 PM Kamil Dudka wrote:
> On Wednesday, March 16, 2016 15:48:54 Dennis Gilmore wrote:
> > On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> > > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > > Kamil Dudka (kdu...@redhat.com) said:
> > > > > Are you reading it from the specfile?
> > > > > 
> > > > > It is just an implementation detail of the packaging (the
> > > > > RemovePathPostfixes feature of rpm).  The string you mentioned
> > > > > neither
> > > > > appears in the SONAME, nor in any file installed by the RPMs in
> > > > > question.
> > > > 
> > > > ... which means if the SONAME is the same, you either are dealing with
> > > 
> > > > Conflicts:
> > > Exactly.  libcurl conflicts with libcurl-minimal, which means that
> > > exactly
> > > one of them will be installed on any Fedora system at a time.  On a
> > > regular
> > > system (server, desktop, etc.) it will always be libcurl.
> > > 
> > > On the other hand, if you need to create a minimal installation of
> > > Fedora
> > > (e.g. a base image for Docker), you will pick libcurl-minimal instead of
> > > libcurl, to make the set of installed packages really minimal.
> > > 
> > > > , or relying on the behavior of ldconfig to figure out which
> > > > library you happen to get at a particular time.
> > > 
> > > ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> > > only
> > > one file with that name installed at a time.
> > 
> > And how do you transition when you need to add the full featured version?
> 
> # dnf install --allowerasing curl

And you are going to be able to do that in the tools taht build layered 
images? 

Dennis

signature.asc
Description: This is a digitally signed message part.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Alexander Bokovoy]

On Wed, 16 Mar 2016, Dennis Gilmore wrote:

On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:

On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> Kamil Dudka (kdu...@redhat.com) said:
> > Are you reading it from the specfile?
> >
> > It is just an implementation detail of the packaging (the
> > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > appears in the SONAME, nor in any file installed by the RPMs in
> > question.
>
> ... which means if the SONAME is the same, you either are dealing with

> Conflicts:
Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly
one of them will be installed on any Fedora system at a time.  On a regular
system (server, desktop, etc.) it will always be libcurl.

On the other hand, if you need to create a minimal installation of Fedora
(e.g. a base image for Docker), you will pick libcurl-minimal instead of
libcurl, to make the set of installed packages really minimal.

> , or relying on the behavior of ldconfig to figure out which
> library you happen to get at a particular time.

ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be only
one file with that name installed at a time.


And how do you transition when you need to add the full featured version?

Just a note on top of what Dennis asked: if curl minimal would lack
XML-RPC and GSSAPI support, FreeIPA client enrollment will be broken
completely. There are many uses of that within kickstart or cloud-init
files.

--
/ Alexander Bokovoy
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 17:07:39 Dennis Gilmore wrote:
> On Wednesday, March 16, 2016 09:52:36 PM Kamil Dudka wrote:
> > On Wednesday, March 16, 2016 15:48:54 Dennis Gilmore wrote:
> > > On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> > > > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > > > Kamil Dudka (kdu...@redhat.com) said:
> > > > > > Are you reading it from the specfile?
> > > > > > 
> > > > > > It is just an implementation detail of the packaging (the
> > > > > > RemovePathPostfixes feature of rpm).  The string you mentioned
> > > > > > neither
> > > > > > appears in the SONAME, nor in any file installed by the RPMs in
> > > > > > question.
> > > > > 
> > > > > ... which means if the SONAME is the same, you either are dealing
> > > > > with
> > > > 
> > > > > Conflicts:
> > > > Exactly.  libcurl conflicts with libcurl-minimal, which means that
> > > > exactly
> > > > one of them will be installed on any Fedora system at a time.  On a
> > > > regular
> > > > system (server, desktop, etc.) it will always be libcurl.
> > > > 
> > > > On the other hand, if you need to create a minimal installation of
> > > > Fedora
> > > > (e.g. a base image for Docker), you will pick libcurl-minimal instead
> > > > of
> > > > libcurl, to make the set of installed packages really minimal.
> > > > 
> > > > > , or relying on the behavior of ldconfig to figure out which
> > > > > library you happen to get at a particular time.
> > > > 
> > > > ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> > > > only
> > > > one file with that name installed at a time.
> > > 
> > > And how do you transition when you need to add the full featured
> > > version?
> > 
> > # dnf install --allowerasing curl
> 
> And you are going to be able to do that in the tools taht build layered
> images?

Sure.  I see no reason why it would not work (or cause any problems).

Kamil

> Dennis
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday 17 March 2016 11:55:31 Bill Nottingham wrote:
> Kamil Dudka (kdu...@redhat.com) said:
> > > If you care about a consistent developer, user, and debugging experience
> > > regardless of mechanism of delivery, you wouldn't do this in the first
> > > place, or you'd change the global curl package. Either the features are
> > > important, or they aren't.
> > 
> > Are you implying that curl maintainers know better than users which
> > features are important for the users themselves?
> 
> ... well, that's sort of what turning off features for this case implies.
> 
> > > If you care about minimizing size overall (neglecting the fact that
> > > cutting out kerberos and to a lesser extent ldap dependencies don't
> > > really
> > > save you anything due to their system library use), then might as well
> > > just
> > > start -Os-ing random packages and throwing in busybox.
> > 
> > Micro-optimization is way to hell.  The system needs to optimized by
> > design.
> EXACTLY!  Maybe I'm not being clear in what I'm trying to say, but this is
> the point - it needs to be designed.
> 
> 1) Start with the design of who are we targeting - what jobs are then trying
> to do?
> 
> 2) Then, what do we produce? (Atomic, Server, Cloud, Workstation)
> 
> 3) Do they have different needs? Do we build components with different
> features for each? (This solution implies 'yes', but I don't recall seeing
> that stated before)
> 
> 4) If so, what are the differentiating features that require different
> components with different feature sets? What's the job that someone is
> trying to do that requires this? (The impliciation is 'download the base
> image somewhat faster', but that's still very vague.)
> 
> 5) Then, pick the targeted points and propose implementation of individual
> features that serve the larger goal.

The motivation behind this effort is pretty straightforward.  I have been 
maintaining curl in Fedora/RHEL since 2009.  From time to time users ask me
to reduce the number of run-time dependencies of (lib)curl.

At the same time, users are continuously requesting to add new features to 
Fedora (lib)curl and this introduces additional run-time dependencies.  The 
last request, which actually triggered this effort, was adding the support
for libpsl, which itself requires libicu and libstdc++:

https://bugzilla.redhat.com/1305701#c1

I wanted to provide a solution to the continuously growing list of (lib)curl 
run-time dependencies.  Having two builds of the same source code that just 
differ in their build-time configuration sounded like a good trade-off from 
the perspective of a maintainer:

- one suitable for the users that request adding new features

- one suitable for the users that request reducing the run-time dependencies

I am open to discuss what features should be included in (lib)curl-minimal
and I already refined my draft base on the discussion in this thread.

I am also open to discuss other solution to the above problem.  The other 
proposals I have captured in this thread are:

- use dlopen() -- already proven wrong (see RHBZ and upstream ML)

- use static linking -- sub-optimal due to high security advisory rate

- multiple libcurl DSOs on a system -- problematic (see RHBZ and upstream ML)

- tweak dnf to use libcurl-minimal if installed in parallel to libcurl -- not
  a generic enough solution for me to consider (as it would work for dnf only)

> It's possible that, just poking my head up, I missed all the 1-4 discussion
> that this is tied to. But given that the following has all popped up in
> the past few months:
> 
> - "Minimizing the fedora docker base image footprint" (by yanking dnf et.al.
> into a seprate container, making size of it much more irrelevant) - "DNF
> into C initiative started" (enabling a much larger depythoning that doesn't
> require differing library buidls)
> - "F24 Self Contained Change: System Python"
> 
> ... the fact that we now have four separate implementation details, all of
> which could make different parts of the others irrelevant, makes me wonder
> about the design behind it.

So what exactly are you proposing?

Kamil

> Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday 17 March 2016 08:57:19 Alexander Bokovoy wrote:
> On Wed, 16 Mar 2016, Dennis Gilmore wrote:
> >On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> >> On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> >> > Kamil Dudka (kdu...@redhat.com) said:
> >> > > Are you reading it from the specfile?
> >> > > 
> >> > > It is just an implementation detail of the packaging (the
> >> > > RemovePathPostfixes feature of rpm).  The string you mentioned
> >> > > neither
> >> > > appears in the SONAME, nor in any file installed by the RPMs in
> >> > > question.
> >> > 
> >> > ... which means if the SONAME is the same, you either are dealing with
> >> 
> >> > Conflicts:
> >> Exactly.  libcurl conflicts with libcurl-minimal, which means that
> >> exactly
> >> one of them will be installed on any Fedora system at a time.  On a
> >> regular
> >> system (server, desktop, etc.) it will always be libcurl.
> >> 
> >> On the other hand, if you need to create a minimal installation of Fedora
> >> (e.g. a base image for Docker), you will pick libcurl-minimal instead of
> >> libcurl, to make the set of installed packages really minimal.
> >> 
> >> > , or relying on the behavior of ldconfig to figure out which
> >> > library you happen to get at a particular time.
> >> 
> >> ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> >> only
> >> one file with that name installed at a time.
> >
> >And how do you transition when you need to add the full featured version?
> 
> Just a note on top of what Dennis asked: if curl minimal would lack
> XML-RPC and GSSAPI support, FreeIPA client enrollment will be broken
> completely. There are many uses of that within kickstart or cloud-init
> files.

I am not against keeping the GSSAPI support in libcurl-minimal.  From the 
discussion, it sounds like the right thing to do.

But what do you mean by XML-RPC?

I am not aware of any such feature in (lib)curl itself.  XML-RPC is usually 
implemented in higher-level libraries.  If they use the http(s) implementation
of libcurl, they will work on top of libcurl-minimal as well.

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Przemek Klosowski]

On 03/17/2016 12:54 PM, Kamil Dudka wrote:

I am also open to discuss other solution to the above problem.  The other
proposals I have captured in this thread are:

- use dlopen() -- already proven wrong (see RHBZ and upstream ML)


I looked and didn't see any discussion of dlopen() in 1305701; are you 
talking about a different RHBZ case?

Could you summarize the argument against something like

libcurl_version=REGULAR;
if(!(lp=dlopen("/usr/lib64/libcurl.so",RTLD_NOW))  {
 lp=dlopen("/usr/lib64/libcurl-minimal.so",RTLD_NOW);
 libcurl_version=MINIMAL;
}
if (!lp) abort("Can't load libcurl because ",dlerror());
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday 17 March 2016 08:14:42 Dennis Gilmore wrote:
> On Thursday, March 17, 2016 07:12:01 AM Kamil Dudka wrote:
> > On Wednesday, March 16, 2016 17:07:39 Dennis Gilmore wrote:
> > > On Wednesday, March 16, 2016 09:52:36 PM Kamil Dudka wrote:
> > > > On Wednesday, March 16, 2016 15:48:54 Dennis Gilmore wrote:
> > > > > On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> > > > > > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > > > > > Kamil Dudka (kdu...@redhat.com) said:
> > > > > > > > Are you reading it from the specfile?
> > > > > > > > 
> > > > > > > > It is just an implementation detail of the packaging (the
> > > > > > > > RemovePathPostfixes feature of rpm).  The string you mentioned
> > > > > > > > neither
> > > > > > > > appears in the SONAME, nor in any file installed by the RPMs
> > > > > > > > in
> > > > > > > > question.
> > > > > > > 
> > > > > > > ... which means if the SONAME is the same, you either are
> > > > > > > dealing
> > > > > > > with
> > > > > > 
> > > > > > > Conflicts:
> > > > > > Exactly.  libcurl conflicts with libcurl-minimal, which means that
> > > > > > exactly
> > > > > > one of them will be installed on any Fedora system at a time.  On
> > > > > > a
> > > > > > regular
> > > > > > system (server, desktop, etc.) it will always be libcurl.
> > > > > > 
> > > > > > On the other hand, if you need to create a minimal installation of
> > > > > > Fedora
> > > > > > (e.g. a base image for Docker), you will pick libcurl-minimal
> > > > > > instead
> > > > > > of
> > > > > > libcurl, to make the set of installed packages really minimal.
> > > > > > 
> > > > > > > , or relying on the behavior of ldconfig to figure out which
> > > > > > > library you happen to get at a particular time.
> > > > > > 
> > > > > > ldconfig will always pick /usr/lib64/libcurl.so.4 because there
> > > > > > can
> > > > > > be
> > > > > > only
> > > > > > one file with that name installed at a time.
> > > > > 
> > > > > And how do you transition when you need to add the full featured
> > > > > version?
> > > > 
> > > > # dnf install --allowerasing curl
> > > 
> > > And you are going to be able to do that in the tools taht build layered
> > > images?
> > 
> > Sure.  I see no reason why it would not work (or cause any problems).
> > 
> > Kamil
> 
> You have more faith than me then. I would want to see it working first.
> 
> Dennis

If you are familiar with those tools, it should be easy to try.  The testing 
packages are available in the following Copr repository:

https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/

Feel free to report any obstacles you encounter.

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Nico Kadel-Garcia]

On Fri, Mar 18, 2016 at 4:29 AM, Petr Pisar  wrote:

> I think the solution is have more packages delivering the same-named
> shared library file with the same soname. Each of the packages
> conflicting each other. Then the non-minimal package would provide RPM
> symbols declaring compiled-in features like "Provides: libcurl(LDAP)"
> and then each application package requiring specific feature would
> explicitly run-require it ("Requied: libcurl(LDAP)"), besides
> automatically genererated dependency on the soname.

I suspect the better solution is to stop burning effort rewriting the
structure of core utilities for something that does not actually
improve the utility, but only helps with edge cases, in this case in
minimalizing build environments. This is extremely difficult to test,
and as likely to cause fracturing of unexpected test environments as
the reduced versions of "parted" caused in anaconda years ago.

> The magic of prefering the minimal package over non-minimal package
> would be kept on package manager. For example it could sort the
> candidates on size or number of dependencies.

Maintaining it is likely to break the ability to test expected
features when compiling in userland, then expect them to work the same
way in mock or koji at build time.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 20:28:16 Ville Skyttä wrote:
> > http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka
> > -libcurl-minimal
> libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
> because it stuffs "minimal" into the version part, but the minimalism
> here is not really related to the version. Did you consider e.g.
> libcurl-minimal.so.x.y.z instead?

Are you reading it from the specfile?

It is just an implementation detail of the packaging (the RemovePathPostfixes 
feature of rpm).  The string you mentioned neither appears in the SONAME, nor 
in any file installed by the RPMs in question.

libcurl-minimal is really a drop-in replacement for libcurl, keeping its ABI.

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Alexander Bokovoy]

On Thu, 17 Mar 2016, Kamil Dudka wrote:

On Thursday 17 March 2016 08:57:19 Alexander Bokovoy wrote:

On Wed, 16 Mar 2016, Dennis Gilmore wrote:
>On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
>> On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
>> > Kamil Dudka (kdu...@redhat.com) said:
>> > > Are you reading it from the specfile?
>> > >
>> > > It is just an implementation detail of the packaging (the
>> > > RemovePathPostfixes feature of rpm).  The string you mentioned
>> > > neither
>> > > appears in the SONAME, nor in any file installed by the RPMs in
>> > > question.
>> >
>> > ... which means if the SONAME is the same, you either are dealing with
>>
>> > Conflicts:
>> Exactly.  libcurl conflicts with libcurl-minimal, which means that
>> exactly
>> one of them will be installed on any Fedora system at a time.  On a
>> regular
>> system (server, desktop, etc.) it will always be libcurl.
>>
>> On the other hand, if you need to create a minimal installation of Fedora
>> (e.g. a base image for Docker), you will pick libcurl-minimal instead of
>> libcurl, to make the set of installed packages really minimal.
>>
>> > , or relying on the behavior of ldconfig to figure out which
>> > library you happen to get at a particular time.
>>
>> ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
>> only
>> one file with that name installed at a time.
>
>And how do you transition when you need to add the full featured version?

Just a note on top of what Dennis asked: if curl minimal would lack
XML-RPC and GSSAPI support, FreeIPA client enrollment will be broken
completely. There are many uses of that within kickstart or cloud-init
files.


I am not against keeping the GSSAPI support in libcurl-minimal.  From the
discussion, it sounds like the right thing to do.

Thanks.


But what do you mean by XML-RPC?

I am not aware of any such feature in (lib)curl itself.  XML-RPC is usually
implemented in higher-level libraries.  If they use the http(s) implementation
of libcurl, they will work on top of libcurl-minimal as well.

Sorry, this is coming from xmlrpc-c which is using libcurl. As long as
the GSSAPI/GSSAPI delegation features are in libcurl, we should be fine.

--
/ Alexander Bokovoy
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Yaakov Selkowitz]

On 2016-03-16 14:07, Kamil Dudka wrote:

On Wednesday, March 16, 2016 20:28:16 Ville Skyttä wrote:

 http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka
 -libcurl-minimal

libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
because it stuffs "minimal" into the version part, but the minimalism
here is not really related to the version. Did you consider e.g.
libcurl-minimal.so.x.y.z instead?


Are you reading it from the specfile?

It is just an implementation detail of the packaging (the RemovePathPostfixes
feature of rpm).  The string you mentioned neither appears in the SONAME, nor
in any file installed by the RPMs in question.

libcurl-minimal is really a drop-in replacement for libcurl, keeping its ABI.


As RPM uses the curl command and not libcurl, maybe curl-minimal should 
be built with a static libcurl to avoid needing two libcurl packages?


--
Yaakov Selkowitz
Associate Software Engineer, ARM
Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Paul Howarth]

On Wed, 16 Mar 2016 20:28:16 +0200
Ville Skyttä  wrote:

> On Wed, Mar 16, 2016 at 6:36 PM, Kamil Dudka 
> wrote:
> > The curl and libcurl packages, which are both required by dnf,  
> 
> Hm, does dnf really require curl? On my F-23 box:
> 
> $ rpm -e --test curl
> error: Failed dependencies:
> curl is needed by (installed) rpmdevtools-8.6-2.fc23.noarch
> curl is needed by (installed) pyrpkg-1.36-1.fc23.noarch
> curl is needed by (installed)
> dracut-live-043-63.git20151211.fc23.x86_64 curl is needed by
> (installed) libreport-plugin-kerneloops-2.6.4-1.fc23.x86_64 curl is
> needed by (installed) abrt-addon-kerneloops-2.8.0-3.fc23.x86_64 curl
> is needed by (installed) abrt-addon-xorg-2.8.0-3.fc23.x86_64 curl is
> needed by (installed) rpm-4.13.0-0.rc1.12.fc23.x86_64 curl is needed
> by (installed) fedora-packager-0.5.10.7-1.fc23.noarch
> 
> Ah, rpm requires it (through %__urlhelpercmd). So I suppose there's
> the use case for curl-minimal I was wondering about.
> 
> > 
> > http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-libcurl-minimal
> >   
> 
> libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
> because it stuffs "minimal" into the version part, but the minimalism
> here is not really related to the version. Did you consider e.g.
> libcurl-minimal.so.x.y.z instead?

The installed soname is exactly the same as the regular libcurl
version, making them interchangeable. The .minimal suffix is removed by
rpm at packaging time using RemovePathPostfixes:

Paul.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Friday 18 March 2016 05:14:28 Nico Kadel-Garcia wrote:
> On Fri, Mar 18, 2016 at 4:29 AM, Petr Pisar  wrote:
> > I think the solution is have more packages delivering the same-named
> > shared library file with the same soname. Each of the packages
> > conflicting each other. Then the non-minimal package would provide RPM
> > symbols declaring compiled-in features like "Provides: libcurl(LDAP)"
> > and then each application package requiring specific feature would
> > explicitly run-require it ("Requied: libcurl(LDAP)"), besides
> > automatically genererated dependency on the soname.
> 
> I suspect the better solution is to stop burning effort rewriting the
> structure of core utilities for something that does not actually
> improve the utility

Exactly.  That is why I am not proposing a crazy plug-in system
for libcurl :-)

> , but only helps with edge cases, in this case in
> minimalizing build environments.

Neither me, nor Petr were talking about minimalizing build environments.

> This is extremely difficult to test,

Really?  What exactly were the problems you encountered while you
were testing the curl-minimal and libcurl-minimal packages?

Kamil

> and as likely to cause fracturing of unexpected test environments as
> the reduced versions of "parted" caused in anaconda years ago.
> 
> > The magic of prefering the minimal package over non-minimal package
> > would be kept on package manager. For example it could sort the
> > candidates on size or number of dependencies.
> 
> Maintaining it is likely to break the ability to test expected
> features when compiling in userland, then expect them to work the same
> way in mock or koji at build time.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Friday 18 March 2016 08:29:27 Petr Pisar wrote:
> On 2016-03-18, Kamil Dudka  wrote:
> > On Thursday, March 17, 2016 14:44:21 Przemek Klosowski wrote:
> > 
> > According to Fedora Packaging Guidelines, unversioned shared library files
> > should be installed by -devel packages:
> > 
> > https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guideline
> > s#Devel_Packages
> No. The guidelines are badly written. "unversioned shared library files"
> means "shared library unversioned file name". Spliting files between
> -libs and -devel packages is not based on sonames, but on purpose of the
> file.
> 
> Dlopening unversioned file name is as bad as linking against librires
> without sonames. To express package dependencies on soanames, one has to
> dlopen the versioned file name. Of course that means to compile in the
> soname when building an application. At the end that's what linker (ld)
> does. Yes, it would mean to patch each application.
> 
> > I will not support any solution that would allow to install multiple
> > instances of libcurl on a single system.  We (as curl maintainers)
> > have no control about which packages link which libraries and it could
> > sooner or later happen that both instances of libcurl are loaded in
> > a single process through higher-level libraries, which is not
> > a scenario supported by upstream.
> 
> I think the solution is have more packages delivering the same-named
> shared library file with the same soname. Each of the packages
> conflicting each other.

Up to this point, you are describing the originally proposed solution.  
Testing packages are available in the following Copr repository if you
want to give it a try:

https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/

> Then the non-minimal package would provide RPM
> symbols declaring compiled-in features like "Provides: libcurl(LDAP)"
> and then each application package requiring specific feature would
> explicitly run-require it ("Requied: libcurl(LDAP)"), besides
> automatically genererated dependency on the soname.

Sounds like a reasonable improvement to me.  Thanks for the suggestion!

Kamil

> The magic of prefering the minimal package over non-minimal package
> would be kept on package manager. For example it could sort the
> candidates on size or number of dependencies.
> 
> -- Petr
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Bill Nottingham]

Kamil Dudka (kdu...@redhat.com) said: 
> Are you reading it from the specfile?
> 
> It is just an implementation detail of the packaging (the RemovePathPostfixes 
> feature of rpm).  The string you mentioned neither appears in the SONAME, nor 
> in any file installed by the RPMs in question.


... which means if the SONAME is the same, you either are dealing with
Conflicts:, or relying on the behavior of ldconfig to figure out which
library you happen to get at a particular time. Neither of those sounds
like a great idea.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> Kamil Dudka (kdu...@redhat.com) said:
> > Are you reading it from the specfile?
> > 
> > It is just an implementation detail of the packaging (the
> > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > appears in the SONAME, nor in any file installed by the RPMs in question.
> 
> ... which means if the SONAME is the same, you either are dealing with
> Conflicts:

Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly one 
of them will be installed on any Fedora system at a time.  On a regular system 
(server, desktop, etc.) it will always be libcurl.

On the other hand, if you need to create a minimal installation of Fedora 
(e.g. a base image for Docker), you will pick libcurl-minimal instead of 
libcurl, to make the set of installed packages really minimal.

> , or relying on the behavior of ldconfig to figure out which
> library you happen to get at a particular time.

ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be only 
one file with that name installed at a time.

Kamil

> Neither of those sounds like a great idea.
>
> Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Bill Nottingham]

Kamil Dudka (kdu...@redhat.com) said: 
> On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > Kamil Dudka (kdu...@redhat.com) said:
> > > Are you reading it from the specfile?
> > > 
> > > It is just an implementation detail of the packaging (the
> > > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > > appears in the SONAME, nor in any file installed by the RPMs in question.
> > 
> > ... which means if the SONAME is the same, you either are dealing with
> > Conflicts:
> 
> Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly 
> one 
> of them will be installed on any Fedora system at a time.  On a regular 
> system 
> (server, desktop, etc.) it will always be libcurl.
> 
> On the other hand, if you need to create a minimal installation of Fedora 
> (e.g. a base image for Docker), you will pick libcurl-minimal instead of 
> libcurl, to make the set of installed packages really minimal.

That just seems an odd place to make a stand on size.

If you care about a consistent developer, user, and debugging experience
regardless of mechanism of delivery, you wouldn't do this in the first
place, or you'd change the global curl package. Either the features are
important, or they aren't.

If you care about minimizing size overall (neglecting the fact that
cutting out kerberos and to a lesser extent ldap dependencies don't really
save you anything due to their system library use), then might as well just
start -Os-ing random packages and throwing in busybox.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Alexander Bokovoy]

On Thu, 17 Mar 2016, Kamil Dudka wrote:

>> Just a note on top of what Dennis asked: if curl minimal would lack
>> XML-RPC and GSSAPI support, FreeIPA client enrollment will be broken
>> completely. There are many uses of that within kickstart or cloud-init
>> files.
>
>I am not against keeping the GSSAPI support in libcurl-minimal.  From the
>discussion, it sounds like the right thing to do.

Thanks.

>But what do you mean by XML-RPC?
>
>I am not aware of any such feature in (lib)curl itself.  XML-RPC is usually
>implemented in higher-level libraries.  If they use the http(s)
>implementation of libcurl, they will work on top of libcurl-minimal as
>well.

Sorry, this is coming from xmlrpc-c which is using libcurl. As long as
the GSSAPI/GSSAPI delegation features are in libcurl, we should be fine.


Thanks for clarifying it!

I have restored the GSSAPI support in the libcurl-minimal package:

http://pkgs.fedoraproject.org/cgit/rpms/curl.git/commit/?id=92700939

... and scheduled a new Copr build for that:

https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/build/168860/

Thanks!
--
/ Alexander Bokovoy
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday 17 March 2016 10:21:33 Alexander Bokovoy wrote:
> On Thu, 17 Mar 2016, Kamil Dudka wrote:
> >On Thursday 17 March 2016 08:57:19 Alexander Bokovoy wrote:
> >> On Wed, 16 Mar 2016, Dennis Gilmore wrote:
> >> >On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> >> >> On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> >> >> > Kamil Dudka (kdu...@redhat.com) said:
> >> >> > > Are you reading it from the specfile?
> >> >> > > 
> >> >> > > It is just an implementation detail of the packaging (the
> >> >> > > RemovePathPostfixes feature of rpm).  The string you mentioned
> >> >> > > neither
> >> >> > > appears in the SONAME, nor in any file installed by the RPMs in
> >> >> > > question.
> >> >> > 
> >> >> > ... which means if the SONAME is the same, you either are dealing
> >> >> > with
> >> >> 
> >> >> > Conflicts:
> >> >> Exactly.  libcurl conflicts with libcurl-minimal, which means that
> >> >> exactly
> >> >> one of them will be installed on any Fedora system at a time.  On a
> >> >> regular
> >> >> system (server, desktop, etc.) it will always be libcurl.
> >> >> 
> >> >> On the other hand, if you need to create a minimal installation of
> >> >> Fedora
> >> >> (e.g. a base image for Docker), you will pick libcurl-minimal instead
> >> >> of
> >> >> libcurl, to make the set of installed packages really minimal.
> >> >> 
> >> >> > , or relying on the behavior of ldconfig to figure out which
> >> >> > library you happen to get at a particular time.
> >> >> 
> >> >> ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> >> >> only
> >> >> one file with that name installed at a time.
> >> >
> >> >And how do you transition when you need to add the full featured
> >> >version?
> >> 
> >> Just a note on top of what Dennis asked: if curl minimal would lack
> >> XML-RPC and GSSAPI support, FreeIPA client enrollment will be broken
> >> completely. There are many uses of that within kickstart or cloud-init
> >> files.
> >
> >I am not against keeping the GSSAPI support in libcurl-minimal.  From the
> >discussion, it sounds like the right thing to do.
> 
> Thanks.
> 
> >But what do you mean by XML-RPC?
> >
> >I am not aware of any such feature in (lib)curl itself.  XML-RPC is usually
> >implemented in higher-level libraries.  If they use the http(s)
> >implementation of libcurl, they will work on top of libcurl-minimal as
> >well.
> 
> Sorry, this is coming from xmlrpc-c which is using libcurl. As long as
> the GSSAPI/GSSAPI delegation features are in libcurl, we should be fine.

Thanks for clarifying it!

I have restored the GSSAPI support in the libcurl-minimal package:

http://pkgs.fedoraproject.org/cgit/rpms/curl.git/commit/?id=92700939

... and scheduled a new Copr build for that:

https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/build/168860/

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Richard W.M. Jones]

On Thu, Mar 17, 2016 at 07:00:09PM +0100, Kamil Dudka wrote:
> On Thursday 17 March 2016 13:21:42 Przemek Klosowski wrote:
> I was (by mistake) speaking about loading libcurl's run-time dependencies
> by dlopen(), which is implemented for OpenLDAP in RHEL-5.  It used to cause 
> problems and was removed from upstream curl long time ago.

Judging by this discussion:

  https://github.com/curl/curl/issues/349

and:

> https://lists.mayfirst.org/pipermail/vtls/2015-February/20.html

that was because they implemented it badly.  They shouldn't dlopen the
external libraries directly.

libcurl should have a plugin system, so that the plugin for (eg)
libssh2 could be pushed off to $plugindir/libcurl-ssh2.so.  This is a
library containing curl code, which is linked normally (NOT using
dlopen) to libssh2.  When core libcurl starts, _it_ dlopens any
plugins found in $plugindir, and they register themselves with the
core libcurl.  The order of library loading is predictable, as if the
main program or another library linked to the main program is already
using libssh2, it won't be loaded a second time.

From there it would be easy to package the core libcurl, and the
plugins in separate RPMs.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Jason L Tibbitts III]

> "JLT" == Jason L Tibbitts  writes:

JLT> Using a different soname and patching python-pycurl to use the
JLT> maximal module if present and the minimal module otherwise would be
JLT> another possibility, assuming it's even doable.

Or, even easier, having python-curl-minimal which requires and is linked
against a curl-minimal package which has a different soname, and then
dnf can just try to import the big one before importing the small one.
Boolean dependencies in the dnf package should take care of the rest.

 - J<
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Jason L Tibbitts III]

> "KD" == Kamil Dudka  writes:

KD> Exactly.  libcurl conflicts with libcurl-minimal, which means that
KD> exactly one of them will be installed on any Fedora system at a
KD> time.  On a regular system (server, desktop, etc.) it will always be
KD> libcurl.

But... we don't really permit conflicts, and especially not for this
kind of thing.  https://fedoraproject.org/wiki/Packaging:Conflicts

There has to be a better way to do this.  Alternatives would be one way,
I guess, except that I don't know if it applies to libraries.  Using a
different soname and patching python-pycurl to use the maximal module if
present and the minimal module otherwise would be another possibility,
assuming it's even doable.

- J<
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Petr Pisar]

On 2016-03-18, Kamil Dudka  wrote:
> On Thursday, March 17, 2016 14:44:21 Przemek Klosowski wrote:
>
> According to Fedora Packaging Guidelines, unversioned shared library files 
> should be installed by -devel packages:
>
> https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages
>
No. The guidelines are badly written. "unversioned shared library files"
means "shared library unversioned file name". Spliting files between
-libs and -devel packages is not based on sonames, but on purpose of the
file.

Dlopening unversioned file name is as bad as linking against librires
without sonames. To express package dependencies on soanames, one has to
dlopen the versioned file name. Of course that means to compile in the
soname when building an application. At the end that's what linker (ld)
does. Yes, it would mean to patch each application.

> I will not support any solution that would allow to install multiple
> instances of libcurl on a single system.  We (as curl maintainers)
> have no control about which packages link which libraries and it could
> sooner or later happen that both instances of libcurl are loaded in
> a single process through higher-level libraries, which is not
> a scenario supported by upstream.
>
I think the solution is have more packages delivering the same-named
shared library file with the same soname. Each of the packages
conflicting each other. Then the non-minimal package would provide RPM
symbols declaring compiled-in features like "Provides: libcurl(LDAP)"
and then each application package requiring specific feature would
explicitly run-require it ("Requied: libcurl(LDAP)"), besides
automatically genererated dependency on the soname.

The magic of prefering the minimal package over non-minimal package
would be kept on package manager. For example it could sort the
candidates on size or number of dependencies.

-- Petr
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Bill Nottingham]

Kamil Dudka (kdu...@redhat.com) said: 
> > - "Minimizing the fedora docker base image footprint" (by yanking dnf et.al.
> > into a seprate container, making size of it much more irrelevant) - "DNF
> > into C initiative started" (enabling a much larger depythoning that doesn't
> > require differing library buidls)
> > - "F24 Self Contained Change: System Python"
> > 
> > ... the fact that we now have four separate implementation details, all of
> > which could make different parts of the others irrelevant, makes me wonder
> > about the design behind it.
> 
> So what exactly are you proposing?

1) do nothing, don't worry as much about the size difference in this context
(and/or wait for the size thing to be solved by other container minimization
efforts)

2) decide that we're OK with differing functionality in things like docker
base images provided by different builds, and figure out the general
policies around this, how we document them for users, and standard ways
Fedora as a whole wants to handle this sort of packaging.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Bill Nottingham]

Kamil Dudka (kdu...@redhat.com) said: 
> > If you care about a consistent developer, user, and debugging experience
> > regardless of mechanism of delivery, you wouldn't do this in the first
> > place, or you'd change the global curl package. Either the features are
> > important, or they aren't.
> 
> Are you implying that curl maintainers know better than users which features 
> are important for the users themselves?

... well, that's sort of what turning off features for this case implies.

> > If you care about minimizing size overall (neglecting the fact that
> > cutting out kerberos and to a lesser extent ldap dependencies don't really
> > save you anything due to their system library use), then might as well just
> > start -Os-ing random packages and throwing in busybox.
> 
> Micro-optimization is way to hell.  The system needs to optimized by design.

EXACTLY!  Maybe I'm not being clear in what I'm trying to say, but this is
the point - it needs to be designed.

1) Start with the design of who are we targeting - what jobs are then trying to
do?

2) Then, what do we produce? (Atomic, Server, Cloud, Workstation)

3) Do they have different needs? Do we build components with different features
for each? (This solution implies 'yes', but I don't recall seeing that
stated before)

4) If so, what are the differentiating features that require different
components with different feature sets? What's the job that someone is
trying to do that requires this? (The impliciation is 'download the base
image somewhat faster', but that's still very vague.)

5) Then, pick the targeted points and propose implementation of individual
features that serve the larger goal.

It's possible that, just poking my head up, I missed all the 1-4 discussion
that this is tied to. But given that the following has all popped up in
the past few months:

- "Minimizing the fedora docker base image footprint" (by yanking dnf et.al.
  into a seprate container, making size of it much more irrelevant)
- "DNF into C initiative started" (enabling a much larger depythoning that
  doesn't require differing library buidls)
- "F24 Self Contained Change: System Python"

... the fact that we now have four separate implementation details, all of which
could make different parts of the others irrelevant, makes me wonder about
the design behind it.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Bill Nottingham]

Ville Skyttä (ville.sky...@iki.fi) said: 
> On Wed, Mar 16, 2016 at 6:36 PM, Kamil Dudka  wrote:
> > The curl and libcurl packages, which are both required by dnf,
> 
> Hm, does dnf really require curl? On my F-23 box:
> 
> $ rpm -e --test curl
> error: Failed dependencies:
> curl is needed by (installed) rpmdevtools-8.6-2.fc23.noarch
> curl is needed by (installed) pyrpkg-1.36-1.fc23.noarch
> curl is needed by (installed) dracut-live-043-63.git20151211.fc23.x86_64
> curl is needed by (installed) libreport-plugin-kerneloops-2.6.4-1.fc23.x86_64
> curl is needed by (installed) abrt-addon-kerneloops-2.8.0-3.fc23.x86_64
> curl is needed by (installed) abrt-addon-xorg-2.8.0-3.fc23.x86_64
> curl is needed by (installed) rpm-4.13.0-0.rc1.12.fc23.x86_64
> curl is needed by (installed) fedora-packager-0.5.10.7-1.fc23.noarch
> 
> Ah, rpm requires it (through %__urlhelpercmd). So I suppose there's
> the use case for curl-minimal I was wondering about.
> 
> > 
> > http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-libcurl-minimal
> 
> libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
> because it stuffs "minimal" into the version part, but the minimalism
> here is not really related to the version. Did you consider e.g.
> libcurl-minimal.so.x.y.z instead?

Shipping entirely non-standard, non-upstream SONAMEs (as appears to be what
is suggested) doesn't seem like a particularly good solution to me.

If it's just used by %__urlhelpercmd, why not make that a Suggests/Enhances?
It's not as if the normal DNF usage case requires RPM to be able to install
from a URL.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 15:48:54 Dennis Gilmore wrote:
> On Wednesday, March 16, 2016 9:39:33 PM CDT Kamil Dudka wrote:
> > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > Kamil Dudka (kdu...@redhat.com) said:
> > > > Are you reading it from the specfile?
> > > > 
> > > > It is just an implementation detail of the packaging (the
> > > > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > > > appears in the SONAME, nor in any file installed by the RPMs in
> > > > question.
> > > 
> > > ... which means if the SONAME is the same, you either are dealing with
> > 
> > > Conflicts:
> > Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly
> > one of them will be installed on any Fedora system at a time.  On a
> > regular
> > system (server, desktop, etc.) it will always be libcurl.
> > 
> > On the other hand, if you need to create a minimal installation of Fedora
> > (e.g. a base image for Docker), you will pick libcurl-minimal instead of
> > libcurl, to make the set of installed packages really minimal.
> > 
> > > , or relying on the behavior of ldconfig to figure out which
> > > library you happen to get at a particular time.
> > 
> > ldconfig will always pick /usr/lib64/libcurl.so.4 because there can be
> > only
> > one file with that name installed at a time.
> 
> And how do you transition when you need to add the full featured version?

# dnf install --allowerasing curl

Kamil

> Dennis
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday, March 17, 2016 19:27:09 Richard W.M. Jones wrote:
> On Thu, Mar 17, 2016 at 07:00:09PM +0100, Kamil Dudka wrote:
> > On Thursday 17 March 2016 13:21:42 Przemek Klosowski wrote:
> > I was (by mistake) speaking about loading libcurl's run-time dependencies
> > by dlopen(), which is implemented for OpenLDAP in RHEL-5.  It used to
> > cause
> > problems and was removed from upstream curl long time ago.
> 
> Judging by this discussion:
> 
>   https://github.com/curl/curl/issues/349
> 
> and:
> > https://lists.mayfirst.org/pipermail/vtls/2015-February/20.html
> 
> that was because they implemented it badly.  They shouldn't dlopen the
> external libraries directly.
> 
> libcurl should have a plugin system, so that the plugin for (eg)
> libssh2 could be pushed off to $plugindir/libcurl-ssh2.so.  This is a
> library containing curl code, which is linked normally (NOT using
> dlopen) to libssh2.  When core libcurl starts, _it_ dlopens any
> plugins found in $plugindir, and they register themselves with the
> core libcurl.  The order of library loading is predictable, as if the
> main program or another library linked to the main program is already
> using libssh2, it won't be loaded a second time.

Sounds like a lot of work compared to my tiny patch at the packaging level.

I do not think that we have manpower to implement/maintain such a solution and 
I am not sure if curl upstream would ever be interested in that.  At least, I 
have not seen any demand on such a plug-in system in the annual surveys filed 
by the upstream community.

> From there it would be easy to package the core libcurl, and the
> plugins in separate RPMs.

Sure.  Once you have finished the upstream part, creating the RPMs is easy :-)

Now seriously.  You are welcome to propose it on the upstream mailing list to 
get some feedback on your proposal from the upstream community directly.

Kamil

> Rich.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday 16 March 2016 17:25:26 Bill Nottingham wrote:
> Kamil Dudka (kdu...@redhat.com) said:
> > On Wednesday, March 16, 2016 16:19:23 Bill Nottingham wrote:
> > > Kamil Dudka (kdu...@redhat.com) said:
> > > > Are you reading it from the specfile?
> > > > 
> > > > It is just an implementation detail of the packaging (the
> > > > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > > > appears in the SONAME, nor in any file installed by the RPMs in
> > > > question.
> > > 
> > > ... which means if the SONAME is the same, you either are dealing with
> > 
> > > Conflicts:
> > Exactly.  libcurl conflicts with libcurl-minimal, which means that exactly
> > one of them will be installed on any Fedora system at a time.  On a
> > regular system (server, desktop, etc.) it will always be libcurl.
> > 
> > On the other hand, if you need to create a minimal installation of Fedora
> > (e.g. a base image for Docker), you will pick libcurl-minimal instead of
> > libcurl, to make the set of installed packages really minimal.
> 
> That just seems an odd place to make a stand on size.
> 
> If you care about a consistent developer, user, and debugging experience
> regardless of mechanism of delivery, you wouldn't do this in the first
> place, or you'd change the global curl package. Either the features are
> important, or they aren't.

Are you implying that curl maintainers know better than users which features 
are important for the users themselves?

> If you care about minimizing size overall (neglecting the fact that
> cutting out kerberos and to a lesser extent ldap dependencies don't really
> save you anything due to their system library use), then might as well just
> start -Os-ing random packages and throwing in busybox.

Micro-optimization is way to hell.  The system needs to optimized by design.

As for Kerberos, I have already restored the feature in libcurl-minimal, based 
on the discussion in this thread:

http://pkgs.fedoraproject.org/cgit/rpms/curl.git/commit/?id=92700939

As for the dependency on OpenLDAP libraries, there was a request to drop it, 
which we were not able to satisfy at that time:

https://bugzilla.redhat.com/1029089

Kamil

> Bill
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Wednesday, March 16, 2016 15:11:07 Yaakov Selkowitz wrote:
> On 2016-03-16 14:07, Kamil Dudka wrote:
> > On Wednesday, March 16, 2016 20:28:16 Ville Skyttä wrote:
> >>>  http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdu
> >>>  dka
> >>>  -libcurl-minimal
> >> 
> >> libcurl.so.x.y.z.minimal looks pretty weird to me, and wrong as well
> >> because it stuffs "minimal" into the version part, but the minimalism
> >> here is not really related to the version. Did you consider e.g.
> >> libcurl-minimal.so.x.y.z instead?
> > 
> > Are you reading it from the specfile?
> > 
> > It is just an implementation detail of the packaging (the
> > RemovePathPostfixes feature of rpm).  The string you mentioned neither
> > appears in the SONAME, nor in any file installed by the RPMs in question.
> > 
> > libcurl-minimal is really a drop-in replacement for libcurl, keeping its
> > ABI.
> As RPM uses the curl command and not libcurl, maybe curl-minimal should
> be built with a static libcurl to avoid needing two libcurl packages?

dnf requires python-pycurl, which requires libcurl.  So any Fedora 
installation containing dnf would pull libcurl in anyway.

As explained in another thread, the libcurl and libcurl-minimal packages
will never ever be installed together on a single system.

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Josh Boyer]

On Thu, Mar 17, 2016 at 11:55 AM, Bill Nottingham  wrote:
> Kamil Dudka (kdu...@redhat.com) said:
>> > If you care about a consistent developer, user, and debugging experience
>> > regardless of mechanism of delivery, you wouldn't do this in the first
>> > place, or you'd change the global curl package. Either the features are
>> > important, or they aren't.
>>
>> Are you implying that curl maintainers know better than users which features
>> are important for the users themselves?
>
> ... well, that's sort of what turning off features for this case implies.
>
>> > If you care about minimizing size overall (neglecting the fact that
>> > cutting out kerberos and to a lesser extent ldap dependencies don't really
>> > save you anything due to their system library use), then might as well just
>> > start -Os-ing random packages and throwing in busybox.
>>
>> Micro-optimization is way to hell.  The system needs to optimized by design.
>
> EXACTLY!  Maybe I'm not being clear in what I'm trying to say, but this is
> the point - it needs to be designed.
>
> 1) Start with the design of who are we targeting - what jobs are then trying 
> to
> do?
>
> 2) Then, what do we produce? (Atomic, Server, Cloud, Workstation)
>
> 3) Do they have different needs? Do we build components with different 
> features
> for each? (This solution implies 'yes', but I don't recall seeing that
> stated before)
>
> 4) If so, what are the differentiating features that require different
> components with different feature sets? What's the job that someone is
> trying to do that requires this? (The impliciation is 'download the base
> image somewhat faster', but that's still very vague.)
>
> 5) Then, pick the targeted points and propose implementation of individual
> features that serve the larger goal.
>
> It's possible that, just poking my head up, I missed all the 1-4 discussion
> that this is tied to. But given that the following has all popped up in
> the past few months:
>
> - "Minimizing the fedora docker base image footprint" (by yanking dnf et.al.
>   into a seprate container, making size of it much more irrelevant)
> - "DNF into C initiative started" (enabling a much larger depythoning that
>   doesn't require differing library buidls)
> - "F24 Self Contained Change: System Python"
>
> ... the fact that we now have four separate implementation details, all of 
> which
> could make different parts of the others irrelevant, makes me wonder about
> the design behind it.

- Fedora Modularity Objective (creating a small Base and building out
Editions or $cool_things by using modules on top of it)

Now, none of the above goals require the curl work Kamil has done, but
it does play a positive in almost all of them if it can be
accomplished in a suitable manner.

josh
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: introducing curl-minimal and libcurl-minimal RPM packages

[Kamil Dudka]

On Thursday, March 17, 2016 14:44:21 Przemek Klosowski wrote:
> On 03/17/2016 02:00 PM, Kamil Dudka wrote:
> > On Thursday 17 March 2016 13:21:42 Przemek Klosowski wrote:
> >> Could you summarize the argument against something like
> >> 
> >> libcurl_version=REGULAR;
> >> if(!(lp=dlopen("/usr/lib64/libcurl.so",RTLD_NOW))  {
> >> 
> >>lp=dlopen("/usr/lib64/libcurl-minimal.so",RTLD_NOW);
> >>libcurl_version=MINIMAL;
> >> 
> >> }
> >> if (!lp) abort("Can't load libcurl because ",dlerror());
> > 
> > I was (by mistake) speaking about loading libcurl's run-time dependencies
> > by dlopen(), which is implemented for OpenLDAP in RHEL-5.  It used to
> > cause
> > problems and was removed from upstream curl long time ago.
> > 
> > Nevertheless, most of the reasons are valid for loading libcurl itself,
> > too:
> > 
> > - Your example would require libcurl-devel to be installed because it
> > provides> 
> >the /usr/lib64/libcurl.so symlink.  This would be yet another
> >*run-time*
> >
> >dependency of the package you patched.  See the following bug for 
example:
> >  https://bugzilla.redhat.com/215928
> 
> wait, why does libcurl.so sit in libcurl-devel? could it be simply in
> libcurl?

According to Fedora Packaging Guidelines, unversioned shared library files 
should be installed by -devel packages:

https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages

If you click on the link above your question, you will see that there was 
exactly the same problem with openldap-devel.  So it is nothing specific
to libcurl.

> of 548 /usr/lib64/lib*.so  files on my system, 358 are in
> lib*-devel packages and 190 are in regular lib* packages.
> 
> Is it a broader problem with how packages keep their libs, i.e. is one
> group or the other violating some policy?

Different packages have different policies for maintaining ABI compatibility.

> > - This approach is not compatible with the dependency scanner of
> > rmp-build.
> > 
> > - You would need to patch this way every single package to be able to load
> > 
> >libcurl-minimal.  It is very likely that maintainers of upstream
> >projects
> >would reject such Fedora-specific patches.
> 
> No, this would be only required for the curl---all the other packages
> should expect regular libcurl.so

Then it would not solve the problem with dnf because dnf uses libcurl via the 
python-pycurl binding (not the curl executable).

> > - Loading libcurl by dlopen() changes the order of global initialization
> > 
> >of system libraries (such as OpenSSL, NSS, and OpenLDAP), which itself
> >causes bugs from time to time.
> > 
> > There was a similar proposal for libcurl to load TLS libraries by
> > dlopen().
> > 
> > You can read the response of the upstream curl maintainer:
> >  https://lists.mayfirst.org/pipermail/vtls/2015-February/20.html
> 
> Yes, the portability argument I see the problem here
> 
> 
> 
> What if you created a 'minimal curl' package that would jam
> libcurl-minimal into curl:
> 
> LD_PRELOAD=/usr/lib64/libcurl-minimal.so curl

I will not support any solution that would allow to install multiple instances
of libcurl on a single system.  We (as curl maintainers) have no control about 
which packages link which libraries and it could sooner or later happen that 
both instances of libcurl are loaded in a single process through higher-level 
libraries, which is not a scenario supported by upstream.

Apart from that, LD_PRELOAD is IMO as evil as RPATH, which we are discouraged 
from using in Fedora packages.  LD_PRELOAD is pretty useful while debugging.  
However, if you use it in production, it usually causes more problems than it 
solves.

Kamil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org