Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Florian,

Yes, it's unavailable for me as well:

$ yum whatprovides /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
0
$ yum whatprovides /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
2

Thanks for making that repo split! Very helpful for us.

best,
-Derek.

On Fri, Apr 27, 2018 at 8:19 AM, Florian Riedl  wrote:

> Here is what I did:
>
> I made a copy of the v8-stable repo and named it v8-stable-build. We
> will use this new repo for building new packages. In the regular
> v8-stable repo we will only publish the RPMs that are intended for
> public use. Also, I deleted all librdkafka support libraries that were
> previously present there.
>
> If you check now, you should find that librdkafka will not be provided
> by our repo. At least, that is what it should be now. My test showed
> it is unavailable from our repo, so it should be unavailable for you,
> too.
>
> Florian
>
> 2018-04-20 23:29 GMT+02:00 Derek DiFilippo :
> > Hi Florian,
> >
> > As of now there are 8 packages in the rsyslog repo providing the
> > librdkafka++.so.1 dependency.
> > (and one package in epel, that's the last in the list below)
> >
> > thanks,
> > -Derek.
> >
> > **
> >
> > $ yum whatprovides --disablerepo=epel /usr/lib64/librdkafka++.so.1
> > Loaded plugins: langpacks, product-id, search-disabled-repos,
> > subscription-manager
> > adiscon-librdkafka1-0.8.5-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adiscon-librdkafka1-0.8.6-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.9.5-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.11.0-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.11.0-2.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.11.1-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.11.3-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > adisconbuild-librdkafka1-0.11.4-1.x86_64 : The Apache Kafka C library
> > Repo: rsyslog_v8
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> > librdkafka-0.11.3-1.el7.x86_64 : The Apache Kafka C library
> > Repo: @epel
> > Matched from:
> > Filename: /usr/lib64/librdkafka++.so.1
> >
> >
> > On Fri, Apr 20, 2018 at 11:53 AM, Derek DiFilippo 
> wrote:
> >
> >> Hi Florian:
> >>
> >> Thanks for removing that package.
> >>
> >> So you know, there are still others in the rsyslog repo that satisfy the
> >> same dependency
> >> We still see a conflict but now with one of the adiscon-librdkafka
> >> packages.
> >>
> >> The conflict used to be this:
> >>
> >> Transaction check error:
> >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> installs of librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> of librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >>
> >> and now it's this:
> >>
> >> Transaction check error:
> >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> installs of adiscon-librdkafka1-0.8.6-1.x86_64 and
> librdkafka-0.11.3-1.el7.x86_64
> >>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> of adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >>
> >>
> >> I'll be that the other packages could be removed as well. Or if
> necessary
> >> put into a separate internal repository you can use for builds.
> >>
> >> thanks,
> >> -D.
> >>
> >> On Fri, Apr 20, 2018 at 12:57 AM, Florian Riedl 
> >> wrote:
> >>
> >>> Derek, quick update.
> >>>
> >>> I removed the librdkafka1 RPM from the repository for EL6 and EL7.
> >>> Since this exact RPM was not used by rsyslog anyway, it seems safe to
> >>> let it go.
> >>>
> >>> I updated the repository accordingly, did a quick install test on my
> >>> lab machine and yum couldn't find it anymore, so I assume this worked
> >>> out right away.
> >>>
> >>> Florian
> >>>
> >>> 2018-04-20 7:46 GMT+02:00 Rainer Gerhards :
> >>> > Derek,
> >>> >
> >>> > thanks for the info. This is very useful for me. Obviously I
> >>> > misunderstood how RPMs work in that regard.
> >>> >
> >>> > As as far as librdkafka1 is involved, the root problem is clear. I
> >>> > probably need to review 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Florian Riedl]

Here is what I did:

I made a copy of the v8-stable repo and named it v8-stable-build. We
will use this new repo for building new packages. In the regular
v8-stable repo we will only publish the RPMs that are intended for
public use. Also, I deleted all librdkafka support libraries that were
previously present there.

If you check now, you should find that librdkafka will not be provided
by our repo. At least, that is what it should be now. My test showed
it is unavailable from our repo, so it should be unavailable for you,
too.

Florian

2018-04-20 23:29 GMT+02:00 Derek DiFilippo :
> Hi Florian,
>
> As of now there are 8 packages in the rsyslog repo providing the
> librdkafka++.so.1 dependency.
> (and one package in epel, that's the last in the list below)
>
> thanks,
> -Derek.
>
> **
>
> $ yum whatprovides --disablerepo=epel /usr/lib64/librdkafka++.so.1
> Loaded plugins: langpacks, product-id, search-disabled-repos,
> subscription-manager
> adiscon-librdkafka1-0.8.5-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adiscon-librdkafka1-0.8.6-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.9.5-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.11.0-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.11.0-2.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.11.1-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.11.3-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> adisconbuild-librdkafka1-0.11.4-1.x86_64 : The Apache Kafka C library
> Repo: rsyslog_v8
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
> librdkafka-0.11.3-1.el7.x86_64 : The Apache Kafka C library
> Repo: @epel
> Matched from:
> Filename: /usr/lib64/librdkafka++.so.1
>
>
> On Fri, Apr 20, 2018 at 11:53 AM, Derek DiFilippo  wrote:
>
>> Hi Florian:
>>
>> Thanks for removing that package.
>>
>> So you know, there are still others in the rsyslog repo that satisfy the
>> same dependency
>> We still see a conflict but now with one of the adiscon-librdkafka
>> packages.
>>
>> The conflict used to be this:
>>
>> Transaction check error:
>>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of 
>> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of 
>> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>
>> and now it's this:
>>
>> Transaction check error:
>>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of 
>> adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of 
>> adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>
>>
>> I'll be that the other packages could be removed as well. Or if necessary
>> put into a separate internal repository you can use for builds.
>>
>> thanks,
>> -D.
>>
>> On Fri, Apr 20, 2018 at 12:57 AM, Florian Riedl 
>> wrote:
>>
>>> Derek, quick update.
>>>
>>> I removed the librdkafka1 RPM from the repository for EL6 and EL7.
>>> Since this exact RPM was not used by rsyslog anyway, it seems safe to
>>> let it go.
>>>
>>> I updated the repository accordingly, did a quick install test on my
>>> lab machine and yum couldn't find it anymore, so I assume this worked
>>> out right away.
>>>
>>> Florian
>>>
>>> 2018-04-20 7:46 GMT+02:00 Rainer Gerhards :
>>> > Derek,
>>> >
>>> > thanks for the info. This is very useful for me. Obviously I
>>> > misunderstood how RPMs work in that regard.
>>> >
>>> > As as far as librdkafka1 is involved, the root problem is clear. I
>>> > probably need to review now if we build non-rsyslog components
>>> > somewhere else which may also cause issues.
>>> >
>>> > I think we probably also can have conflicts in regard to support
>>> > libraries, like libfastjson. This is more or less inevitable. The
>>> > rsyslog package obviously also conflicts with OS packages. Maybe a
>>> > warning might be due that folks should not add our repositories if
>>> > they also use "this and that" component (e.g. another application that
>>> > requires a different libfastjson)?
>>> >
>>> > David: what do you think?
>>> >
>>> > Rainer
>>> > PS: I never was a big fan of 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Florian,

As of now there are 8 packages in the rsyslog repo providing the
librdkafka++.so.1 dependency.
(and one package in epel, that's the last in the list below)

thanks,
-Derek.

**

$ yum whatprovides --disablerepo=epel /usr/lib64/librdkafka++.so.1
Loaded plugins: langpacks, product-id, search-disabled-repos,
subscription-manager
adiscon-librdkafka1-0.8.5-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adiscon-librdkafka1-0.8.6-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.9.5-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.11.0-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.11.0-2.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.11.1-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.11.3-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

adisconbuild-librdkafka1-0.11.4-1.x86_64 : The Apache Kafka C library
Repo: rsyslog_v8
Matched from:
Filename: /usr/lib64/librdkafka++.so.1

librdkafka-0.11.3-1.el7.x86_64 : The Apache Kafka C library
Repo: @epel
Matched from:
Filename: /usr/lib64/librdkafka++.so.1


On Fri, Apr 20, 2018 at 11:53 AM, Derek DiFilippo  wrote:

> Hi Florian:
>
> Thanks for removing that package.
>
> So you know, there are still others in the rsyslog repo that satisfy the
> same dependency
> We still see a conflict but now with one of the adiscon-librdkafka
> packages.
>
> The conflict used to be this:
>
> Transaction check error:
>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of 
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of 
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>
> and now it's this:
>
> Transaction check error:
>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of 
> adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of 
> adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>
>
> I'll be that the other packages could be removed as well. Or if necessary
> put into a separate internal repository you can use for builds.
>
> thanks,
> -D.
>
> On Fri, Apr 20, 2018 at 12:57 AM, Florian Riedl 
> wrote:
>
>> Derek, quick update.
>>
>> I removed the librdkafka1 RPM from the repository for EL6 and EL7.
>> Since this exact RPM was not used by rsyslog anyway, it seems safe to
>> let it go.
>>
>> I updated the repository accordingly, did a quick install test on my
>> lab machine and yum couldn't find it anymore, so I assume this worked
>> out right away.
>>
>> Florian
>>
>> 2018-04-20 7:46 GMT+02:00 Rainer Gerhards :
>> > Derek,
>> >
>> > thanks for the info. This is very useful for me. Obviously I
>> > misunderstood how RPMs work in that regard.
>> >
>> > As as far as librdkafka1 is involved, the root problem is clear. I
>> > probably need to review now if we build non-rsyslog components
>> > somewhere else which may also cause issues.
>> >
>> > I think we probably also can have conflicts in regard to support
>> > libraries, like libfastjson. This is more or less inevitable. The
>> > rsyslog package obviously also conflicts with OS packages. Maybe a
>> > warning might be due that folks should not add our repositories if
>> > they also use "this and that" component (e.g. another application that
>> > requires a different libfastjson)?
>> >
>> > David: what do you think?
>> >
>> > Rainer
>> > PS: I never was a big fan of doing packaging but it seems to be
>> > important enough to justify the troubles associated with it.
>> >
>> > 2018-04-20 0:36 GMT+02:00 Derek DiFilippo :
>> >> It's done by file name, not package name.
>> >>
>> >> To double check I made a simple rpm that builds and packages two of the
>> >> examples from the librdkafka repo.
>> >>
>> >> There's no mention of rsyslog or librdkafka packages anywhere. The
>> >> dependencies are driven by the files.
>> >>
>> >> Two examples below. In the second I disable epel and it finds the
>> >> librdkafka1 in the rsyslog repository.
>> >>
>> >> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install
>> >> kafka-conflict-1.1.0-Linux.rpm
>> >> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
>> >> Examining 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Florian:

Thanks for removing that package.

So you know, there are still others in the rsyslog repo that satisfy the
same dependency
We still see a conflict but now with one of the adiscon-librdkafka packages.

The conflict used to be this:

Transaction check error:
  file /usr/lib64/librdkafka++.so.1 conflicts between attempted
installs of librdkafka1-0.8.5-0.x86_64 and
librdkafka-0.11.3-1.el7.x86_64
  file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
of librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64

and now it's this:

Transaction check error:
  file /usr/lib64/librdkafka++.so.1 conflicts between attempted
installs of adiscon-librdkafka1-0.8.6-1.x86_64 and
librdkafka-0.11.3-1.el7.x86_64
  file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
of adiscon-librdkafka1-0.8.6-1.x86_64 and
librdkafka-0.11.3-1.el7.x86_64


I'll be that the other packages could be removed as well. Or if necessary
put into a separate internal repository you can use for builds.

thanks,
-D.

On Fri, Apr 20, 2018 at 12:57 AM, Florian Riedl  wrote:

> Derek, quick update.
>
> I removed the librdkafka1 RPM from the repository for EL6 and EL7.
> Since this exact RPM was not used by rsyslog anyway, it seems safe to
> let it go.
>
> I updated the repository accordingly, did a quick install test on my
> lab machine and yum couldn't find it anymore, so I assume this worked
> out right away.
>
> Florian
>
> 2018-04-20 7:46 GMT+02:00 Rainer Gerhards :
> > Derek,
> >
> > thanks for the info. This is very useful for me. Obviously I
> > misunderstood how RPMs work in that regard.
> >
> > As as far as librdkafka1 is involved, the root problem is clear. I
> > probably need to review now if we build non-rsyslog components
> > somewhere else which may also cause issues.
> >
> > I think we probably also can have conflicts in regard to support
> > libraries, like libfastjson. This is more or less inevitable. The
> > rsyslog package obviously also conflicts with OS packages. Maybe a
> > warning might be due that folks should not add our repositories if
> > they also use "this and that" component (e.g. another application that
> > requires a different libfastjson)?
> >
> > David: what do you think?
> >
> > Rainer
> > PS: I never was a big fan of doing packaging but it seems to be
> > important enough to justify the troubles associated with it.
> >
> > 2018-04-20 0:36 GMT+02:00 Derek DiFilippo :
> >> It's done by file name, not package name.
> >>
> >> To double check I made a simple rpm that builds and packages two of the
> >> examples from the librdkafka repo.
> >>
> >> There's no mention of rsyslog or librdkafka packages anywhere. The
> >> dependencies are driven by the files.
> >>
> >> Two examples below. In the second I disable epel and it finds the
> >> librdkafka1 in the rsyslog repository.
> >>
> >> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install
> >> kafka-conflict-1.1.0-Linux.rpm
> >> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
> >> Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
> >> Marking kafka-conflict-1.1.0-Linux.rpm to be installed
> >> Resolving Dependencies
> >> --> Running transaction check
> >> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
> >> --> Processing Dependency: librdkafka++.so.1()(64bit) for package:
> >> kafka-conflict-1.1.0-1.x86_64
> >> --> Running transaction check
> >> ---> Package librdkafka.x86_64 0:0.11.3-1.el7 will be installed
> >> --> Finished Dependency Resolution
> >>
> >> Dependencies Resolved
> >>
> >> 
> 
> ==
> >>  Package  Arch Version
> >>   Repository Size
> >> 
> 
> ==
> >> Installing:
> >>  kafka-conflict   x86_64   1.1.0-1
> >>   /kafka-conflict-1.1.0-Linux92 k
> >> Installing for dependencies:
> >>  librdkafka   x86_64   0.11.3-1.el7
> >>epel  326 k
> >>
> >> Transaction Summary
> >> 
> 
> ==
> >> Install  1 Package (+1 Dependent package)
> >>
> >> Total size: 418 k
> >> Installed size: 1.1 M
> >>
> >> 
> >> 
> >>
> >> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install --disablerepo=epel
> >> kafka-conflict-1.1.0-Linux.rpm
> >> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
> >> Examining kafka-conflict-1.1.0-Linux.rpm: 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Florian Riedl]

Derek, quick update.

I removed the librdkafka1 RPM from the repository for EL6 and EL7.
Since this exact RPM was not used by rsyslog anyway, it seems safe to
let it go.

I updated the repository accordingly, did a quick install test on my
lab machine and yum couldn't find it anymore, so I assume this worked
out right away.

Florian

2018-04-20 7:46 GMT+02:00 Rainer Gerhards :
> Derek,
>
> thanks for the info. This is very useful for me. Obviously I
> misunderstood how RPMs work in that regard.
>
> As as far as librdkafka1 is involved, the root problem is clear. I
> probably need to review now if we build non-rsyslog components
> somewhere else which may also cause issues.
>
> I think we probably also can have conflicts in regard to support
> libraries, like libfastjson. This is more or less inevitable. The
> rsyslog package obviously also conflicts with OS packages. Maybe a
> warning might be due that folks should not add our repositories if
> they also use "this and that" component (e.g. another application that
> requires a different libfastjson)?
>
> David: what do you think?
>
> Rainer
> PS: I never was a big fan of doing packaging but it seems to be
> important enough to justify the troubles associated with it.
>
> 2018-04-20 0:36 GMT+02:00 Derek DiFilippo :
>> It's done by file name, not package name.
>>
>> To double check I made a simple rpm that builds and packages two of the
>> examples from the librdkafka repo.
>>
>> There's no mention of rsyslog or librdkafka packages anywhere. The
>> dependencies are driven by the files.
>>
>> Two examples below. In the second I disable epel and it finds the
>> librdkafka1 in the rsyslog repository.
>>
>> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install
>> kafka-conflict-1.1.0-Linux.rpm
>> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
>> Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
>> Marking kafka-conflict-1.1.0-Linux.rpm to be installed
>> Resolving Dependencies
>> --> Running transaction check
>> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
>> --> Processing Dependency: librdkafka++.so.1()(64bit) for package:
>> kafka-conflict-1.1.0-1.x86_64
>> --> Running transaction check
>> ---> Package librdkafka.x86_64 0:0.11.3-1.el7 will be installed
>> --> Finished Dependency Resolution
>>
>> Dependencies Resolved
>>
>> ==
>>  Package  Arch Version
>>   Repository Size
>> ==
>> Installing:
>>  kafka-conflict   x86_64   1.1.0-1
>>   /kafka-conflict-1.1.0-Linux92 k
>> Installing for dependencies:
>>  librdkafka   x86_64   0.11.3-1.el7
>>epel  326 k
>>
>> Transaction Summary
>> ==
>> Install  1 Package (+1 Dependent package)
>>
>> Total size: 418 k
>> Installed size: 1.1 M
>>
>> 
>> 
>>
>> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install --disablerepo=epel
>> kafka-conflict-1.1.0-Linux.rpm
>> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
>> Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
>> Marking kafka-conflict-1.1.0-Linux.rpm to be installed
>> Resolving Dependencies
>> --> Running transaction check
>> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
>> --> Processing Dependency: librdkafka++.so.1()(64bit) for package:
>> kafka-conflict-1.1.0-1.x86_64
>> --> Running transaction check
>> ---> Package librdkafka1.x86_64 0:0.8.5-0 will be installed
>> --> Finished Dependency Resolution
>>
>> Dependencies Resolved
>>
>> ==
>>  Package   Arch  Version
>>  Repository  Size
>> ==
>> Installing:
>>  kafka-conflictx86_641.1.0-1
>>  /kafka-conflict-1.1.0-Linux 92 k
>> Installing for dependencies:
>>  librdkafka1   x86_640.8.5-0
>>  rsyslog_v8 100 k
>>
>> Transaction Summary
>> 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Derek,

thanks for the info. This is very useful for me. Obviously I
misunderstood how RPMs work in that regard.

As as far as librdkafka1 is involved, the root problem is clear. I
probably need to review now if we build non-rsyslog components
somewhere else which may also cause issues.

I think we probably also can have conflicts in regard to support
libraries, like libfastjson. This is more or less inevitable. The
rsyslog package obviously also conflicts with OS packages. Maybe a
warning might be due that folks should not add our repositories if
they also use "this and that" component (e.g. another application that
requires a different libfastjson)?

David: what do you think?

Rainer
PS: I never was a big fan of doing packaging but it seems to be
important enough to justify the troubles associated with it.

2018-04-20 0:36 GMT+02:00 Derek DiFilippo :
> It's done by file name, not package name.
>
> To double check I made a simple rpm that builds and packages two of the
> examples from the librdkafka repo.
>
> There's no mention of rsyslog or librdkafka packages anywhere. The
> dependencies are driven by the files.
>
> Two examples below. In the second I disable epel and it finds the
> librdkafka1 in the rsyslog repository.
>
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install
> kafka-conflict-1.1.0-Linux.rpm
> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
> Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
> Marking kafka-conflict-1.1.0-Linux.rpm to be installed
> Resolving Dependencies
> --> Running transaction check
> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
> --> Processing Dependency: librdkafka++.so.1()(64bit) for package:
> kafka-conflict-1.1.0-1.x86_64
> --> Running transaction check
> ---> Package librdkafka.x86_64 0:0.11.3-1.el7 will be installed
> --> Finished Dependency Resolution
>
> Dependencies Resolved
>
> ==
>  Package  Arch Version
>   Repository Size
> ==
> Installing:
>  kafka-conflict   x86_64   1.1.0-1
>   /kafka-conflict-1.1.0-Linux92 k
> Installing for dependencies:
>  librdkafka   x86_64   0.11.3-1.el7
>epel  326 k
>
> Transaction Summary
> ==
> Install  1 Package (+1 Dependent package)
>
> Total size: 418 k
> Installed size: 1.1 M
>
> 
> 
>
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install --disablerepo=epel
> kafka-conflict-1.1.0-Linux.rpm
> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
> Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
> Marking kafka-conflict-1.1.0-Linux.rpm to be installed
> Resolving Dependencies
> --> Running transaction check
> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
> --> Processing Dependency: librdkafka++.so.1()(64bit) for package:
> kafka-conflict-1.1.0-1.x86_64
> --> Running transaction check
> ---> Package librdkafka1.x86_64 0:0.8.5-0 will be installed
> --> Finished Dependency Resolution
>
> Dependencies Resolved
>
> ==
>  Package   Arch  Version
>  Repository  Size
> ==
> Installing:
>  kafka-conflictx86_641.1.0-1
>  /kafka-conflict-1.1.0-Linux 92 k
> Installing for dependencies:
>  librdkafka1   x86_640.8.5-0
>  rsyslog_v8 100 k
>
> Transaction Summary
> ==
> Install  1 Package (+1 Dependent package)
>
> Total size: 193 k
> Total download size: 100 k
> Installed size: 366 k
>
>
>
> On Thu, Apr 19, 2018 at 2:09 PM, Rainer Gerhards 
> wrote:
>
>> But packages require other packages by package name, not file name - or am
>> I wrong here? If I am not wrong, it would still mean a manual installation?
>>
>> Rainer
>>
>> Sent from phone, thus brief.
>>
>> Derek 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

It's done by file name, not package name.

To double check I made a simple rpm that builds and packages two of the
examples from the librdkafka repo.

There's no mention of rsyslog or librdkafka packages anywhere. The
dependencies are driven by the files.

Two examples below. In the second I disable epel and it finds the
librdkafka1 in the rsyslog repository.

[ec2-user@ip-172-31-18-125 ~]$ sudo yum install
kafka-conflict-1.1.0-Linux.rpm
Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
Marking kafka-conflict-1.1.0-Linux.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
--> Processing Dependency: librdkafka++.so.1()(64bit) for package:
kafka-conflict-1.1.0-1.x86_64
--> Running transaction check
---> Package librdkafka.x86_64 0:0.11.3-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==
 Package  Arch Version
  Repository Size
==
Installing:
 kafka-conflict   x86_64   1.1.0-1
  /kafka-conflict-1.1.0-Linux92 k
Installing for dependencies:
 librdkafka   x86_64   0.11.3-1.el7
   epel  326 k

Transaction Summary
==
Install  1 Package (+1 Dependent package)

Total size: 418 k
Installed size: 1.1 M




[ec2-user@ip-172-31-18-125 ~]$ sudo yum install --disablerepo=epel
kafka-conflict-1.1.0-Linux.rpm
Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
Examining kafka-conflict-1.1.0-Linux.rpm: kafka-conflict-1.1.0-1.x86_64
Marking kafka-conflict-1.1.0-Linux.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed
--> Processing Dependency: librdkafka++.so.1()(64bit) for package:
kafka-conflict-1.1.0-1.x86_64
--> Running transaction check
---> Package librdkafka1.x86_64 0:0.8.5-0 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==
 Package   Arch  Version
 Repository  Size
==
Installing:
 kafka-conflictx86_641.1.0-1
 /kafka-conflict-1.1.0-Linux 92 k
Installing for dependencies:
 librdkafka1   x86_640.8.5-0
 rsyslog_v8 100 k

Transaction Summary
==
Install  1 Package (+1 Dependent package)

Total size: 193 k
Total download size: 100 k
Installed size: 366 k



On Thu, Apr 19, 2018 at 2:09 PM, Rainer Gerhards 
wrote:

> But packages require other packages by package name, not file name - or am
> I wrong here? If I am not wrong, it would still mean a manual installation?
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 22:47:
>
> > Well, here's one way to create the bug, but it doesn't have anything to
> do
> > with the rsyslog package and it doesn't require any willing installation
> of
> > librdkafka1... this is what I've been trying to communicate -- apologies
> > that it's been so long winded.
> >
> > Imagine someone had added the rsyslog repo and either had not added epel
> or
> > disabled it.
> >
> > Then yum will find librdkafka1 to satisfy the dependency, like so:
> >
> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
> > /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
> > 0
> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
> > /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
> > 10
> >
> > No need to willingly install librdkafka1 or even care about the rsyslog
> > package.
> >
> > -D.
> >
> >
> > On Thu, Apr 19, 2018 at 1:34 PM, Rainer Gerhards <
> rgerha...@hq.adiscon.com
> > >
> > wrote:
> >
> > > Sent from phone, thus brief.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

But packages require other packages by package name, not file name - or am
I wrong here? If I am not wrong, it would still mean a manual installation?

Rainer

Sent from phone, thus brief.

Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 22:47:

> Well, here's one way to create the bug, but it doesn't have anything to do
> with the rsyslog package and it doesn't require any willing installation of
> librdkafka1... this is what I've been trying to communicate -- apologies
> that it's been so long winded.
>
> Imagine someone had added the rsyslog repo and either had not added epel or
> disabled it.
>
> Then yum will find librdkafka1 to satisfy the dependency, like so:
>
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
> /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
> 0
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
> /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
> 10
>
> No need to willingly install librdkafka1 or even care about the rsyslog
> package.
>
> -D.
>
>
> On Thu, Apr 19, 2018 at 1:34 PM, Rainer Gerhards  >
> wrote:
>
> > Sent from phone, thus brief.
> >
> > Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 22:05:
> >
> > > We install rsyslog by adding the rsyslog repository to /etc/yum.repos.d
> > and
> > > rely on yum from there.
> > >
> > > We do not modify or repackage rsyslog in any way.
> > >
> > > Florian can't guarantee anything about librdkafka1, unfortunately. Once
> > > you've added the repositories to yum.repos.d, you get the following:
> > >
> > > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> > > /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
> > > 10
> > > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> > > /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
> > > 1
> > >
> > > Notice -- yum can locate a package that fulfills the dependency based
> on
> > > file. This is a straight up conflict.
> > >
> > > Once again -- we've solved this by building librdkafka from source and
> > > linking statically. But, I'm pestering here because if your team thinks
> > > that everything is ok because they gave the rpm  package a different
> > > name... that's not the case.
> > >
> >
> > No, it's clear for several hours now that this is wrong and not needed.
> >
> > BUT: the question is how you experienced that EXCEPT by willingly
> > installing librdkafka1?
> >
> > As far as we know, it is never used nor required by anyone. If it is
> > dragged in by rsyslog, there is a bug inside the rsyslog package. So
> rather
> > than just removing librdkafka1, I want to make sure we fix the bug that
> > invalidly installs librdkafka1.
> >
> > This is what I am after.
> >
> > Rainer
> >
> > >
> > > -Derek.
> > >
> > > On Thu, Apr 19, 2018 at 12:55 PM, Rainer Gerhards <
> > > rgerha...@hq.adiscon.com>
> > > wrote:
> > >
> > > > Mmm. Even then it doesn't totally explain. At least if Florian is
> > > > correct in that librdkafka1 was never installed anywhere
> automatically
> > > and
> > > > was never needed by rsyslog. I should call it a day...
> > > >
> > > > Rainer
> > > >
> > > > Sent from phone, thus brief.
> > > >
> > > > Rainer Gerhards  schrieb am Do., 19. Apr.
> > > 2018,
> > > > 21:46:
> > > >
> > > > > Ohhh... Are you a software developer who packages for third parties
> > > (like
> > > > > we)? Are you concerned the a user of both of our products rubs into
> > > this
> > > > > problem? Then this explains everything...
> > > > >
> > > > > I was always under the impression that you are a user who installs
> a
> > > > > different package in addition to rsyslog. Guess I got that wrong,
> > > > correct?
> > > > > If so, that explains everything to me... If so, sorry for not
> > > > understanding
> > > > > the use case.
> > > > >
> > > > > Rainer
> > > > >
> > > > > Sent from phone, thus brief.
> > > > >
> > > > > Derek DiFilippo  schrieb am Mo., 16. Apr. 2018,
> > > 20:19:
> > > > >
> > > > >> Hi Rainer, hi everyone,
> > > > >>
> > > > >> In addition to a dependency on rsyslog, one of our packages now
> > > depends
> > > > on
> > > > >> librdkafka and librdkafka-devel.
> > > > >>
> > > > >> We want to use the librdkafka packages provided by epel, nothing
> > > fancy.
> > > > >>
> > > > >> I'm seeing a conflict between the librdkafka1 package in the
> rsyslog
> > > > repo
> > > > >> (note the "1" at the end of the library name) and the ones
> provided
> > by
> > > > >> epel.
> > > > >>
> > > > >> ...
> > > > >> ...
> > > > >>  librdkafka  x86_64 0.11.3-1.el7   epel
> > > > >> 326 k
> > > > >>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > > > >>  43 k
> > > > >>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > > > >>   100 k
> > > > >> ...
> > > > >> ...
> > > > >>
> > > > >> Transaction check error:
> > > > >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> > > installs
> > > > >> 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Well, here's one way to create the bug, but it doesn't have anything to do
with the rsyslog package and it doesn't require any willing installation of
librdkafka1... this is what I've been trying to communicate -- apologies
that it's been so long winded.

Imagine someone had added the rsyslog repo and either had not added epel or
disabled it.

Then yum will find librdkafka1 to satisfy the dependency, like so:

[ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
/usr/lib64/librdkafka++.so.1 | grep epel | wc -l
0
[ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides --disablerepo=epel
/usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
10

No need to willingly install librdkafka1 or even care about the rsyslog
package.

-D.


On Thu, Apr 19, 2018 at 1:34 PM, Rainer Gerhards 
wrote:

> Sent from phone, thus brief.
>
> Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 22:05:
>
> > We install rsyslog by adding the rsyslog repository to /etc/yum.repos.d
> and
> > rely on yum from there.
> >
> > We do not modify or repackage rsyslog in any way.
> >
> > Florian can't guarantee anything about librdkafka1, unfortunately. Once
> > you've added the repositories to yum.repos.d, you get the following:
> >
> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> > /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
> > 10
> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> > /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
> > 1
> >
> > Notice -- yum can locate a package that fulfills the dependency based on
> > file. This is a straight up conflict.
> >
> > Once again -- we've solved this by building librdkafka from source and
> > linking statically. But, I'm pestering here because if your team thinks
> > that everything is ok because they gave the rpm  package a different
> > name... that's not the case.
> >
>
> No, it's clear for several hours now that this is wrong and not needed.
>
> BUT: the question is how you experienced that EXCEPT by willingly
> installing librdkafka1?
>
> As far as we know, it is never used nor required by anyone. If it is
> dragged in by rsyslog, there is a bug inside the rsyslog package. So rather
> than just removing librdkafka1, I want to make sure we fix the bug that
> invalidly installs librdkafka1.
>
> This is what I am after.
>
> Rainer
>
> >
> > -Derek.
> >
> > On Thu, Apr 19, 2018 at 12:55 PM, Rainer Gerhards <
> > rgerha...@hq.adiscon.com>
> > wrote:
> >
> > > Mmm. Even then it doesn't totally explain. At least if Florian is
> > > correct in that librdkafka1 was never installed anywhere automatically
> > and
> > > was never needed by rsyslog. I should call it a day...
> > >
> > > Rainer
> > >
> > > Sent from phone, thus brief.
> > >
> > > Rainer Gerhards  schrieb am Do., 19. Apr.
> > 2018,
> > > 21:46:
> > >
> > > > Ohhh... Are you a software developer who packages for third parties
> > (like
> > > > we)? Are you concerned the a user of both of our products rubs into
> > this
> > > > problem? Then this explains everything...
> > > >
> > > > I was always under the impression that you are a user who installs a
> > > > different package in addition to rsyslog. Guess I got that wrong,
> > > correct?
> > > > If so, that explains everything to me... If so, sorry for not
> > > understanding
> > > > the use case.
> > > >
> > > > Rainer
> > > >
> > > > Sent from phone, thus brief.
> > > >
> > > > Derek DiFilippo  schrieb am Mo., 16. Apr. 2018,
> > 20:19:
> > > >
> > > >> Hi Rainer, hi everyone,
> > > >>
> > > >> In addition to a dependency on rsyslog, one of our packages now
> > depends
> > > on
> > > >> librdkafka and librdkafka-devel.
> > > >>
> > > >> We want to use the librdkafka packages provided by epel, nothing
> > fancy.
> > > >>
> > > >> I'm seeing a conflict between the librdkafka1 package in the rsyslog
> > > repo
> > > >> (note the "1" at the end of the library name) and the ones provided
> by
> > > >> epel.
> > > >>
> > > >> ...
> > > >> ...
> > > >>  librdkafka  x86_64 0.11.3-1.el7   epel
> > > >> 326 k
> > > >>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > > >>  43 k
> > > >>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > > >>   100 k
> > > >> ...
> > > >> ...
> > > >>
> > > >> Transaction check error:
> > > >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> > installs
> > > >> of
> > > >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > > >>   file /usr/lib64/librdkafka.so.1 conflicts between attempted
> installs
> > > of
> > > >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > > >>
> > > >> Can librdkafka1 be removed from the rsyslog repo? What is the
> > rationale
> > > >> for
> > > >> pinning a version of librdkafka via the librdkafka1 package?
> > > >>
> > > >> If librdkafka1 must stay in the rsyslog repo what do you recommend
> as
> > > the
> > > >> best practice for resolving 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Sent from phone, thus brief.

Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 22:05:

> We install rsyslog by adding the rsyslog repository to /etc/yum.repos.d and
> rely on yum from there.
>
> We do not modify or repackage rsyslog in any way.
>
> Florian can't guarantee anything about librdkafka1, unfortunately. Once
> you've added the repositories to yum.repos.d, you get the following:
>
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
> 10
> [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
> /usr/lib64/librdkafka++.so.1 | grep epel | wc -l
> 1
>
> Notice -- yum can locate a package that fulfills the dependency based on
> file. This is a straight up conflict.
>
> Once again -- we've solved this by building librdkafka from source and
> linking statically. But, I'm pestering here because if your team thinks
> that everything is ok because they gave the rpm  package a different
> name... that's not the case.
>

No, it's clear for several hours now that this is wrong and not needed.

BUT: the question is how you experienced that EXCEPT by willingly
installing librdkafka1?

As far as we know, it is never used nor required by anyone. If it is
dragged in by rsyslog, there is a bug inside the rsyslog package. So rather
than just removing librdkafka1, I want to make sure we fix the bug that
invalidly installs librdkafka1.

This is what I am after.

Rainer

>
> -Derek.
>
> On Thu, Apr 19, 2018 at 12:55 PM, Rainer Gerhards <
> rgerha...@hq.adiscon.com>
> wrote:
>
> > Mmm. Even then it doesn't totally explain. At least if Florian is
> > correct in that librdkafka1 was never installed anywhere automatically
> and
> > was never needed by rsyslog. I should call it a day...
> >
> > Rainer
> >
> > Sent from phone, thus brief.
> >
> > Rainer Gerhards  schrieb am Do., 19. Apr.
> 2018,
> > 21:46:
> >
> > > Ohhh... Are you a software developer who packages for third parties
> (like
> > > we)? Are you concerned the a user of both of our products rubs into
> this
> > > problem? Then this explains everything...
> > >
> > > I was always under the impression that you are a user who installs a
> > > different package in addition to rsyslog. Guess I got that wrong,
> > correct?
> > > If so, that explains everything to me... If so, sorry for not
> > understanding
> > > the use case.
> > >
> > > Rainer
> > >
> > > Sent from phone, thus brief.
> > >
> > > Derek DiFilippo  schrieb am Mo., 16. Apr. 2018,
> 20:19:
> > >
> > >> Hi Rainer, hi everyone,
> > >>
> > >> In addition to a dependency on rsyslog, one of our packages now
> depends
> > on
> > >> librdkafka and librdkafka-devel.
> > >>
> > >> We want to use the librdkafka packages provided by epel, nothing
> fancy.
> > >>
> > >> I'm seeing a conflict between the librdkafka1 package in the rsyslog
> > repo
> > >> (note the "1" at the end of the library name) and the ones provided by
> > >> epel.
> > >>
> > >> ...
> > >> ...
> > >>  librdkafka  x86_64 0.11.3-1.el7   epel
> > >> 326 k
> > >>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > >>  43 k
> > >>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > >>   100 k
> > >> ...
> > >> ...
> > >>
> > >> Transaction check error:
> > >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> installs
> > >> of
> > >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > >>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> > of
> > >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > >>
> > >> Can librdkafka1 be removed from the rsyslog repo? What is the
> rationale
> > >> for
> > >> pinning a version of librdkafka via the librdkafka1 package?
> > >>
> > >> If librdkafka1 must stay in the rsyslog repo what do you recommend as
> > the
> > >> best practice for resolving this conflict?
> > >>
> > >> I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
> are
> > >> they there? Is the solution to change the dependency in *our* package
> > form
> > >> librdkafka to adiscon-librdkafka? I might be comfortable with that if
> I
> > >> know why the adiscon packages are there.
> > >>
> > >> As a fallback we could add librdkafka as an external project via CMake
> > and
> > >> build from source but I wanted to check in here first before
> proceeding.
> > >>
> > >> Apologies if I missed something obvious. I did my best googling
> > >> due-diligence and couldn't find anyhing satisfactory.
> > >>
> > >> Thanks for your time & patience!
> > >> ___
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com/professional-services/
> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

We install rsyslog by adding the rsyslog repository to /etc/yum.repos.d and
rely on yum from there.

We do not modify or repackage rsyslog in any way.

Florian can't guarantee anything about librdkafka1, unfortunately. Once
you've added the repositories to yum.repos.d, you get the following:

[ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
/usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l
10
[ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides
/usr/lib64/librdkafka++.so.1 | grep epel | wc -l
1

Notice -- yum can locate a package that fulfills the dependency based on
file. This is a straight up conflict.

Once again -- we've solved this by building librdkafka from source and
linking statically. But, I'm pestering here because if your team thinks
that everything is ok because they gave the rpm  package a different
name... that's not the case.

-Derek.

On Thu, Apr 19, 2018 at 12:55 PM, Rainer Gerhards 
wrote:

> Mmm. Even then it doesn't totally explain. At least if Florian is
> correct in that librdkafka1 was never installed anywhere automatically and
> was never needed by rsyslog. I should call it a day...
>
> Rainer
>
> Sent from phone, thus brief.
>
> Rainer Gerhards  schrieb am Do., 19. Apr. 2018,
> 21:46:
>
> > Ohhh... Are you a software developer who packages for third parties (like
> > we)? Are you concerned the a user of both of our products rubs into this
> > problem? Then this explains everything...
> >
> > I was always under the impression that you are a user who installs a
> > different package in addition to rsyslog. Guess I got that wrong,
> correct?
> > If so, that explains everything to me... If so, sorry for not
> understanding
> > the use case.
> >
> > Rainer
> >
> > Sent from phone, thus brief.
> >
> > Derek DiFilippo  schrieb am Mo., 16. Apr. 2018, 20:19:
> >
> >> Hi Rainer, hi everyone,
> >>
> >> In addition to a dependency on rsyslog, one of our packages now depends
> on
> >> librdkafka and librdkafka-devel.
> >>
> >> We want to use the librdkafka packages provided by epel, nothing fancy.
> >>
> >> I'm seeing a conflict between the librdkafka1 package in the rsyslog
> repo
> >> (note the "1" at the end of the library name) and the ones provided by
> >> epel.
> >>
> >> ...
> >> ...
> >>  librdkafka  x86_64 0.11.3-1.el7   epel
> >> 326 k
> >>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> >>  43 k
> >>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> >>   100 k
> >> ...
> >> ...
> >>
> >> Transaction check error:
> >>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs
> >> of
> >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> of
> >> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >>
> >> Can librdkafka1 be removed from the rsyslog repo? What is the rationale
> >> for
> >> pinning a version of librdkafka via the librdkafka1 package?
> >>
> >> If librdkafka1 must stay in the rsyslog repo what do you recommend as
> the
> >> best practice for resolving this conflict?
> >>
> >> I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why are
> >> they there? Is the solution to change the dependency in *our* package
> form
> >> librdkafka to adiscon-librdkafka? I might be comfortable with that if I
> >> know why the adiscon packages are there.
> >>
> >> As a fallback we could add librdkafka as an external project via CMake
> and
> >> build from source but I wanted to check in here first before proceeding.
> >>
> >> Apologies if I missed something obvious. I did my best googling
> >> due-diligence and couldn't find anyhing satisfactory.
> >>
> >> Thanks for your time & patience!
> >> ___
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >>
> >
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Mmm. Even then it doesn't totally explain. At least if Florian is
correct in that librdkafka1 was never installed anywhere automatically and
was never needed by rsyslog. I should call it a day...

Rainer

Sent from phone, thus brief.

Rainer Gerhards  schrieb am Do., 19. Apr. 2018,
21:46:

> Ohhh... Are you a software developer who packages for third parties (like
> we)? Are you concerned the a user of both of our products rubs into this
> problem? Then this explains everything...
>
> I was always under the impression that you are a user who installs a
> different package in addition to rsyslog. Guess I got that wrong, correct?
> If so, that explains everything to me... If so, sorry for not understanding
> the use case.
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo  schrieb am Mo., 16. Apr. 2018, 20:19:
>
>> Hi Rainer, hi everyone,
>>
>> In addition to a dependency on rsyslog, one of our packages now depends on
>> librdkafka and librdkafka-devel.
>>
>> We want to use the librdkafka packages provided by epel, nothing fancy.
>>
>> I'm seeing a conflict between the librdkafka1 package in the rsyslog repo
>> (note the "1" at the end of the library name) and the ones provided by
>> epel.
>>
>> ...
>> ...
>>  librdkafka  x86_64 0.11.3-1.el7   epel
>> 326 k
>>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
>>  43 k
>>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
>>   100 k
>> ...
>> ...
>>
>> Transaction check error:
>>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs
>> of
>> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of
>> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>>
>> Can librdkafka1 be removed from the rsyslog repo? What is the rationale
>> for
>> pinning a version of librdkafka via the librdkafka1 package?
>>
>> If librdkafka1 must stay in the rsyslog repo what do you recommend as the
>> best practice for resolving this conflict?
>>
>> I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why are
>> they there? Is the solution to change the dependency in *our* package form
>> librdkafka to adiscon-librdkafka? I might be comfortable with that if I
>> know why the adiscon packages are there.
>>
>> As a fallback we could add librdkafka as an external project via CMake and
>> build from source but I wanted to check in here first before proceeding.
>>
>> Apologies if I missed something obvious. I did my best googling
>> due-diligence and couldn't find anyhing satisfactory.
>>
>> Thanks for your time & patience!
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Ohhh... Are you a software developer who packages for third parties (like
we)? Are you concerned the a user of both of our products rubs into this
problem? Then this explains everything...

I was always under the impression that you are a user who installs a
different package in addition to rsyslog. Guess I got that wrong, correct?
If so, that explains everything to me... If so, sorry for not understanding
the use case.

Rainer

Sent from phone, thus brief.

Derek DiFilippo  schrieb am Mo., 16. Apr. 2018, 20:19:

> Hi Rainer, hi everyone,
>
> In addition to a dependency on rsyslog, one of our packages now depends on
> librdkafka and librdkafka-devel.
>
> We want to use the librdkafka packages provided by epel, nothing fancy.
>
> I'm seeing a conflict between the librdkafka1 package in the rsyslog repo
> (note the "1" at the end of the library name) and the ones provided by
> epel.
>
> ...
> ...
>  librdkafka  x86_64 0.11.3-1.el7   epel
> 326 k
>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
>  43 k
>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
>   100 k
> ...
> ...
>
> Transaction check error:
>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>
> Can librdkafka1 be removed from the rsyslog repo? What is the rationale for
> pinning a version of librdkafka via the librdkafka1 package?
>
> If librdkafka1 must stay in the rsyslog repo what do you recommend as the
> best practice for resolving this conflict?
>
> I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why are
> they there? Is the solution to change the dependency in *our* package form
> librdkafka to adiscon-librdkafka? I might be comfortable with that if I
> know why the adiscon packages are there.
>
> As a fallback we could add librdkafka as an external project via CMake and
> build from source but I wanted to check in here first before proceeding.
>
> Apologies if I missed something obvious. I did my best googling
> due-diligence and couldn't find anyhing satisfactory.
>
> Thanks for your time & patience!
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Rainer,

yum sees those as different packages, not different versions of the same
package.

So we have two differently named packages installing the same file. That's
a conflict.

It's easy to reproduce. First yum install librdkafka1 from rsyslog then try
installing librdkafka from epel.

$ yum install librdkafka1
$ yum install librdkafka

Transaction check error:
  file /usr/lib64/librdkafka++.so.1 from install of
librdkafka-0.11.3-1.el7.x86_64 conflicts with file from package
librdkafka1-0.8.5-0.x86_64
  file /usr/lib64/librdkafka.so.1 from install of
librdkafka-0.11.3-1.el7.x86_64 conflicts with file from package
librdkafka1-0.8.5-0.x86_64

-D.

On Thu, Apr 19, 2018 at 12:40 PM, Rainer Gerhards 
wrote:

> Or is it so that yum takes the 1 in librdkafka1 as a version and this means
> librdkafka1 has higher priority than librdkafka?
>
> If so, why does in our setup a yum install librdkafka indeed install
> librdkafka and not librdkafka1 albeit both repos are configured?
>
> That counters the idea of version numbers...
>
> Rainer
>
> Sent from phone, thus brief.
>
> Rainer Gerhards  schrieb am Do., 19. Apr. 2018,
> 21:32:
>
> > Sorry if I am dense, but why did you install librdkafka1?
> >
> > Maybe I really don't understand anything at all, then so be it. I just
> > prefer to know why things happen.
> >
> > I understand that there should be no kafka package at all in our repo.
> > That I always requested. I understand that two conflicting packages are
> > bad. But I do not understand why there is a problem if the packages are
> > differently named and only one of them is requested by other packages.
> >
> > Or does yum search all packages and checks for conflicts even if the
> > conflicting ones are never to be installed together? That would explain
> the
> > issue to me.
> >
> > If that's the case we need to reconsider providing packages for
> components
> > that need addtl requirements and so I think it is an important issue to
> > clean up. It makes me feel very uncomfortable that we cannot reproduce
> the
> > issue.
> >
> > Rainer
> >
> > Sent from phone, thus brief.
> >
> > Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 21:06:
> >
> >> Hi everyone,
> >>
> >> The situation is exactly as described in my original email. We had a
> >> combination of packages and dependencies that led to the yum conflict.
> >> It's
> >> there, it's hard to get at, but we did it.  The solution was to build
> from
> >> source and link statically because we can't play around with random
> >> packages from your repository.
> >>
> >> To illustrate this I built a test package that requires librdkafka. On a
> >> clean AWS system if I happen to install librdkafka1 from the rsyslog
> repo
> >> (say something, somewhere, somehow, did this in the past), my test
> package
> >> installs happily... but, unbeknownst to me I am pinned to the *old*
> >> version
> >> of librdkafka and not the most recent one from epel that I was expecting
> >> to
> >> use in my product. Sure I could add some paranoid workarounds -- but for
> >> all of my other packages I am assuming there aren't magical packages out
> >> there spoofing the ones I'm expecting from epel. How paranoid should I
> >> get?
> >>
> >> There's no way having librdkafka1 in your repo is correct behaviour. In
> >> principle you should remove it and let epel provide it.
> >>
> >> In practice, if no one cares, so be it.
> >>
> >> Try this: what would stop you from having a "libstdc++1" package in your
> >> public rsyslog repo that fulfills all of the dependencies provided by
> the
> >> RHEL7 ibstdc++. Would that make sense to you? Would that be a "conflict
> >> with other software that I'm running"? I would argue it's a lack of
> >> understanding by whomever put the "libstdc++1" package up on the rsyslog
> >> repo.
> >>
> >> 
> >>
> >> To illustrate this a bit more, try the following on a RHEL system. And
> as
> >> a
> >> mental exercise replace "librdkafka" with "libstdc++" to get a sense of
> >> what I mean.
> >>
> >> $ repoquery -il librdkafka
> >> Name: librdkafka
> >> Version : 0.11.3
> >> Release : 1.el7
> >> Architecture: x86_64
> >> Size: 1020930
> >> Packager: Fedora Project
> >> Group   : Development/Libraries
> >> URL : https://github.com/edenhill/librdkafka
> >> Repository  : epel
> >> Summary : The Apache Kafka C library
> >> Source  : librdkafka-0.11.3-1.el7.src.rpm
> >> Description :
> >> Librdkafka is a C/C++ library implementation of the Apache Kafka
> protocol,
> >> containing both Producer and Consumer support.
> >> It was designed with message delivery reliability and high performance
> in
> >> mind,
> >> current figures exceed 80 messages/second for the producer and 3
> >> million
> >> messages/second for the consumer.
> >> /usr/lib64/librdkafka++.so.1
> >> /usr/lib64/librdkafka.so.1
> >> /usr/share/doc/librdkafka-0.11.3
> >> 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Or is it so that yum takes the 1 in librdkafka1 as a version and this means
librdkafka1 has higher priority than librdkafka?

If so, why does in our setup a yum install librdkafka indeed install
librdkafka and not librdkafka1 albeit both repos are configured?

That counters the idea of version numbers...

Rainer

Sent from phone, thus brief.

Rainer Gerhards  schrieb am Do., 19. Apr. 2018,
21:32:

> Sorry if I am dense, but why did you install librdkafka1?
>
> Maybe I really don't understand anything at all, then so be it. I just
> prefer to know why things happen.
>
> I understand that there should be no kafka package at all in our repo.
> That I always requested. I understand that two conflicting packages are
> bad. But I do not understand why there is a problem if the packages are
> differently named and only one of them is requested by other packages.
>
> Or does yum search all packages and checks for conflicts even if the
> conflicting ones are never to be installed together? That would explain the
> issue to me.
>
> If that's the case we need to reconsider providing packages for components
> that need addtl requirements and so I think it is an important issue to
> clean up. It makes me feel very uncomfortable that we cannot reproduce the
> issue.
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 21:06:
>
>> Hi everyone,
>>
>> The situation is exactly as described in my original email. We had a
>> combination of packages and dependencies that led to the yum conflict.
>> It's
>> there, it's hard to get at, but we did it.  The solution was to build from
>> source and link statically because we can't play around with random
>> packages from your repository.
>>
>> To illustrate this I built a test package that requires librdkafka. On a
>> clean AWS system if I happen to install librdkafka1 from the rsyslog repo
>> (say something, somewhere, somehow, did this in the past), my test package
>> installs happily... but, unbeknownst to me I am pinned to the *old*
>> version
>> of librdkafka and not the most recent one from epel that I was expecting
>> to
>> use in my product. Sure I could add some paranoid workarounds -- but for
>> all of my other packages I am assuming there aren't magical packages out
>> there spoofing the ones I'm expecting from epel. How paranoid should I
>> get?
>>
>> There's no way having librdkafka1 in your repo is correct behaviour. In
>> principle you should remove it and let epel provide it.
>>
>> In practice, if no one cares, so be it.
>>
>> Try this: what would stop you from having a "libstdc++1" package in your
>> public rsyslog repo that fulfills all of the dependencies provided by the
>> RHEL7 ibstdc++. Would that make sense to you? Would that be a "conflict
>> with other software that I'm running"? I would argue it's a lack of
>> understanding by whomever put the "libstdc++1" package up on the rsyslog
>> repo.
>>
>> 
>>
>> To illustrate this a bit more, try the following on a RHEL system. And as
>> a
>> mental exercise replace "librdkafka" with "libstdc++" to get a sense of
>> what I mean.
>>
>> $ repoquery -il librdkafka
>> Name: librdkafka
>> Version : 0.11.3
>> Release : 1.el7
>> Architecture: x86_64
>> Size: 1020930
>> Packager: Fedora Project
>> Group   : Development/Libraries
>> URL : https://github.com/edenhill/librdkafka
>> Repository  : epel
>> Summary : The Apache Kafka C library
>> Source  : librdkafka-0.11.3-1.el7.src.rpm
>> Description :
>> Librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
>> containing both Producer and Consumer support.
>> It was designed with message delivery reliability and high performance in
>> mind,
>> current figures exceed 80 messages/second for the producer and 3
>> million
>> messages/second for the consumer.
>> /usr/lib64/librdkafka++.so.1
>> /usr/lib64/librdkafka.so.1
>> /usr/share/doc/librdkafka-0.11.3
>> /usr/share/doc/librdkafka-0.11.3/CONFIGURATION.md
>> /usr/share/doc/librdkafka-0.11.3/INTRODUCTION.md
>> /usr/share/doc/librdkafka-0.11.3/README.md
>> /usr/share/licenses/librdkafka-0.11.3
>> /usr/share/licenses/librdkafka-0.11.3/LICENSE
>> /usr/share/licenses/librdkafka-0.11.3/LICENSE.pycrc
>> /usr/share/licenses/librdkafka-0.11.3/LICENSE.snappy
>>
>>
>> $ repoquery -il librdkafka1
>> Name: librdkafka1
>> Version : 0.8.5
>> Release : 0
>> Architecture: x86_64
>> Size: 280660
>> Packager: None
>> Group   : Development/Libraries/C and C++
>> URL : https://github.com/edenhill/librdkafka
>> Repository  : rsyslog_v8
>> Summary : The Apache Kafka C library
>> Source  : librdkafka-0.8.5-0.src.rpm
>> Description :
>> librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
>> containing both Producer and Consumer support.
>> /usr/lib64/librdkafka++.so.1
>> /usr/lib64/librdkafka.so.1
>> 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Rainier,

We install software on systems that aren't always under our control.

I have zero interest in librdkafka1 but I also have zero control over
whether some other package has installed it.

If yum already sees that dependency as satisfied by your librdkafka1, I'm
either stuck with that old version or headed towards a conflict when I try
to pull in the "offical" librdkafka package from epel.

thanks,
-Derek.




On Thu, Apr 19, 2018 at 12:32 PM, Rainer Gerhards 
wrote:

> Sorry if I am dense, but why did you install librdkafka1?
>
> Maybe I really don't understand anything at all, then so be it. I just
> prefer to know why things happen.
>
> I understand that there should be no kafka package at all in our repo. That
> I always requested. I understand that two conflicting packages are bad. But
> I do not understand why there is a problem if the packages are differently
> named and only one of them is requested by other packages.
>
> Or does yum search all packages and checks for conflicts even if the
> conflicting ones are never to be installed together? That would explain the
> issue to me.
>
> If that's the case we need to reconsider providing packages for components
> that need addtl requirements and so I think it is an important issue to
> clean up. It makes me feel very uncomfortable that we cannot reproduce the
> issue.
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 21:06:
>
> > Hi everyone,
> >
> > The situation is exactly as described in my original email. We had a
> > combination of packages and dependencies that led to the yum conflict.
> It's
> > there, it's hard to get at, but we did it.  The solution was to build
> from
> > source and link statically because we can't play around with random
> > packages from your repository.
> >
> > To illustrate this I built a test package that requires librdkafka. On a
> > clean AWS system if I happen to install librdkafka1 from the rsyslog repo
> > (say something, somewhere, somehow, did this in the past), my test
> package
> > installs happily... but, unbeknownst to me I am pinned to the *old*
> version
> > of librdkafka and not the most recent one from epel that I was expecting
> to
> > use in my product. Sure I could add some paranoid workarounds -- but for
> > all of my other packages I am assuming there aren't magical packages out
> > there spoofing the ones I'm expecting from epel. How paranoid should I
> get?
> >
> > There's no way having librdkafka1 in your repo is correct behaviour. In
> > principle you should remove it and let epel provide it.
> >
> > In practice, if no one cares, so be it.
> >
> > Try this: what would stop you from having a "libstdc++1" package in your
> > public rsyslog repo that fulfills all of the dependencies provided by the
> > RHEL7 ibstdc++. Would that make sense to you? Would that be a "conflict
> > with other software that I'm running"? I would argue it's a lack of
> > understanding by whomever put the "libstdc++1" package up on the rsyslog
> > repo.
> >
> > 
> >
> > To illustrate this a bit more, try the following on a RHEL system. And
> as a
> > mental exercise replace "librdkafka" with "libstdc++" to get a sense of
> > what I mean.
> >
> > $ repoquery -il librdkafka
> > Name: librdkafka
> > Version : 0.11.3
> > Release : 1.el7
> > Architecture: x86_64
> > Size: 1020930
> > Packager: Fedora Project
> > Group   : Development/Libraries
> > URL : https://github.com/edenhill/librdkafka
> > Repository  : epel
> > Summary : The Apache Kafka C library
> > Source  : librdkafka-0.11.3-1.el7.src.rpm
> > Description :
> > Librdkafka is a C/C++ library implementation of the Apache Kafka
> protocol,
> > containing both Producer and Consumer support.
> > It was designed with message delivery reliability and high performance in
> > mind,
> > current figures exceed 80 messages/second for the producer and 3
> > million
> > messages/second for the consumer.
> > /usr/lib64/librdkafka++.so.1
> > /usr/lib64/librdkafka.so.1
> > /usr/share/doc/librdkafka-0.11.3
> > /usr/share/doc/librdkafka-0.11.3/CONFIGURATION.md
> > /usr/share/doc/librdkafka-0.11.3/INTRODUCTION.md
> > /usr/share/doc/librdkafka-0.11.3/README.md
> > /usr/share/licenses/librdkafka-0.11.3
> > /usr/share/licenses/librdkafka-0.11.3/LICENSE
> > /usr/share/licenses/librdkafka-0.11.3/LICENSE.pycrc
> > /usr/share/licenses/librdkafka-0.11.3/LICENSE.snappy
> >
> >
> > $ repoquery -il librdkafka1
> > Name: librdkafka1
> > Version : 0.8.5
> > Release : 0
> > Architecture: x86_64
> > Size: 280660
> > Packager: None
> > Group   : Development/Libraries/C and C++
> > URL : https://github.com/edenhill/librdkafka
> > Repository  : rsyslog_v8
> > Summary : The Apache Kafka C library
> > Source  : librdkafka-0.8.5-0.src.rpm
> > Description :
> > librdkafka is a C/C++ library implementation of 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Sorry if I am dense, but why did you install librdkafka1?

Maybe I really don't understand anything at all, then so be it. I just
prefer to know why things happen.

I understand that there should be no kafka package at all in our repo. That
I always requested. I understand that two conflicting packages are bad. But
I do not understand why there is a problem if the packages are differently
named and only one of them is requested by other packages.

Or does yum search all packages and checks for conflicts even if the
conflicting ones are never to be installed together? That would explain the
issue to me.

If that's the case we need to reconsider providing packages for components
that need addtl requirements and so I think it is an important issue to
clean up. It makes me feel very uncomfortable that we cannot reproduce the
issue.

Rainer

Sent from phone, thus brief.

Derek DiFilippo  schrieb am Do., 19. Apr. 2018, 21:06:

> Hi everyone,
>
> The situation is exactly as described in my original email. We had a
> combination of packages and dependencies that led to the yum conflict. It's
> there, it's hard to get at, but we did it.  The solution was to build from
> source and link statically because we can't play around with random
> packages from your repository.
>
> To illustrate this I built a test package that requires librdkafka. On a
> clean AWS system if I happen to install librdkafka1 from the rsyslog repo
> (say something, somewhere, somehow, did this in the past), my test package
> installs happily... but, unbeknownst to me I am pinned to the *old* version
> of librdkafka and not the most recent one from epel that I was expecting to
> use in my product. Sure I could add some paranoid workarounds -- but for
> all of my other packages I am assuming there aren't magical packages out
> there spoofing the ones I'm expecting from epel. How paranoid should I get?
>
> There's no way having librdkafka1 in your repo is correct behaviour. In
> principle you should remove it and let epel provide it.
>
> In practice, if no one cares, so be it.
>
> Try this: what would stop you from having a "libstdc++1" package in your
> public rsyslog repo that fulfills all of the dependencies provided by the
> RHEL7 ibstdc++. Would that make sense to you? Would that be a "conflict
> with other software that I'm running"? I would argue it's a lack of
> understanding by whomever put the "libstdc++1" package up on the rsyslog
> repo.
>
> 
>
> To illustrate this a bit more, try the following on a RHEL system. And as a
> mental exercise replace "librdkafka" with "libstdc++" to get a sense of
> what I mean.
>
> $ repoquery -il librdkafka
> Name: librdkafka
> Version : 0.11.3
> Release : 1.el7
> Architecture: x86_64
> Size: 1020930
> Packager: Fedora Project
> Group   : Development/Libraries
> URL : https://github.com/edenhill/librdkafka
> Repository  : epel
> Summary : The Apache Kafka C library
> Source  : librdkafka-0.11.3-1.el7.src.rpm
> Description :
> Librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
> containing both Producer and Consumer support.
> It was designed with message delivery reliability and high performance in
> mind,
> current figures exceed 80 messages/second for the producer and 3
> million
> messages/second for the consumer.
> /usr/lib64/librdkafka++.so.1
> /usr/lib64/librdkafka.so.1
> /usr/share/doc/librdkafka-0.11.3
> /usr/share/doc/librdkafka-0.11.3/CONFIGURATION.md
> /usr/share/doc/librdkafka-0.11.3/INTRODUCTION.md
> /usr/share/doc/librdkafka-0.11.3/README.md
> /usr/share/licenses/librdkafka-0.11.3
> /usr/share/licenses/librdkafka-0.11.3/LICENSE
> /usr/share/licenses/librdkafka-0.11.3/LICENSE.pycrc
> /usr/share/licenses/librdkafka-0.11.3/LICENSE.snappy
>
>
> $ repoquery -il librdkafka1
> Name: librdkafka1
> Version : 0.8.5
> Release : 0
> Architecture: x86_64
> Size: 280660
> Packager: None
> Group   : Development/Libraries/C and C++
> URL : https://github.com/edenhill/librdkafka
> Repository  : rsyslog_v8
> Summary : The Apache Kafka C library
> Source  : librdkafka-0.8.5-0.src.rpm
> Description :
> librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
> containing both Producer and Consumer support.
> /usr/lib64/librdkafka++.so.1
> /usr/lib64/librdkafka.so.1
> /usr/share/doc/librdkafka1-0.8.5
> /usr/share/doc/librdkafka1-0.8.5/CONFIGURATION.md
> /usr/share/doc/librdkafka1-0.8.5/INTRODUCTION.md
> /usr/share/doc/librdkafka1-0.8.5/LICENSE
> /usr/share/doc/librdkafka1-0.8.5/LICENSE.pycrc
> /usr/share/doc/librdkafka1-0.8.5/LICENSE.snappy
> /usr/share/doc/librdkafka1-0.8.5/README.md
>
> On Thu, Apr 19, 2018 at 11:28 AM, Rainer Gerhards <
> rgerha...@hq.adiscon.com>
> wrote:
>
> > But the package names are different, so that other app's package should
> > depend on the other kafka package name, so nobody should drag in ours as
> > that name is 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi everyone,

The situation is exactly as described in my original email. We had a
combination of packages and dependencies that led to the yum conflict. It's
there, it's hard to get at, but we did it.  The solution was to build from
source and link statically because we can't play around with random
packages from your repository.

To illustrate this I built a test package that requires librdkafka. On a
clean AWS system if I happen to install librdkafka1 from the rsyslog repo
(say something, somewhere, somehow, did this in the past), my test package
installs happily... but, unbeknownst to me I am pinned to the *old* version
of librdkafka and not the most recent one from epel that I was expecting to
use in my product. Sure I could add some paranoid workarounds -- but for
all of my other packages I am assuming there aren't magical packages out
there spoofing the ones I'm expecting from epel. How paranoid should I get?

There's no way having librdkafka1 in your repo is correct behaviour. In
principle you should remove it and let epel provide it.

In practice, if no one cares, so be it.

Try this: what would stop you from having a "libstdc++1" package in your
public rsyslog repo that fulfills all of the dependencies provided by the
RHEL7 ibstdc++. Would that make sense to you? Would that be a "conflict
with other software that I'm running"? I would argue it's a lack of
understanding by whomever put the "libstdc++1" package up on the rsyslog
repo.



To illustrate this a bit more, try the following on a RHEL system. And as a
mental exercise replace "librdkafka" with "libstdc++" to get a sense of
what I mean.

$ repoquery -il librdkafka
Name: librdkafka
Version : 0.11.3
Release : 1.el7
Architecture: x86_64
Size: 1020930
Packager: Fedora Project
Group   : Development/Libraries
URL : https://github.com/edenhill/librdkafka
Repository  : epel
Summary : The Apache Kafka C library
Source  : librdkafka-0.11.3-1.el7.src.rpm
Description :
Librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
containing both Producer and Consumer support.
It was designed with message delivery reliability and high performance in
mind,
current figures exceed 80 messages/second for the producer and 3 million
messages/second for the consumer.
/usr/lib64/librdkafka++.so.1
/usr/lib64/librdkafka.so.1
/usr/share/doc/librdkafka-0.11.3
/usr/share/doc/librdkafka-0.11.3/CONFIGURATION.md
/usr/share/doc/librdkafka-0.11.3/INTRODUCTION.md
/usr/share/doc/librdkafka-0.11.3/README.md
/usr/share/licenses/librdkafka-0.11.3
/usr/share/licenses/librdkafka-0.11.3/LICENSE
/usr/share/licenses/librdkafka-0.11.3/LICENSE.pycrc
/usr/share/licenses/librdkafka-0.11.3/LICENSE.snappy


$ repoquery -il librdkafka1
Name: librdkafka1
Version : 0.8.5
Release : 0
Architecture: x86_64
Size: 280660
Packager: None
Group   : Development/Libraries/C and C++
URL : https://github.com/edenhill/librdkafka
Repository  : rsyslog_v8
Summary : The Apache Kafka C library
Source  : librdkafka-0.8.5-0.src.rpm
Description :
librdkafka is a C/C++ library implementation of the Apache Kafka protocol,
containing both Producer and Consumer support.
/usr/lib64/librdkafka++.so.1
/usr/lib64/librdkafka.so.1
/usr/share/doc/librdkafka1-0.8.5
/usr/share/doc/librdkafka1-0.8.5/CONFIGURATION.md
/usr/share/doc/librdkafka1-0.8.5/INTRODUCTION.md
/usr/share/doc/librdkafka1-0.8.5/LICENSE
/usr/share/doc/librdkafka1-0.8.5/LICENSE.pycrc
/usr/share/doc/librdkafka1-0.8.5/LICENSE.snappy
/usr/share/doc/librdkafka1-0.8.5/README.md

On Thu, Apr 19, 2018 at 11:28 AM, Rainer Gerhards 
wrote:

> But the package names are different, so that other app's package should
> depend on the other kafka package name, so nobody should drag in ours as
> that name is never requested. That's at least my weak understanding ;-)
>
> Rainer
>
> Sent from phone, thus brief.
>
> David Lang  schrieb am Do., 19. Apr. 2018, 20:14:
>
> > The conflict is that he has other software on his system that requires a
> > different version of librdkafka, and the version that gets pulled in from
> > the
> > adiscon repo is wrong for his other software.
> >
> > It's not a conflict with anything that rsyslog uses, it's a conflict with
> > the
> > other software he's running.
> > ___
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> >
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow 

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

But the package names are different, so that other app's package should
depend on the other kafka package name, so nobody should drag in ours as
that name is never requested. That's at least my weak understanding ;-)

Rainer

Sent from phone, thus brief.

David Lang  schrieb am Do., 19. Apr. 2018, 20:14:

> The conflict is that he has other software on his system that requires a
> different version of librdkafka, and the version that gets pulled in from
> the
> adiscon repo is wrong for his other software.
>
> It's not a conflict with anything that rsyslog uses, it's a conflict with
> the
> other software he's running.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[David Lang]

The conflict is that he has other software on his system that requires a 
different version of librdkafka, and the version that gets pulled in from the 
adiscon repo is wrong for his other software.


It's not a conflict with anything that rsyslog uses, it's a conflict with the 
other software he's running.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Florian Riedl]

ibrdkafka1 from your repo
>> > > ... and there's a conflict as shown in my original email in this
>> thread.
>> > >
>> > > There's no opportunity to "delete" the librdkafka1 package from the
>> > rsyslog
>> > > repo.
>> > > Maybe there's some yum-foo I am unaware of (would priority fix this?)
>> to
>> > > resolve the conflict?
>> > > But I think that misses the point and is a workaround to be avoided.
>> > >
>> > > Note: we will end up building librdkafka from source and linking
>> > statically
>> > > as a CMake external project to reduce our dependency pressure.
>> > > As a result, I didn't want to linger on this. But since we're here ---
>> I
>> > > don't understand Florian's "it won't work if we remove it from the
>> > rsyslog
>> > > repo" reply.
>> > > And there's no "deleting" a package that never gets installed because
>> of
>> > a
>> > > conflict.
>> > >
>> > > Based on the rsyslog repo (it says epel-7 in the path...), why
> wouldn't
>> > you
>> > > use the librdkafka from epel as we do?
>> > > If you need to pin a version you should specify that when you build
> the
>> > > rpm, not by putting that particular package in your repo.
>> > >
>> > > I'm sure everyone *knows* this (not trying to be patronizing) so --
> why
>> > is
>> > > librdkafka1 there?
>> > > Maybe the older rsyslog rpms were built without specifying a
> librdkafka
>> > > version and newer versions of librdkafka broke older rsyslog releases?
>> > >
>> > > Thanks everyone for reading through all of this!
>> > >
>> > > -Derek.
>> > >
>> > > On Wed, Apr 18, 2018 at 10:52 AM, Rainer Gerhards <
>> > > rgerha...@hq.adiscon.com>
>> > > wrote:
>> > >
>> > > > Derek,
>> > > >
>> > > > I don't understand the issue here: do you intend to keep an outdated
>> > > > rsyslog version? With the current ones, no issue should exist, and
> if
>> > it
>> > > > does, we should fix it.
>> > > >
>> > > > So I am very interested to understand your issue.
>> > > >
>> > > > Rainer
>> > > >
>> > > > Sent from phone, thus brief.
>> > > >
>> > > > Derek DiFilippo <de...@jsonar.com> schrieb am Mi., 18. Apr. 2018,
>> > 19:24:
>> > > >
>> > > > > Hi Florian,
>> > > > >
>> > > > > Yes, that was a bit aggressive/silly of me to ask for it to be
>> > > removed...
>> > > > >
>> > > > > We're going to build from source and link statically to remove the
>> > > > > dependency/conflict. Deleting the rpm from the affected machine
>> won't
>> > > fix
>> > > > > the install conflict.
>> > > > >
>> > > > > Thanks everyone for the rapid replies, much appreciated.
>> > > > >
>> > > > > best,
>> > > > > -Derek
>> > > > >
>> > > > > On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Derek, we cannot simply delete this from the repository. It is
>> > still
>> > > > > > needed for older versions or they won't be usable anymore.
>> > > > > >
>> > > > > > But, it should be sufficient to delete the librdkafka1 RPM from
>> the
>> > > > > > affected machine manually.
>> > > > > >
>> > > > > > Florian
>> > > > > >
>> > > > > > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
>> > > > > > > If it is statically linked, can librdkafka1 be removed from
> the
>> > > > > > repository?
>> > > > > > > It's quite an old version of the library. It should be
>> sufficient
>> > > to
>> > > > > use
>> > > > > > > the one from epel, correct?
>> > > > > > >
>> > > > > > > That would fix our conflict.
>> > > > > > >
>> > &

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

gt; version and newer versions of librdkafka broke older rsyslog releases?
> > >
> > > Thanks everyone for reading through all of this!
> > >
> > > -Derek.
> > >
> > > On Wed, Apr 18, 2018 at 10:52 AM, Rainer Gerhards <
> > > rgerha...@hq.adiscon.com>
> > > wrote:
> > >
> > > > Derek,
> > > >
> > > > I don't understand the issue here: do you intend to keep an outdated
> > > > rsyslog version? With the current ones, no issue should exist, and
if
> > it
> > > > does, we should fix it.
> > > >
> > > > So I am very interested to understand your issue.
> > > >
> > > > Rainer
> > > >
> > > > Sent from phone, thus brief.
> > > >
> > > > Derek DiFilippo <de...@jsonar.com> schrieb am Mi., 18. Apr. 2018,
> > 19:24:
> > > >
> > > > > Hi Florian,
> > > > >
> > > > > Yes, that was a bit aggressive/silly of me to ask for it to be
> > > removed...
> > > > >
> > > > > We're going to build from source and link statically to remove the
> > > > > dependency/conflict. Deleting the rpm from the affected machine
> won't
> > > fix
> > > > > the install conflict.
> > > > >
> > > > > Thanks everyone for the rapid replies, much appreciated.
> > > > >
> > > > > best,
> > > > > -Derek
> > > > >
> > > > > On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com
> >
> > > > wrote:
> > > > >
> > > > > > Derek, we cannot simply delete this from the repository. It is
> > still
> > > > > > needed for older versions or they won't be usable anymore.
> > > > > >
> > > > > > But, it should be sufficient to delete the librdkafka1 RPM from
> the
> > > > > > affected machine manually.
> > > > > >
> > > > > > Florian
> > > > > >
> > > > > > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > > > > > > If it is statically linked, can librdkafka1 be removed from
the
> > > > > > repository?
> > > > > > > It's quite an old version of the library. It should be
> sufficient
> > > to
> > > > > use
> > > > > > > the one from epel, correct?
> > > > > > >
> > > > > > > That would fix our conflict.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > -D.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <
> alorb...@adiscon.com>
> > > > > wrote:
> > > > > > >
> > > > > > >> It should be statically linked.
> > > > > > >> Maybe the dependencies are not completely removed everywhere?
> > > > > > >>
> > > > > > >> Florian can you check please if the package scripts are using
> > > > > > >> --enable-kafka-static properly?
> > > > > > >>
> > > > > > >> The adisconbuild-librdkafka packages are used for building
> only,
> > > and
> > > > > we
> > > > > > >> need them to get the static builds running on Launchpad!
> > > > > > >>
> > > > > > >> Best regards,
> > > > > > >> Andre Lorbach
> > > > > > >>
> > > > > > >> > -Original Message-
> > > > > > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On
> > > > Behalf
> > > > > Of
> > > > > > >> > Rainer Gerhards
> > > > > > >> > Sent: Monday, April 16, 2018 8:26 PM
> > > > > > >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > > > > >> > Subject: Re: [rsyslog] librdkafka packages in
> > > > > > >> v8-stable/epel-7/x86_64/RPMS
> > > > > > >> >
> > > > > > >> > Mmm... I was told that we link statically to
librdkafka,
> > so
> > > > > there
> > > > > > >> should
> > > > > > >> > no kaf

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

> > > > So I am very interested to understand your issue.
> > > >
> > > > Rainer
> > > >
> > > > Sent from phone, thus brief.
> > > >
> > > > Derek DiFilippo <de...@jsonar.com> schrieb am Mi., 18. Apr. 2018,
> > 19:24:
> > > >
> > > > > Hi Florian,
> > > > >
> > > > > Yes, that was a bit aggressive/silly of me to ask for it to be
> > > removed...
> > > > >
> > > > > We're going to build from source and link statically to remove the
> > > > > dependency/conflict. Deleting the rpm from the affected machine
> won't
> > > fix
> > > > > the install conflict.
> > > > >
> > > > > Thanks everyone for the rapid replies, much appreciated.
> > > > >
> > > > > best,
> > > > > -Derek
> > > > >
> > > > > On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com
> >
> > > > wrote:
> > > > >
> > > > > > Derek, we cannot simply delete this from the repository. It is
> > still
> > > > > > needed for older versions or they won't be usable anymore.
> > > > > >
> > > > > > But, it should be sufficient to delete the librdkafka1 RPM from
> the
> > > > > > affected machine manually.
> > > > > >
> > > > > > Florian
> > > > > >
> > > > > > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > > > > > > If it is statically linked, can librdkafka1 be removed from the
> > > > > > repository?
> > > > > > > It's quite an old version of the library. It should be
> sufficient
> > > to
> > > > > use
> > > > > > > the one from epel, correct?
> > > > > > >
> > > > > > > That would fix our conflict.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > -D.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <
> alorb...@adiscon.com>
> > > > > wrote:
> > > > > > >
> > > > > > >> It should be statically linked.
> > > > > > >> Maybe the dependencies are not completely removed everywhere?
> > > > > > >>
> > > > > > >> Florian can you check please if the package scripts are using
> > > > > > >> --enable-kafka-static properly?
> > > > > > >>
> > > > > > >> The adisconbuild-librdkafka packages are used for building
> only,
> > > and
> > > > > we
> > > > > > >> need them to get the static builds running on Launchpad!
> > > > > > >>
> > > > > > >> Best regards,
> > > > > > >> Andre Lorbach
> > > > > > >>
> > > > > > >> > -Original Message-
> > > > > > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On
> > > > Behalf
> > > > > Of
> > > > > > >> > Rainer Gerhards
> > > > > > >> > Sent: Monday, April 16, 2018 8:26 PM
> > > > > > >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > > > > >> > Subject: Re: [rsyslog] librdkafka packages in
> > > > > > >> v8-stable/epel-7/x86_64/RPMS
> > > > > > >> >
> > > > > > >> > Mmm... I was told that we link statically to librdkafka,
> > so
> > > > > there
> > > > > > >> should
> > > > > > >> > no kafka package at all be installed (actually this was
> what I
> > > > asked
> > > > > > >> for).
> > > > > > >> >
> > > > > > >> > @florian, @andre can you comment?
> > > > > > >> >
> > > > > > >> > Rainer
> > > > > > >> >
> > > > > > >> > Sent from phone, thus brief.
> > > > > > >> >
> > > > > > >> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr.
> > > 2018,
> > >

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

t; > > On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com>
> > > wrote:
> > > >
> > > > > Derek, we cannot simply delete this from the repository. It is
> still
> > > > > needed for older versions or they won't be usable anymore.
> > > > >
> > > > > But, it should be sufficient to delete the librdkafka1 RPM from the
> > > > > affected machine manually.
> > > > >
> > > > > Florian
> > > > >
> > > > > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > > > > > If it is statically linked, can librdkafka1 be removed from the
> > > > > repository?
> > > > > > It's quite an old version of the library. It should be sufficient
> > to
> > > > use
> > > > > > the one from epel, correct?
> > > > > >
> > > > > > That would fix our conflict.
> > > > > >
> > > > > > Thanks,
> > > > > > -D.
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com>
> > > > wrote:
> > > > > >
> > > > > >> It should be statically linked.
> > > > > >> Maybe the dependencies are not completely removed everywhere?
> > > > > >>
> > > > > >> Florian can you check please if the package scripts are using
> > > > > >> --enable-kafka-static properly?
> > > > > >>
> > > > > >> The adisconbuild-librdkafka packages are used for building only,
> > and
> > > > we
> > > > > >> need them to get the static builds running on Launchpad!
> > > > > >>
> > > > > >> Best regards,
> > > > > >> Andre Lorbach
> > > > > >>
> > > > > >> > -Original Message-
> > > > > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On
> > > Behalf
> > > > Of
> > > > > >> > Rainer Gerhards
> > > > > >> > Sent: Monday, April 16, 2018 8:26 PM
> > > > > >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > > > >> > Subject: Re: [rsyslog] librdkafka packages in
> > > > > >> v8-stable/epel-7/x86_64/RPMS
> > > > > >> >
> > > > > >> > Mmm... I was told that we link statically to librdkafka,
> so
> > > > there
> > > > > >> should
> > > > > >> > no kafka package at all be installed (actually this was what I
> > > asked
> > > > > >> for).
> > > > > >> >
> > > > > >> > @florian, @andre can you comment?
> > > > > >> >
> > > > > >> > Rainer
> > > > > >> >
> > > > > >> > Sent from phone, thus brief.
> > > > > >> >
> > > > > >> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr.
> > 2018,
> > > > > 20:19:
> > > > > >> >
> > > > > >> > > Hi Rainer, hi everyone,
> > > > > >> > >
> > > > > >> > > In addition to a dependency on rsyslog, one of our packages
> > now
> > > > > >> > > depends on librdkafka and librdkafka-devel.
> > > > > >> > >
> > > > > >> > > We want to use the librdkafka packages provided by epel,
> > nothing
> > > > > >> fancy.
> > > > > >> > >
> > > > > >> > > I'm seeing a conflict between the librdkafka1 package in the
> > > > rsyslog
> > > > > >> > > repo (note the "1" at the end of the library name) and the
> > ones
> > > > > >> > > provided by epel.
> > > > > >> > >
> > > > > >> > > ...
> > > > > >> > > ...
> > > > > >> > >  librdkafka  x86_64 0.11.3-1.el7   epel
> > > > > >> > > 326 k
> > > > > >> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > > > > >> > >  43 k
> > > > > >> &g

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

; > > > >
> > > > >
> > > > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com>
> > > wrote:
> > > > >
> > > > >> It should be statically linked.
> > > > >> Maybe the dependencies are not completely removed everywhere?
> > > > >>
> > > > >> Florian can you check please if the package scripts are using
> > > > >> --enable-kafka-static properly?
> > > > >>
> > > > >> The adisconbuild-librdkafka packages are used for building only,
> and
> > > we
> > > > >> need them to get the static builds running on Launchpad!
> > > > >>
> > > > >> Best regards,
> > > > >> Andre Lorbach
> > > > >>
> > > > >> > -Original Message-
> > > > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On
> > Behalf
> > > Of
> > > > >> > Rainer Gerhards
> > > > >> > Sent: Monday, April 16, 2018 8:26 PM
> > > > >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > > >> > Subject: Re: [rsyslog] librdkafka packages in
> > > > >> v8-stable/epel-7/x86_64/RPMS
> > > > >> >
> > > > >> > Mmm... I was told that we link statically to librdkafka, so
> > > there
> > > > >> should
> > > > >> > no kafka package at all be installed (actually this was what I
> > asked
> > > > >> for).
> > > > >> >
> > > > >> > @florian, @andre can you comment?
> > > > >> >
> > > > >> > Rainer
> > > > >> >
> > > > >> > Sent from phone, thus brief.
> > > > >> >
> > > > >> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr.
> 2018,
> > > > 20:19:
> > > > >> >
> > > > >> > > Hi Rainer, hi everyone,
> > > > >> > >
> > > > >> > > In addition to a dependency on rsyslog, one of our packages
> now
> > > > >> > > depends on librdkafka and librdkafka-devel.
> > > > >> > >
> > > > >> > > We want to use the librdkafka packages provided by epel,
> nothing
> > > > >> fancy.
> > > > >> > >
> > > > >> > > I'm seeing a conflict between the librdkafka1 package in the
> > > rsyslog
> > > > >> > > repo (note the "1" at the end of the library name) and the
> ones
> > > > >> > > provided by epel.
> > > > >> > >
> > > > >> > > ...
> > > > >> > > ...
> > > > >> > >  librdkafka  x86_64 0.11.3-1.el7   epel
> > > > >> > > 326 k
> > > > >> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > > > >> > >  43 k
> > > > >> > >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > > > >> > >   100 k
> > > > >> > > ...
> > > > >> > > ...
> > > > >> > >
> > > > >> > > Transaction check error:
> > > > >> > >   file /usr/lib64/librdkafka++.so.1 conflicts between
> attempted
> > > > >> > > installs of
> > > > >> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > > > >> > >   file /usr/lib64/librdkafka.so.1 conflicts between attempted
> > > > installs
> > > > >> > > of
> > > > >> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > > > >> > >
> > > > >> > > Can librdkafka1 be removed from the rsyslog repo? What is the
> > > > >> > > rationale for pinning a version of librdkafka via the
> > librdkafka1
> > > > >> package?
> > > > >> > >
> > > > >> > > If librdkafka1 must stay in the rsyslog repo what do you
> > recommend
> > > > as
> > > > >> > > the best practice for resolving this conflict?
> > > > >> > >
> > > > >> > > I see a lot of "adisconbuild-librdkafka*" packages

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Rainier,

On the contrary, we do not pin our rsyslog version. We adopt your releases
as soon as they are available (subject to our internal CI system).

What's changed here is that one of our products recently brought in
librdkafka as a dynamic dependency.

$ rpm -qpR sonarw.2.2-gabcd123.jsonar.x86_64 | grep librdkafka ## older
version sonarw 2.2
#  nothing
$ rpm -qpR sonarw.2.3-g1234abc.jsonar.x86_64 | grep librdkafka ## latest
version sonarw 2.3
librdkafka++.so.1()(64bit)
librdkafka.so.1()(64bit)

Before we do a yum install, we add the rsyslog repo (and ours, and others)
to the /etc/yum.repos.d on the target machine.
When yum installs our top level package suite:
+ sonarw brings in librdkafka from epel
+ rsyslog brings in librdkafka1 from your repo
... and there's a conflict as shown in my original email in this thread.

There's no opportunity to "delete" the librdkafka1 package from the rsyslog
repo.
Maybe there's some yum-foo I am unaware of (would priority fix this?) to
resolve the conflict?
But I think that misses the point and is a workaround to be avoided.

Note: we will end up building librdkafka from source and linking statically
as a CMake external project to reduce our dependency pressure.
As a result, I didn't want to linger on this. But since we're here --- I
don't understand Florian's "it won't work if we remove it from the rsyslog
repo" reply.
And there's no "deleting" a package that never gets installed because of a
conflict.

Based on the rsyslog repo (it says epel-7 in the path...), why wouldn't you
use the librdkafka from epel as we do?
If you need to pin a version you should specify that when you build the
rpm, not by putting that particular package in your repo.

I'm sure everyone *knows* this (not trying to be patronizing) so -- why is
librdkafka1 there?
Maybe the older rsyslog rpms were built without specifying a librdkafka
version and newer versions of librdkafka broke older rsyslog releases?

Thanks everyone for reading through all of this!

-Derek.

On Wed, Apr 18, 2018 at 10:52 AM, Rainer Gerhards <rgerha...@hq.adiscon.com>
wrote:

> Derek,
>
> I don't understand the issue here: do you intend to keep an outdated
> rsyslog version? With the current ones, no issue should exist, and if it
> does, we should fix it.
>
> So I am very interested to understand your issue.
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo <de...@jsonar.com> schrieb am Mi., 18. Apr. 2018, 19:24:
>
> > Hi Florian,
> >
> > Yes, that was a bit aggressive/silly of me to ask for it to be removed...
> >
> > We're going to build from source and link statically to remove the
> > dependency/conflict. Deleting the rpm from the affected machine won't fix
> > the install conflict.
> >
> > Thanks everyone for the rapid replies, much appreciated.
> >
> > best,
> > -Derek
> >
> > On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com>
> wrote:
> >
> > > Derek, we cannot simply delete this from the repository. It is still
> > > needed for older versions or they won't be usable anymore.
> > >
> > > But, it should be sufficient to delete the librdkafka1 RPM from the
> > > affected machine manually.
> > >
> > > Florian
> > >
> > > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > > > If it is statically linked, can librdkafka1 be removed from the
> > > repository?
> > > > It's quite an old version of the library. It should be sufficient to
> > use
> > > > the one from epel, correct?
> > > >
> > > > That would fix our conflict.
> > > >
> > > > Thanks,
> > > > -D.
> > > >
> > > >
> > > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com>
> > wrote:
> > > >
> > > >> It should be statically linked.
> > > >> Maybe the dependencies are not completely removed everywhere?
> > > >>
> > > >> Florian can you check please if the package scripts are using
> > > >> --enable-kafka-static properly?
> > > >>
> > > >> The adisconbuild-librdkafka packages are used for building only, and
> > we
> > > >> need them to get the static builds running on Launchpad!
> > > >>
> > > >> Best regards,
> > > >> Andre Lorbach
> > > >>
> > > >> > -Original Message-
> > > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On
> Behalf
> > Of
> > > >> > Rainer Gerhards
&g

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Derek,

I don't understand the issue here: do you intend to keep an outdated
rsyslog version? With the current ones, no issue should exist, and if it
does, we should fix it.

So I am very interested to understand your issue.

Rainer

Sent from phone, thus brief.

Derek DiFilippo <de...@jsonar.com> schrieb am Mi., 18. Apr. 2018, 19:24:

> Hi Florian,
>
> Yes, that was a bit aggressive/silly of me to ask for it to be removed...
>
> We're going to build from source and link statically to remove the
> dependency/conflict. Deleting the rpm from the affected machine won't fix
> the install conflict.
>
> Thanks everyone for the rapid replies, much appreciated.
>
> best,
> -Derek
>
> On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com> wrote:
>
> > Derek, we cannot simply delete this from the repository. It is still
> > needed for older versions or they won't be usable anymore.
> >
> > But, it should be sufficient to delete the librdkafka1 RPM from the
> > affected machine manually.
> >
> > Florian
> >
> > 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > > If it is statically linked, can librdkafka1 be removed from the
> > repository?
> > > It's quite an old version of the library. It should be sufficient to
> use
> > > the one from epel, correct?
> > >
> > > That would fix our conflict.
> > >
> > > Thanks,
> > > -D.
> > >
> > >
> > > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com>
> wrote:
> > >
> > >> It should be statically linked.
> > >> Maybe the dependencies are not completely removed everywhere?
> > >>
> > >> Florian can you check please if the package scripts are using
> > >> --enable-kafka-static properly?
> > >>
> > >> The adisconbuild-librdkafka packages are used for building only, and
> we
> > >> need them to get the static builds running on Launchpad!
> > >>
> > >> Best regards,
> > >> Andre Lorbach
> > >>
> > >> > -Original Message-
> > >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf
> Of
> > >> > Rainer Gerhards
> > >> > Sent: Monday, April 16, 2018 8:26 PM
> > >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > >> > Subject: Re: [rsyslog] librdkafka packages in
> > >> v8-stable/epel-7/x86_64/RPMS
> > >> >
> > >> > Mmm... I was told that we link statically to librdkafka, so
> there
> > >> should
> > >> > no kafka package at all be installed (actually this was what I asked
> > >> for).
> > >> >
> > >> > @florian, @andre can you comment?
> > >> >
> > >> > Rainer
> > >> >
> > >> > Sent from phone, thus brief.
> > >> >
> > >> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018,
> > 20:19:
> > >> >
> > >> > > Hi Rainer, hi everyone,
> > >> > >
> > >> > > In addition to a dependency on rsyslog, one of our packages now
> > >> > > depends on librdkafka and librdkafka-devel.
> > >> > >
> > >> > > We want to use the librdkafka packages provided by epel, nothing
> > >> fancy.
> > >> > >
> > >> > > I'm seeing a conflict between the librdkafka1 package in the
> rsyslog
> > >> > > repo (note the "1" at the end of the library name) and the ones
> > >> > > provided by epel.
> > >> > >
> > >> > > ...
> > >> > > ...
> > >> > >  librdkafka  x86_64 0.11.3-1.el7   epel
> > >> > > 326 k
> > >> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > >> > >  43 k
> > >> > >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > >> > >   100 k
> > >> > > ...
> > >> > > ...
> > >> > >
> > >> > > Transaction check error:
> > >> > >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> > >> > > installs of
> > >> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > >> > >   file /usr/lib64/librdkafka.so.1 conflicts between attempted
> > installs
> >

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Florian,

Yes, that was a bit aggressive/silly of me to ask for it to be removed...

We're going to build from source and link statically to remove the
dependency/conflict. Deleting the rpm from the affected machine won't fix
the install conflict.

Thanks everyone for the rapid replies, much appreciated.

best,
-Derek

On Wed, Apr 18, 2018 at 8:59 AM, Florian Riedl <fri...@adiscon.com> wrote:

> Derek, we cannot simply delete this from the repository. It is still
> needed for older versions or they won't be usable anymore.
>
> But, it should be sufficient to delete the librdkafka1 RPM from the
> affected machine manually.
>
> Florian
>
> 2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> > If it is statically linked, can librdkafka1 be removed from the
> repository?
> > It's quite an old version of the library. It should be sufficient to use
> > the one from epel, correct?
> >
> > That would fix our conflict.
> >
> > Thanks,
> > -D.
> >
> >
> > On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com> wrote:
> >
> >> It should be statically linked.
> >> Maybe the dependencies are not completely removed everywhere?
> >>
> >> Florian can you check please if the package scripts are using
> >> --enable-kafka-static properly?
> >>
> >> The adisconbuild-librdkafka packages are used for building only, and we
> >> need them to get the static builds running on Launchpad!
> >>
> >> Best regards,
> >> Andre Lorbach
> >>
> >> > -Original Message-----
> >> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of
> >> > Rainer Gerhards
> >> > Sent: Monday, April 16, 2018 8:26 PM
> >> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> >> > Subject: Re: [rsyslog] librdkafka packages in
> >> v8-stable/epel-7/x86_64/RPMS
> >> >
> >> > Mmm... I was told that we link statically to librdkafka, so there
> >> should
> >> > no kafka package at all be installed (actually this was what I asked
> >> for).
> >> >
> >> > @florian, @andre can you comment?
> >> >
> >> > Rainer
> >> >
> >> > Sent from phone, thus brief.
> >> >
> >> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018,
> 20:19:
> >> >
> >> > > Hi Rainer, hi everyone,
> >> > >
> >> > > In addition to a dependency on rsyslog, one of our packages now
> >> > > depends on librdkafka and librdkafka-devel.
> >> > >
> >> > > We want to use the librdkafka packages provided by epel, nothing
> >> fancy.
> >> > >
> >> > > I'm seeing a conflict between the librdkafka1 package in the rsyslog
> >> > > repo (note the "1" at the end of the library name) and the ones
> >> > > provided by epel.
> >> > >
> >> > > ...
> >> > > ...
> >> > >  librdkafka  x86_64 0.11.3-1.el7   epel
> >> > > 326 k
> >> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> >> > >  43 k
> >> > >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> >> > >   100 k
> >> > > ...
> >> > > ...
> >> > >
> >> > > Transaction check error:
> >> > >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> >> > > installs of
> >> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >> > >   file /usr/lib64/librdkafka.so.1 conflicts between attempted
> installs
> >> > > of
> >> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >> > >
> >> > > Can librdkafka1 be removed from the rsyslog repo? What is the
> >> > > rationale for pinning a version of librdkafka via the librdkafka1
> >> package?
> >> > >
> >> > > If librdkafka1 must stay in the rsyslog repo what do you recommend
> as
> >> > > the best practice for resolving this conflict?
> >> > >
> >> > > I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
> >> > > are they there? Is the solution to change the dependency in *our*
> >> > > package form librdkafka to adiscon-librdkafka? I might be
> comfortable
> >> > > with that if I know why

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Florian Riedl]

Derek, we cannot simply delete this from the repository. It is still
needed for older versions or they won't be usable anymore.

But, it should be sufficient to delete the librdkafka1 RPM from the
affected machine manually.

Florian

2018-04-17 18:26 GMT+02:00 Derek DiFilippo <de...@jsonar.com>:
> If it is statically linked, can librdkafka1 be removed from the repository?
> It's quite an old version of the library. It should be sufficient to use
> the one from epel, correct?
>
> That would fix our conflict.
>
> Thanks,
> -D.
>
>
> On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com> wrote:
>
>> It should be statically linked.
>> Maybe the dependencies are not completely removed everywhere?
>>
>> Florian can you check please if the package scripts are using
>> --enable-kafka-static properly?
>>
>> The adisconbuild-librdkafka packages are used for building only, and we
>> need them to get the static builds running on Launchpad!
>>
>> Best regards,
>> Andre Lorbach
>>
>> > -Original Message-
>> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of
>> > Rainer Gerhards
>> > Sent: Monday, April 16, 2018 8:26 PM
>> > To: rsyslog-users <rsyslog@lists.adiscon.com>
>> > Subject: Re: [rsyslog] librdkafka packages in
>> v8-stable/epel-7/x86_64/RPMS
>> >
>> > Mmm... I was told that we link statically to librdkafka, so there
>> should
>> > no kafka package at all be installed (actually this was what I asked
>> for).
>> >
>> > @florian, @andre can you comment?
>> >
>> > Rainer
>> >
>> > Sent from phone, thus brief.
>> >
>> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018, 20:19:
>> >
>> > > Hi Rainer, hi everyone,
>> > >
>> > > In addition to a dependency on rsyslog, one of our packages now
>> > > depends on librdkafka and librdkafka-devel.
>> > >
>> > > We want to use the librdkafka packages provided by epel, nothing
>> fancy.
>> > >
>> > > I'm seeing a conflict between the librdkafka1 package in the rsyslog
>> > > repo (note the "1" at the end of the library name) and the ones
>> > > provided by epel.
>> > >
>> > > ...
>> > > ...
>> > >  librdkafka  x86_64 0.11.3-1.el7   epel
>> > > 326 k
>> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
>> > >  43 k
>> > >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
>> > >   100 k
>> > > ...
>> > > ...
>> > >
>> > > Transaction check error:
>> > >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
>> > > installs of
>> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>> > >   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
>> > > of
>> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>> > >
>> > > Can librdkafka1 be removed from the rsyslog repo? What is the
>> > > rationale for pinning a version of librdkafka via the librdkafka1
>> package?
>> > >
>> > > If librdkafka1 must stay in the rsyslog repo what do you recommend as
>> > > the best practice for resolving this conflict?
>> > >
>> > > I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
>> > > are they there? Is the solution to change the dependency in *our*
>> > > package form librdkafka to adiscon-librdkafka? I might be comfortable
>> > > with that if I know why the adiscon packages are there.
>> > >
>> > > As a fallback we could add librdkafka as an external project via CMake
>> > > and build from source but I wanted to check in here first before
>> > proceeding.
>> > >
>> > > Apologies if I missed something obvious. I did my best googling
>> > > due-diligence and couldn't find anyhing satisfactory.
>> > >
>> > > Thanks for your time & patience!
>> > > ___
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com/professional-services/
>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>> > > W

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

If it is statically linked, can librdkafka1 be removed from the repository?
It's quite an old version of the library. It should be sufficient to use
the one from epel, correct?

That would fix our conflict.

Thanks,
-D.


On Tue, Apr 17, 2018, 01:05 Andre Lorbach <alorb...@adiscon.com> wrote:

> It should be statically linked.
> Maybe the dependencies are not completely removed everywhere?
>
> Florian can you check please if the package scripts are using
> --enable-kafka-static properly?
>
> The adisconbuild-librdkafka packages are used for building only, and we
> need them to get the static builds running on Launchpad!
>
> Best regards,
> Andre Lorbach
>
> > -Original Message-
> > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of
> > Rainer Gerhards
> > Sent: Monday, April 16, 2018 8:26 PM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] librdkafka packages in
> v8-stable/epel-7/x86_64/RPMS
> >
> > Mmm... I was told that we link statically to librdkafka, so there
> should
> > no kafka package at all be installed (actually this was what I asked
> for).
> >
> > @florian, @andre can you comment?
> >
> > Rainer
> >
> > Sent from phone, thus brief.
> >
> > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018, 20:19:
> >
> > > Hi Rainer, hi everyone,
> > >
> > > In addition to a dependency on rsyslog, one of our packages now
> > > depends on librdkafka and librdkafka-devel.
> > >
> > > We want to use the librdkafka packages provided by epel, nothing
> fancy.
> > >
> > > I'm seeing a conflict between the librdkafka1 package in the rsyslog
> > > repo (note the "1" at the end of the library name) and the ones
> > > provided by epel.
> > >
> > > ...
> > > ...
> > >  librdkafka  x86_64 0.11.3-1.el7   epel
> > > 326 k
> > >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> > >  43 k
> > >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> > >   100 k
> > > ...
> > > ...
> > >
> > > Transaction check error:
> > >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> > > installs of
> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > >   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> > > of
> > > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> > >
> > > Can librdkafka1 be removed from the rsyslog repo? What is the
> > > rationale for pinning a version of librdkafka via the librdkafka1
> package?
> > >
> > > If librdkafka1 must stay in the rsyslog repo what do you recommend as
> > > the best practice for resolving this conflict?
> > >
> > > I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
> > > are they there? Is the solution to change the dependency in *our*
> > > package form librdkafka to adiscon-librdkafka? I might be comfortable
> > > with that if I know why the adiscon packages are there.
> > >
> > > As a fallback we could add librdkafka as an external project via CMake
> > > and build from source but I wanted to check in here first before
> > proceeding.
> > >
> > > Apologies if I missed something obvious. I did my best googling
> > > due-diligence and couldn't find anyhing satisfactory.
> > >
> > > Thanks for your time & patience!
> > > ___
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > > DON'T LIKE THAT.
> > >
> > ___
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
> > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
> beyond
> > our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Florian Riedl]

Hi Derek,

there is no requirement to install any of the librdkafka packages
available in the rsyslog repo (at least not for current rsyslog
versions). The adisconbuild-librdkafka RPMs are solely for building
rsyslog RPMs and are not to be installed for use.

The old librdkafka1 RPM is still in the repository for legacy purposes
in old rsyslog versions, if a user requires said old version with
Kafka enabled. At some point we tried building our own RPM for
providing rsyslog-kafka, but later changed the procedure. I assume you
were/are having an old version of rsyslog with rsyslog-kafka
installed, right? And now you are trying to use a newer librdkafka for
your dependency?

This whole workaround we do with adisconbuild-librdkafka and static
linking is only required, because a) there are no official librdkafka
RPMs for the most recent release that we can use for building rsyslog
and b) because we decided not to provide RPMs for public use for a
third party project.

If you are on a current version of rsyslog I advise:
- removing the old librdkafka1 RPM installed from the rsyslog repo.
- Install librdkafka from "official" RPMs or source tarball.
- NOT using adisconbuild-librdkafka for your dependency. We will not
support third party use in any case.

Florian

2018-04-17 10:05 GMT+02:00 Andre Lorbach <alorb...@adiscon.com>:
> It should be statically linked.
> Maybe the dependencies are not completely removed everywhere?
>
> Florian can you check please if the package scripts are using
> --enable-kafka-static properly?
>
> The adisconbuild-librdkafka packages are used for building only, and we
> need them to get the static builds running on Launchpad!
>
> Best regards,
> Andre Lorbach
>
>> -Original Message-
>> From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of
>> Rainer Gerhards
>> Sent: Monday, April 16, 2018 8:26 PM
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] librdkafka packages in
> v8-stable/epel-7/x86_64/RPMS
>>
>> Mmm... I was told that we link statically to librdkafka, so there
> should
>> no kafka package at all be installed (actually this was what I asked
> for).
>>
>> @florian, @andre can you comment?
>>
>> Rainer
>>
>> Sent from phone, thus brief.
>>
>> Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018, 20:19:
>>
>> > Hi Rainer, hi everyone,
>> >
>> > In addition to a dependency on rsyslog, one of our packages now
>> > depends on librdkafka and librdkafka-devel.
>> >
>> > We want to use the librdkafka packages provided by epel, nothing
> fancy.
>> >
>> > I'm seeing a conflict between the librdkafka1 package in the rsyslog
>> > repo (note the "1" at the end of the library name) and the ones
>> > provided by epel.
>> >
>> > ...
>> > ...
>> >  librdkafka  x86_64 0.11.3-1.el7   epel
>> > 326 k
>> >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
>> >  43 k
>> >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
>> >   100 k
>> > ...
>> > ...
>> >
>> > Transaction check error:
>> >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
>> > installs of
>> > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>> >   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
>> > of
>> > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>> >
>> > Can librdkafka1 be removed from the rsyslog repo? What is the
>> > rationale for pinning a version of librdkafka via the librdkafka1
> package?
>> >
>> > If librdkafka1 must stay in the rsyslog repo what do you recommend as
>> > the best practice for resolving this conflict?
>> >
>> > I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
>> > are they there? Is the solution to change the dependency in *our*
>> > package form librdkafka to adiscon-librdkafka? I might be comfortable
>> > with that if I know why the adiscon packages are there.
>> >
>> > As a fallback we could add librdkafka as an external project via CMake
>> > and build from source but I wanted to check in here first before
>> proceeding.
>> >
>> > Apologies if I missed something obvious. I did my best googling
>> > due-diligence and couldn't find anyhing satisfactory.
>> >
>> > Thanks for your time & patience!
>> > ___
>> > rsyslog mailing list
>> &g

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Andre Lorbach]

It should be statically linked.
Maybe the dependencies are not completely removed everywhere?

Florian can you check please if the package scripts are using
--enable-kafka-static properly?

The adisconbuild-librdkafka packages are used for building only, and we
need them to get the static builds running on Launchpad!

Best regards,
Andre Lorbach

> -Original Message-
> From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of
> Rainer Gerhards
> Sent: Monday, April 16, 2018 8:26 PM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] librdkafka packages in
v8-stable/epel-7/x86_64/RPMS
>
> Mmm... I was told that we link statically to librdkafka, so there
should
> no kafka package at all be installed (actually this was what I asked
for).
>
> @florian, @andre can you comment?
>
> Rainer
>
> Sent from phone, thus brief.
>
> Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. 2018, 20:19:
>
> > Hi Rainer, hi everyone,
> >
> > In addition to a dependency on rsyslog, one of our packages now
> > depends on librdkafka and librdkafka-devel.
> >
> > We want to use the librdkafka packages provided by epel, nothing
fancy.
> >
> > I'm seeing a conflict between the librdkafka1 package in the rsyslog
> > repo (note the "1" at the end of the library name) and the ones
> > provided by epel.
> >
> > ...
> > ...
> >  librdkafka  x86_64 0.11.3-1.el7   epel
> > 326 k
> >  librdkafka-devel  x86_64 0.11.3-1.el7   epel
> >  43 k
> >  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
> >   100 k
> > ...
> > ...
> >
> > Transaction check error:
> >   file /usr/lib64/librdkafka++.so.1 conflicts between attempted
> > installs of
> > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs
> > of
> > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
> >
> > Can librdkafka1 be removed from the rsyslog repo? What is the
> > rationale for pinning a version of librdkafka via the librdkafka1
package?
> >
> > If librdkafka1 must stay in the rsyslog repo what do you recommend as
> > the best practice for resolving this conflict?
> >
> > I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why
> > are they there? Is the solution to change the dependency in *our*
> > package form librdkafka to adiscon-librdkafka? I might be comfortable
> > with that if I know why the adiscon packages are there.
> >
> > As a fallback we could add librdkafka as an external project via CMake
> > and build from source but I wanted to check in here first before
> proceeding.
> >
> > Apologies if I missed something obvious. I did my best googling
> > due-diligence and couldn't find anyhing satisfactory.
> >
> > Thanks for your time & patience!
> > ___
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> >
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
> This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond
> our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Rainer Gerhards]

Mmm... I was told that we link statically to librdkafka, so there
should no kafka package at all be installed (actually this was what I asked
for).

@florian, @andre can you comment?

Rainer

Sent from phone, thus brief.

Derek DiFilippo  schrieb am Mo., 16. Apr. 2018, 20:19:

> Hi Rainer, hi everyone,
>
> In addition to a dependency on rsyslog, one of our packages now depends on
> librdkafka and librdkafka-devel.
>
> We want to use the librdkafka packages provided by epel, nothing fancy.
>
> I'm seeing a conflict between the librdkafka1 package in the rsyslog repo
> (note the "1" at the end of the library name) and the ones provided by
> epel.
>
> ...
> ...
>  librdkafka  x86_64 0.11.3-1.el7   epel
> 326 k
>  librdkafka-devel  x86_64 0.11.3-1.el7   epel
>  43 k
>  librdkafka1 x86_64 0.8.5-0 rsyslog_v8
>   100 k
> ...
> ...
>
> Transaction check error:
>   file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>   file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of
> librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
>
> Can librdkafka1 be removed from the rsyslog repo? What is the rationale for
> pinning a version of librdkafka via the librdkafka1 package?
>
> If librdkafka1 must stay in the rsyslog repo what do you recommend as the
> best practice for resolving this conflict?
>
> I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why are
> they there? Is the solution to change the dependency in *our* package form
> librdkafka to adiscon-librdkafka? I might be comfortable with that if I
> know why the adiscon packages are there.
>
> As a fallback we could add librdkafka as an external project via CMake and
> build from source but I wanted to check in here first before proceeding.
>
> Apologies if I missed something obvious. I did my best googling
> due-diligence and couldn't find anyhing satisfactory.
>
> Thanks for your time & patience!
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] librdkafka packages in v8-stable/epel-7/x86_64/RPMS

[Derek DiFilippo]

Hi Rainer, hi everyone,

In addition to a dependency on rsyslog, one of our packages now depends on
librdkafka and librdkafka-devel.

We want to use the librdkafka packages provided by epel, nothing fancy.

I'm seeing a conflict between the librdkafka1 package in the rsyslog repo
(note the "1" at the end of the library name) and the ones provided by epel.

...
...
 librdkafka  x86_64 0.11.3-1.el7   epel
326 k
 librdkafka-devel  x86_64 0.11.3-1.el7   epel
 43 k
 librdkafka1 x86_64 0.8.5-0 rsyslog_v8
  100 k
...
...

Transaction check error:
  file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of
librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64
  file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of
librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64

Can librdkafka1 be removed from the rsyslog repo? What is the rationale for
pinning a version of librdkafka via the librdkafka1 package?

If librdkafka1 must stay in the rsyslog repo what do you recommend as the
best practice for resolving this conflict?

I see a lot of "adisconbuild-librdkafka*" packages in the repo. Why are
they there? Is the solution to change the dependency in *our* package form
librdkafka to adiscon-librdkafka? I might be comfortable with that if I
know why the adiscon packages are there.

As a fallback we could add librdkafka as an external project via CMake and
build from source but I wanted to check in here first before proceeding.

Apologies if I missed something obvious. I did my best googling
due-diligence and couldn't find anyhing satisfactory.

Thanks for your time & patience!
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.