Re: Making rpm5 work with dnf

[Alexander Kanavin]

On 11/28/2016 06:35 PM, Jeffrey Johnson wrote:

2) There are several repetitive cosmetic (but annoying) API differences.
These problems could (almost) be handled with some #defines
* the “match iterator” rpmmi* interface for rpmdb retrieval
* the “problem set” rpmps* interface for problem sets


As far as libhif build errors related to these differences go, a 
rpm46compat.h include seems to deal with them.


I'm not sure if this is the right thing to do, because I don't know how 
supported that compatibility layer is, and also it produces a duplicate 
declaration error instead:


| In file included from 
/home/ak/development/poky/build/tmp/sysroots/x86_64-linux/usr/include/rpm/rpm46compat.h:10:0,
|  from 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-rpmts.c:39:
| 
/home/ak/development/poky/build/tmp/sysroots/x86_64-linux/usr/include/rpm/rpm4compat.h:354:21: 
error: static declaration of ‘fdSize’ follows non-static declaration

|  static inline off_t fdSize(FD_t fd){
|  ^~
| In file included from 
/home/ak/development/poky/build/tmp/sysroots/x86_64-linux/usr/include/rpm/rpm4compat.h:52:0,
|  from 
/home/ak/development/poky/build/tmp/sysroots/x86_64-linux/usr/include/rpm/rpm46compat.h:10,
|  from 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-rpmts.c:39:
| 
/home/ak/development/poky/build/tmp/sysroots/x86_64-linux/usr/include/rpm/rpmio.h:543:9: 
note: previous declaration of ‘fdSize’ was here

|  off_t   fdSize(FD_t fd)
|  ^~

What would you suggest?


3) Some rpm.org interfaces are intractably different and mostly useless for RPM5
* the “tag data” rpmtd*
* the rpmkeyring* pubkey handling
While these interfaces have been merged into rpm5 libraries (rpmtd for 
python bindings,
rpmkeyring just in case), that isn’t/wasn't the best engineering 
because RPM5 itself
doesn’t use or need or test those interfaces. The better approach going 
forward (imho)
will be to refactor those api’s somewhere other than in rpm5 libraries. 
I don’t yet
know what to do with those apis, but I’d like _NOT_ to be forced to 
maintain
multiple divergent/redundant API’s in RPM5 simply to support a dnf (or 
rpm-python) port.


rpmtd and rpmkeyring are used in various places around libhif - I didn't 
look yet how exactly, but they are a serious blocker.


A couple other issues I see:

1)
| 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-rpmts.c: 
In function ‘dnf_rpmts_get_problem_str’:
| 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-rpmts.c:241:10: 
error: ‘RPMPROB_OBSOLETES’ undeclared (first use in this function)

|  case RPMPROB_OBSOLETES:
|   ^

2)
| 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-context.c: 
In function ‘dnf_context_get_base_arch’:
| 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-context.c:297:5: 
error: implicit declaration of function ‘rpmGetOsInfo’ 
[-Werror=implicit-function-declaration]

|  rpmGetOsInfo(, NULL);
|  ^~~~
| 
/home/ak/development/poky/build/workspace/sources/libhif/libdnf/dnf-context.c:299:5: 
error: implicit declaration of function ‘rpmGetArchInfo’ 
[-Werror=implicit-function-declaration]

|  rpmGetArchInfo(, NULL);
|  ^~



Meanwhile:
Are you able to build the RPM5 rpm-5_4 branch from CVS?
That is likely needed pretty soon. I can help as necessary.


Openembedded build system is currently packaging rpm 5.4.16 (plus a lot 
of custom patches). I can check out the latest CVS separately and cherry 
pick needed commits from there as needed (and append them to the list of 
custom patches), if that is okay.


Otherwise, Mark Hatle needs to provide me with a working rpm recipe that 
is taking latest CVS code. I'll ask him now.



(aside)
btw, this thread likely should be moved to , I’ve added a 
CC … *shrug*


I'm CCing this message to rpm-devel too, the following messages will go 
to rpm-devel only.


Alex
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Alexander Kanavin]

On 11/26/2016 01:07 AM, Jeffrey Johnson wrote:

FWIW, your invitation expired or is otherwise unusable (but at least
I can read your code, todo++).


Yeah, I pressed the button on github and only then figured out that it's 
just for allowing you to write to my repository, and then I withdrew the 
invite. We can publish and share all work via your rpm5 repos.



You have chosen to fork libhif from
rpm-software-management/libhif

rather than a fork (of a fork) from
https://github.com/rpm5/libhif

That forces our coordination to be pulled from the only common
root at
rpm-software-management/libhif

which almost certainly precludes any participation from me and rpm5.org

for various reasons.


libhif has been in active development lately (51 commits ahead of your 
repo), and so I wanted to use their latest code as a starting point. 
Also your repo has a single commit that seems to be incomplete and 
squashing together several unrelated fixes (without documenting them) 
[1]. Can I move that commit to a 'jeff' branch, so the master branch 
contains only proper commits and is updated from upstream regularly?


[1] 
https://github.com/rpm5/libhif/commit/1515ca9e9669a4c65caccb38cbe578775c63f282



Note that there are several other efforts attempting a dnf->…->rpm5 tool
chain that I
am aware of. Which is why I attempted RPM5 repositories to permit
collaboration, and
am perfectly willing to give write access to anyone who wishes.

I am also perfectly willing to let someone other than rpm5.org
 administrate the mess if that
is what is desired. I do encourage all of you to collaborate early and
work forward from
working tools. There’s a fair amount of subtle work that will be needed
imho.
What is your intent: collaboration with rpm5.org  or
collaboration with rpm-software-management?


My idea is that once we get something working in rpm5 repositories and 
have a reasonable set of commits, I can try to approach rpm4 people with 
those patches (you don't have to be involved in that). If they agree to 
take them, great, if not, we'll figure out some way to maintain them 
separately.



Um rpm5.org  and rpm.org  have
different API’s, and are most definitely different
implementations these days. Its not just include files, and more than
libhif is going to
be needed to build a working dnf->…->rpm5 tool chain.


Yep, but let's start with libhif and its build errors.


Alex
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Jeff Johnson]

Sent from my iPad

> On Nov 25, 2016, at 11:42 PM, Mark Hatle  wrote:
> 
>> On 11/25/16 5:07 PM, Jeffrey Johnson wrote:
>> 
>>> On Nov 24, 2016, at 8:50 AM, Alexander Kanavin
>>> >> >
>>> wrote:
>>> 
 On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
 
 Again: I cannot do much more than suggest porting approaches unless I can
 attempt reproducers.
 Please try to expose your git repositories somehow, either publicly or 
 privately.
>>> 
>>> Here it is:
>>> 
>>> https://github.com/kanavin/libhif/commits/master
>>> 
>> 
>> Good. I will take a look at your libhif efforts. Note that libhif is only
>> one of several repositories that are going to be needed.
>> 
>> FWIW, your invitation expired or is otherwise unusable (but at least
>> I can read your code, todo++).
>> 
>> (aside)
>> Before we go too much further here:
>> 
>> What is the intent of this collaboration?
> 
> The functional goal in this work is to stop using smart in the Yocto Project 
> and
> switch to DNF.  (We've found nothing else that is a reasonable alternative 
> that
> is still under active development.)
> 

This part I understand (from conversations mid-summer).

> As far as the way things are done, I'll leave that between you, Alexander and
> anyone else interested.
> 

We're getting there ... perhaps some OpenMandriva involvement after
their weekly meeting 11am EST next Wed. I believe there is PLD interest
as well, perhaps a few other efforts interested in collaborating too.

hth

73 de Jeff

> --Mark
> 
>> You have chosen to fork libhif from
>> rpm-software-management/libhif 
>> 
>> rather than a fork (of a fork) from
>> https://github.com/rpm5/libhif
>> 
>> That forces our coordination to be pulled from the only common
>> root at
>> rpm-software-management/libhif 
>> 
>> which almost certainly precludes any participation from me and rpm5.org
>> 
>> for various reasons.
>> 
>> Note that there are several other efforts attempting a dnf->…->rpm5 tool 
>> chain
>> that I
>> am aware of. Which is why I attempted RPM5 repositories to permit 
>> collaboration, and
>> am perfectly willing to give write access to anyone who wishes.
>> 
>> I am also perfectly willing to let someone other than rpm5.org 
>> 
>> administrate the mess if that
>> is what is desired. I do encourage all of you to collaborate early and work
>> forward from
>> working tools. There’s a fair amount of subtle work that will be needed imho.
>> 
>> What is your intent: collaboration with rpm5.org  or
>> collaboration with rpm-software-management?
>> 
>>> I've fixed what I could by adding rpm5 includes, but the remaining build
>>> issues are all caused by actual API incompatibilities between 4 and 5. I can
>>> file them as separate github issues if you want.
>>> 
>> 
>> Um rpm5.org  and rpm.org  have different 
>> API’s,
>> and are most definitely different
>> implementations these days. Its not just include files, and more than libhif 
>> is
>> going to
>> be needed to build a working dnf->…->rpm5 tool chain.
>> 
>> I’ll take a look at your libhif this weekend.
>> 
>> 73 de Jeff
>>> Alex
>>> 
>>> __
>>> RPM Package Managerhttp://rpm5.org
>>> User Communication List rpm-users@rpm5.org
>>> 
>> 
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Mark Hatle]

On 11/25/16 5:07 PM, Jeffrey Johnson wrote:
> 
>> On Nov 24, 2016, at 8:50 AM, Alexander Kanavin
>> > >
>> wrote:
>>
>> On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
>>
>>> Again: I cannot do much more than suggest porting approaches unless I can
>>> attempt reproducers.
>>> Please try to expose your git repositories somehow, either publicly or 
>>> privately.
>>
>> Here it is:
>>
>> https://github.com/kanavin/libhif/commits/master
>>
> 
> Good. I will take a look at your libhif efforts. Note that libhif is only
> one of several repositories that are going to be needed.
> 
> FWIW, your invitation expired or is otherwise unusable (but at least
> I can read your code, todo++).
> 
> (aside)
> Before we go too much further here:
> 
> What is the intent of this collaboration?

The functional goal in this work is to stop using smart in the Yocto Project and
switch to DNF.  (We've found nothing else that is a reasonable alternative that
is still under active development.)

As far as the way things are done, I'll leave that between you, Alexander and
anyone else interested.

--Mark

> You have chosen to fork libhif from
> rpm-software-management/libhif 
> 
> rather than a fork (of a fork) from
> https://github.com/rpm5/libhif
> 
> That forces our coordination to be pulled from the only common
> root at
> rpm-software-management/libhif 
> 
> which almost certainly precludes any participation from me and rpm5.org
> 
> for various reasons.
> 
> Note that there are several other efforts attempting a dnf->…->rpm5 tool chain
> that I
> am aware of. Which is why I attempted RPM5 repositories to permit 
> collaboration, and
> am perfectly willing to give write access to anyone who wishes.
> 
> I am also perfectly willing to let someone other than rpm5.org 
> 
> administrate the mess if that
> is what is desired. I do encourage all of you to collaborate early and work
> forward from
> working tools. There’s a fair amount of subtle work that will be needed imho.
> 
> What is your intent: collaboration with rpm5.org  or
> collaboration with rpm-software-management?
> 
>> I've fixed what I could by adding rpm5 includes, but the remaining build
>> issues are all caused by actual API incompatibilities between 4 and 5. I can
>> file them as separate github issues if you want.
>>
> 
> Um rpm5.org  and rpm.org  have different 
> API’s,
> and are most definitely different
> implementations these days. Its not just include files, and more than libhif 
> is
> going to
> be needed to build a working dnf->…->rpm5 tool chain.
> 
> I’ll take a look at your libhif this weekend.
> 
> 73 de Jeff
>> Alex
>>
>> __
>> RPM Package Managerhttp://rpm5.org
>> User Communication List rpm-users@rpm5.org
>> 
> 

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Jeffrey Johnson]

> On Nov 24, 2016, at 8:50 AM, Alexander Kanavin 
>  wrote:
> 
> On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
> 
>> Again: I cannot do much more than suggest porting approaches unless I can 
>> attempt reproducers.
>> Please try to expose your git repositories somehow, either publicly or 
>> privately.
> 
> Here it is:
> 
> https://github.com/kanavin/libhif/commits/master
> 

Good. I will take a look at your libhif efforts. Note that libhif is only
one of several repositories that are going to be needed.

FWIW, your invitation expired or is otherwise unusable (but at least
I can read your code, todo++).

(aside)
Before we go too much further here:

What is the intent of this collaboration?

You have chosen to fork libhif from
rpm-software-management/libhif 

rather than a fork (of a fork) from
https://github.com/rpm5/libhif

That forces our coordination to be pulled from the only common
root at
rpm-software-management/libhif 

which almost certainly precludes any participation from me and rpm5.org
for various reasons.

Note that there are several other efforts attempting a dnf->…->rpm5 tool chain 
that I
am aware of. Which is why I attempted RPM5 repositories to permit 
collaboration, and
am perfectly willing to give write access to anyone who wishes.

I am also perfectly willing to let someone other than rpm5.org administrate the 
mess if that
is what is desired. I do encourage all of you to collaborate early and work 
forward from
working tools. There’s a fair amount of subtle work that will be needed imho.

What is your intent: collaboration with rpm5.org or collaboration with 
rpm-software-management?

> I've fixed what I could by adding rpm5 includes, but the remaining build 
> issues are all caused by actual API incompatibilities between 4 and 5. I can 
> file them as separate github issues if you want.
> 

Um rpm5.org and rpm.org have different API’s, and are most definitely different
implementations these days. Its not just include files, and more than libhif is 
going to
be needed to build a working dnf->…->rpm5 tool chain.

I’ll take a look at your libhif this weekend.

73 de Jeff
> Alex
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Alexander Kanavin]

On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:


Again: I cannot do much more than suggest porting approaches unless I can 
attempt reproducers.
Please try to expose your git repositories somehow, either publicly or 
privately.


Here it is:

https://github.com/kanavin/libhif/commits/master

I've fixed what I could by adding rpm5 includes, but the remaining build 
issues are all caused by actual API incompatibilities between 4 and 5. I 
can file them as separate github issues if you want.


Alex

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Jeffrey Johnson]

> On Nov 22, 2016, at 11:20 AM, Alexander Kanavin 
>  wrote:
> 
> On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
> 
>> Again: I cannot do much more than suggest porting approaches unless I can 
>> attempt reproducers.
>> Please try to expose your git repositories somehow, either publicly or 
>> privately.
> 
> I'll push those to somewhere public after running the unit tests and fixing 
> any resulting trivial issues.
> 

I suggest we start trying to work from the same repositories:

https://github.com/rpm5

and use github forks/issues rather than e-mnail to coordinate efforts.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Alexander Kanavin]

On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:


Again: I cannot do much more than suggest porting approaches unless I can 
attempt reproducers.
Please try to expose your git repositories somehow, either publicly or 
privately.


I'll push those to somewhere public after running the unit tests and 
fixing any resulting trivial issues.



Alex
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Jeffrey Johnson]

> On Nov 22, 2016, at 10:39 AM, Alexander Kanavin 
>  wrote:
> 
> On 11/22/2016 05:16 PM, Alexander Kanavin wrote:
>> AttributeError: 'module' object has no attribute 'RPMTRANS_FLAG_NOCONTEXTS'
> 
> There's a similar issue with RPMTRANS_FLAG_NOFILEDIGEST. If I patch these out 
> from dnf, then dnf is at least able to start - I haven't run any of its unit 
> tests yet.
> 

Both NOFILEDIGEST and NOCONTEXTS are disablers unnecessary to dnf execution,
with alternative means to enable/disable other than through python bindings.

Note that the RPM5 SELinux implementation does not include or permit loadable 
modules.
A SELinux port is much more than exposing a NOCONTEXTS bit. NOFILEDIGEST also 
has
a slightly different  effect between rpm.org and rpm5.org.

Disabling SELinux is still common functionality however: setting 
NOCONTEXTS/NOFILEDIGEST to 0
is one way to achieve.

The fundamental porting problem (in both python modules) is how symbols are 
exposed.

The better way going forward is to set global values in __init__.py as needed.

Again: I cannot do much more than suggest porting approaches unless I can 
attempt reproducers.
Please try to expose your git repositories somehow, either publicly or 
privately.

73 de Jeff
> Alex
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Making rpm5 work with dnf

[Alexander Kanavin]

On 11/22/2016 05:39 PM, Alexander Kanavin wrote:

On 11/22/2016 05:16 PM, Alexander Kanavin wrote:

AttributeError: 'module' object has no attribute
'RPMTRANS_FLAG_NOCONTEXTS'


There's a similar issue with RPMTRANS_FLAG_NOFILEDIGEST. If I patch
these out from dnf, then dnf is at least able to start - I haven't run
any of its unit tests yet.


On closer look, neither of these flags is used anywhere in dnf or its 
unit tests. So maybe patching them out is best. I'll run the tests and 
let you know how it goes.


Alex

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org