Re: [Rpm-maint] Reserve ALT RPM tags range

[Panu Matilainen]

On 03/07/2018 10:08 PM, Vladimir D. Seleznev wrote:

On Wed, Mar 07, 2018 at 05:03:54PM +0200, Panu Matilainen wrote:

On 03/07/2018 03:48 PM, Vladimir D. Seleznev wrote:

On Thu, Mar 01, 2018 at 09:58:58AM +0200, Panu Matilainen wrote:

On 02/28/2018 12:03 AM, Vladimir D. Seleznev wrote:

Hello, rpm-maint@!

We in ALT Linux Team implemented some new RPM tags for our RPM package
header and RPM database. Since RPM tags are numerical identifiers and we
want to keep compatibility with mainstream RPM tools and packages, we
are requesting to reserve some RPM tag range for our purpose (around 100
RPM tags) to avoid the collision with RPM tag numbers, that can possibly
will be added to rpm.org in the future.


Yes we can reserve tags, it's not like we're going to run out of 32bit
integers anytime soon. However we're not going to foster distro
proprietary development & forks by blindly handing you 100 "do whatever
you want" tags. That's not how this works.


This is not about proprietary development, all ALT rpm features are
available in git repo at git.alt [1] [2].


Yes, I know. Of course it's not proprietary in the strict sense, but for
all practical purposes the results of that work are not available to
anybody not using ALT. I should've said "proprietary" with quotes.


It would be better to consolidate efforts but currently ALT rpm is a
fork with more than a decade-long history and a lot of good features
implemented such as autogenerated dependency for various interpreted
languages, set-versions, checks and a lot of other enhancements, and we
just can't drop these. Still we tried to port ours changes to new at
that time rpm 4.13 to keep our rpm closer to upstream (for now it
provides rpm(8) and some other, but no rpm-build).


Yeah I know and I was glad to see that happen.




If a feature is interesting to Alt, there's more than a slight change
it's interesting to others as well, so you should always initially
target getting the thing upstream. The price of tag reservations is
discussing your plans upstream first, either here on rpm-maint or GH
ticket/PR.


And what about features ALT already has? They are integrated to ALT
package ecosystem with repositories with thousands of packages which
should be able to be installed and handled by package manager. And what
about the features like set-versions, which you don't want to integrate
[3]?


Just send a patch that makes the reservations for the tags you already
use, with their actual names and types and all? See the RPMTAG_MSSF*
tags for an example of how to do it. If there are clashes or other
issues, the only way to work them out is by laying it all on the table.



We asked for 100 rpm tags with a big reserve.

Understood, but handing out a big pile of free tags to grab and run is
not going to bring your rpm any closer to upstream. Quite the contrary.
I'm just trying to encourage you folks into an upstream first approach
by whatever means I have available. If it feels a bit like extortion to
you, sorry ;)


At the moment we want to use RPMTAG_AUTOINSTALLED in rpm db that helps
apt to determine what package was installed automatically as a
dependency of manually installed packages.


Tracking the install reason is a common need across pretty much all the
depsolvers. Send a patch and we'll see where it goes from there?


Ok, but the tag is just for record to RPM database, all the logic is on apt
side.


Sure, rpm cli wouldn't set such a tag by itself because the notion of 
"pulled in as a dependency" doesn't happen there. But if/when present, 
there might be some uses for it in rpm too - even if just informational 
display.





Second tag we want to use is RPMTAG_IDENTITY which determine a
characteristic of package build and is represented hash of significant
tags (in opposite to RPMTAG_SHA1HEADER, that is a hash of all package
tags).


A bit hard to comment based on just that, but if I understood correctly
then I don't see why not, sounds potentially quite useful in fact. Send
a patch so we can discuss the specifics?


I should've explained the purpose of this tag more specific, but I
think it is ALT specific one. It is a hash representation of build
result that solves two main problems:

* check reproducible build (the tag values of two builds are equal in
that case);
* generate more strict inter-subpackage dependencies.


The first case more or less figured out, the second one I didn't but I 
think it's an interesting idea.


Why would you think this is ALT specific? I see *absolutely nothing* 
distro specific about that. Ditto with the autoinstalled tag, BTW.




For now it isn't in ALT rpm code tree because we have to make some
decisions. But is would be good to already have reserver rpm tag for it.


Please, please, please, bring that discussion upstream! Start by sending 
the patch you have. If there are open questions or it needs cleaning up 
or such, simply state that it's an RFC patch.



In recent years this here market has become quite a bit 

Re: [Rpm-maint] Reserve ALT RPM tags range

[Vladimir D. Seleznev]

On Wed, Mar 07, 2018 at 05:03:54PM +0200, Panu Matilainen wrote:
> On 03/07/2018 03:48 PM, Vladimir D. Seleznev wrote:
> > On Thu, Mar 01, 2018 at 09:58:58AM +0200, Panu Matilainen wrote:
> >> On 02/28/2018 12:03 AM, Vladimir D. Seleznev wrote:
> >>> Hello, rpm-maint@!
> >>>
> >>> We in ALT Linux Team implemented some new RPM tags for our RPM package
> >>> header and RPM database. Since RPM tags are numerical identifiers and we
> >>> want to keep compatibility with mainstream RPM tools and packages, we
> >>> are requesting to reserve some RPM tag range for our purpose (around 100
> >>> RPM tags) to avoid the collision with RPM tag numbers, that can possibly
> >>> will be added to rpm.org in the future.
> >>
> >> Yes we can reserve tags, it's not like we're going to run out of 32bit
> >> integers anytime soon. However we're not going to foster distro
> >> proprietary development & forks by blindly handing you 100 "do whatever
> >> you want" tags. That's not how this works.
> > 
> > This is not about proprietary development, all ALT rpm features are
> > available in git repo at git.alt [1] [2].
> 
> Yes, I know. Of course it's not proprietary in the strict sense, but for 
> all practical purposes the results of that work are not available to 
> anybody not using ALT. I should've said "proprietary" with quotes.
> 
> > It would be better to consolidate efforts but currently ALT rpm is a
> > fork with more than a decade-long history and a lot of good features
> > implemented such as autogenerated dependency for various interpreted
> > languages, set-versions, checks and a lot of other enhancements, and we
> > just can't drop these. Still we tried to port ours changes to new at
> > that time rpm 4.13 to keep our rpm closer to upstream (for now it
> > provides rpm(8) and some other, but no rpm-build).
> 
> Yeah I know and I was glad to see that happen.
> 
> > 
> >> If a feature is interesting to Alt, there's more than a slight change
> >> it's interesting to others as well, so you should always initially
> >> target getting the thing upstream. The price of tag reservations is
> >> discussing your plans upstream first, either here on rpm-maint or GH
> >> ticket/PR.
> > 
> > And what about features ALT already has? They are integrated to ALT
> > package ecosystem with repositories with thousands of packages which
> > should be able to be installed and handled by package manager. And what
> > about the features like set-versions, which you don't want to integrate
> > [3]?
> 
> Just send a patch that makes the reservations for the tags you already 
> use, with their actual names and types and all? See the RPMTAG_MSSF* 
> tags for an example of how to do it. If there are clashes or other 
> issues, the only way to work them out is by laying it all on the table.
> 
> > 
> > We asked for 100 rpm tags with a big reserve.
> Understood, but handing out a big pile of free tags to grab and run is 
> not going to bring your rpm any closer to upstream. Quite the contrary. 
> I'm just trying to encourage you folks into an upstream first approach 
> by whatever means I have available. If it feels a bit like extortion to 
> you, sorry ;)
> 
> > At the moment we want to use RPMTAG_AUTOINSTALLED in rpm db that helps
> > apt to determine what package was installed automatically as a
> > dependency of manually installed packages.
> 
> Tracking the install reason is a common need across pretty much all the 
> depsolvers. Send a patch and we'll see where it goes from there?

Ok, but the tag is just for record to RPM database, all the logic is on apt
side.

> > Second tag we want to use is RPMTAG_IDENTITY which determine a
> > characteristic of package build and is represented hash of significant
> > tags (in opposite to RPMTAG_SHA1HEADER, that is a hash of all package
> > tags).
> 
> A bit hard to comment based on just that, but if I understood correctly 
> then I don't see why not, sounds potentially quite useful in fact. Send 
> a patch so we can discuss the specifics?

I should've explained the purpose of this tag more specific, but I
think it is ALT specific one. It is a hash representation of build
result that solves two main problems:

* check reproducible build (the tag values of two builds are equal in
that case);
* generate more strict inter-subpackage dependencies.

For now it isn't in ALT rpm code tree because we have to make some
decisions. But is would be good to already have reserver rpm tag for it.

> In recent years this here market has become quite a bit more open than 
> it once was and with many more parties involved. But nothing will get 
> upstreamed if it's not submitted at all. Even if upstream doesn't take 
> it right away, at least people will then be aware of the work and might 
> grow interested, which might lead to being used in other distros too -> 
> more pressure for upstream to include it. In some cases you might even 
> see others jump in to help with a feature, shock horror! And even if 
> nothing 

Re: [Rpm-maint] Reserve ALT RPM tags range

[Panu Matilainen]

On 03/07/2018 03:48 PM, Vladimir D. Seleznev wrote:

On Thu, Mar 01, 2018 at 09:58:58AM +0200, Panu Matilainen wrote:

On 02/28/2018 12:03 AM, Vladimir D. Seleznev wrote:

Hello, rpm-maint@!

We in ALT Linux Team implemented some new RPM tags for our RPM package
header and RPM database. Since RPM tags are numerical identifiers and we
want to keep compatibility with mainstream RPM tools and packages, we
are requesting to reserve some RPM tag range for our purpose (around 100
RPM tags) to avoid the collision with RPM tag numbers, that can possibly
will be added to rpm.org in the future.


Yes we can reserve tags, it's not like we're going to run out of 32bit
integers anytime soon. However we're not going to foster distro
proprietary development & forks by blindly handing you 100 "do whatever
you want" tags. That's not how this works.


This is not about proprietary development, all ALT rpm features are
available in git repo at git.alt [1] [2].


Yes, I know. Of course it's not proprietary in the strict sense, but for 
all practical purposes the results of that work are not available to 
anybody not using ALT. I should've said "proprietary" with quotes.



It would be better to consolidate efforts but currently ALT rpm is a
fork with more than a decade-long history and a lot of good features
implemented such as autogenerated dependency for various interpreted
languages, set-versions, checks and a lot of other enhancements, and we
just can't drop these. Still we tried to port ours changes to new at
that time rpm 4.13 to keep our rpm closer to upstream (for now it
provides rpm(8) and some other, but no rpm-build).


Yeah I know and I was glad to see that happen.




If a feature is interesting to Alt, there's more than a slight change
it's interesting to others as well, so you should always initially
target getting the thing upstream. The price of tag reservations is
discussing your plans upstream first, either here on rpm-maint or GH
ticket/PR.


And what about features ALT already has? They are integrated to ALT
package ecosystem with repositories with thousands of packages which
should be able to be installed and handled by package manager. And what
about the features like set-versions, which you don't want to integrate
[3]?


Just send a patch that makes the reservations for the tags you already 
use, with their actual names and types and all? See the RPMTAG_MSSF* 
tags for an example of how to do it. If there are clashes or other 
issues, the only way to work them out is by laying it all on the table.




We asked for 100 rpm tags with a big reserve.
Understood, but handing out a big pile of free tags to grab and run is 
not going to bring your rpm any closer to upstream. Quite the contrary. 
I'm just trying to encourage you folks into an upstream first approach 
by whatever means I have available. If it feels a bit like extortion to 
you, sorry ;)



At the moment we want to use RPMTAG_AUTOINSTALLED in rpm db that helps
apt to determine what package was installed automatically as a
dependency of manually installed packages.


Tracking the install reason is a common need across pretty much all the 
depsolvers. Send a patch and we'll see where it goes from there?




Second tag we want to use is RPMTAG_IDENTITY which determine a
characteristic of package build and is represented hash of significant
tags (in opposite to RPMTAG_SHA1HEADER, that is a hash of all package
tags).


A bit hard to comment based on just that, but if I understood correctly 
then I don't see why not, sounds potentially quite useful in fact. Send 
a patch so we can discuss the specifics?


In recent years this here market has become quite a bit more open than 
it once was and with many more parties involved. But nothing will get 
upstreamed if it's not submitted at all. Even if upstream doesn't take 
it right away, at least people will then be aware of the work and might 
grow interested, which might lead to being used in other distros too -> 
more pressure for upstream to include it. In some cases you might even 
see others jump in to help with a feature, shock horror! And even if 
nothing else comes out of it, at the very least you'll get your tags 
reserved in time.


- Panu -
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Reserve ALT RPM tags range

[Vladimir D. Seleznev]

On Wed, Mar 07, 2018 at 11:31:17AM +0200, Panu Matilainen wrote:
> On 03/06/2018 06:29 PM, Vladimir D. Seleznev wrote:
> > On Wed, Feb 28, 2018 at 05:54:27AM -0500, Jeff Johnson wrote:
> >>> You might try the convention of assigning tags in the 1Gb space from 
> >>> 0x4000 to 0x4fff as you wish.
> >>>
> >>> (aside)
> >>> RPM5 has something called "arbitrary tags" in the 0x4000 -> 
> >>> 0x4fff range.
> >>>
> >>> Short answer:
> >>> Choose any tagno in that range and do what you wish.
> >>>
> >>> Longer answer:
> >>>
> >>> Choose a tag name string.
> >>>
> >>> The  tagno is computed from a (configurable) tag name string as follows:
> >>>
> >>> 1) the tag name plain text is canonicalized (leading alphabetic, 
> >>> alphanumeric characters, 1st letter uppercase, rest lowercase). E.g. 
> >>> "Mynewtag42"
> >>>
> >>> 2) the  4 binary bytes of the SHA1 of the canonical string are copied 
> >>> into a uint32 (which needs to be swabbed on big endian platforms).
> >>>
> >>> 3) the tagno is assigned by then masking on the 0x4000 arbitrary tag 
> >>> identifier onto the least significant 30 bits of the uint32.
> > 
> > Thank you for the suggestion, I think we shall follow this!
> > 
> 
> Sigh. What's so secret about that work that you can't at least briefly 
> explain the actual use for those tags here?

Sorry for delaying with answer, I sent it just now.

-- 
   With best regards,
   Vladimir D. Seleznev
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Reserve ALT RPM tags range

[Panu Matilainen]

On 03/06/2018 06:29 PM, Vladimir D. Seleznev wrote:

On Wed, Feb 28, 2018 at 05:54:27AM -0500, Jeff Johnson wrote:

You might try the convention of assigning tags in the 1Gb space from 0x4000 
to 0x4fff as you wish.

(aside)
RPM5 has something called "arbitrary tags" in the 0x4000 -> 0x4fff 
range.

Short answer:
Choose any tagno in that range and do what you wish.

Longer answer:

Choose a tag name string.

The  tagno is computed from a (configurable) tag name string as follows:

1) the tag name plain text is canonicalized (leading alphabetic, alphanumeric characters, 
1st letter uppercase, rest lowercase). E.g. "Mynewtag42"

2) the  4 binary bytes of the SHA1 of the canonical string are copied into a 
uint32 (which needs to be swabbed on big endian platforms).

3) the tagno is assigned by then masking on the 0x4000 arbitrary tag 
identifier onto the least significant 30 bits of the uint32.


Thank you for the suggestion, I think we shall follow this!



Sigh. What's so secret about that work that you can't at least briefly 
explain the actual use for those tags here?


- Panu -
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Reserve ALT RPM tags range

[Vladimir D. Seleznev]

On Wed, Feb 28, 2018 at 05:54:27AM -0500, Jeff Johnson wrote:
> > You might try the convention of assigning tags in the 1Gb space from 
> > 0x4000 to 0x4fff as you wish.
> > 
> > (aside)
> > RPM5 has something called "arbitrary tags" in the 0x4000 -> 0x4fff 
> > range.
> > 
> > Short answer:
> > Choose any tagno in that range and do what you wish.
> > 
> > Longer answer:
> > 
> > Choose a tag name string.
> > 
> > The  tagno is computed from a (configurable) tag name string as follows:
> > 
> > 1) the tag name plain text is canonicalized (leading alphabetic, 
> > alphanumeric characters, 1st letter uppercase, rest lowercase). E.g. 
> > "Mynewtag42"
> > 
> > 2) the  4 binary bytes of the SHA1 of the canonical string are copied into 
> > a uint32 (which needs to be swabbed on big endian platforms).
> > 
> > 3) the tagno is assigned by then masking on the 0x4000 arbitrary tag 
> > identifier onto the least significant 30 bits of the uint32.

Thank you for the suggestion, I think we shall follow this!

-- 
   With best regards,
   Vladimir D. Seleznev
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Reserve ALT RPM tags range

[Panu Matilainen]

On 02/28/2018 12:03 AM, Vladimir D. Seleznev wrote:

Hello, rpm-maint@!

We in ALT Linux Team implemented some new RPM tags for our RPM package
header and RPM database. Since RPM tags are numerical identifiers and we
want to keep compatibility with mainstream RPM tools and packages, we
are requesting to reserve some RPM tag range for our purpose (around 100
RPM tags) to avoid the collision with RPM tag numbers, that can possibly
will be added to rpm.org in the future.


Yes we can reserve tags, it's not like we're going to run out of 32bit 
integers anytime soon. However we're not going to foster distro 
proprietary development & forks by blindly handing you 100 "do whatever 
you want" tags. That's not how this works.


If a feature is interesting to Alt, there's more than a slight change 
it's interesting to others as well, so you should always initially 
target getting the thing upstream. The price of tag reservations is 
discussing your plans upstream first, either here on rpm-maint or GH 
ticket/PR.


- Panu -
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Reserve ALT RPM tags range

[Neal Gompa]

On Tue, Feb 27, 2018 at 5:03 PM, Vladimir D. Seleznev
 wrote:
> Hello, rpm-maint@!
>
> We in ALT Linux Team implemented some new RPM tags for our RPM package
> header and RPM database. Since RPM tags are numerical identifiers and we
> want to keep compatibility with mainstream RPM tools and packages, we
> are requesting to reserve some RPM tag range for our purpose (around 100
> RPM tags) to avoid the collision with RPM tag numbers, that can possibly
> will be added to rpm.org in the future.
>

Can you propose a patch series as a pull request at
https://github.com/rpm-software-management/rpm/pulls ?

Or send an email patch series?

Also, 100 tags? What did you implement?


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint