Re: [Rpm-maint] Proposal for rpmbuild patch

[Panu Matilainen]

On 05/19/2018 12:25 AM, Francesco wrote:

Hi Jeff,


2018-05-18 18:54 GMT+02:00 Jeff Johnson >:


You do realize that rpm already computes a digest (usually SHA256,
but MD5 can be configured) for all files in a *.rpm package file?
That information can be retrieved using --queryformat easily (if
make can use digests other than MD5).


No, I really did not know that. I explored --queryformat and found out 
on my Centos7 system that indeed the command

      rpm -q --qf '[%{filenames} %{FILEMD5S}\n]' coreutils | grep date
provides:
/usr/bin/date 
0955c235d9f069b32d22830f661071a33d99425dc5eca69b8da99654c67e0cc7


that really looks exactly what I needed!
Actually as you say that's not a md5 checksum (as the tag name FILEMD5S 
would suggest) but rather a sha256 checksum, but for my purpose there's 
no difference.


The FILEMD5S tag is a backwards compatibility leftover, the actual tag 
name is FILEDIGESTS since rpm 4.6.x and the used algorithm in 
FILEDIGESTALGO. See 
https://github.com/rpm-software-management/rpm/blob/master/macros.in#L393 
for the configuration + values.


- Panu -

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Proposal for rpmbuild patch

[Francesco]

Hi,
FYI: I released version 1.4 of the Python utility that helps interfacing
rpmbuild to GNU make at:
   https://github.com/f18m/rpm-make-rules-dependency-lister

ChangeLog:

   - changed the way the dependency file is generated: query sha256sums
   directly using rpm program instead of decompressing the rpm with rpm2cpio!
   this speeds up the process and makes it more robust
   - allow searching packaged files in multiple search folders
   - add --strip-dirname option
   - improve docs


HTH,
Francesco


2018-05-18 23:25 GMT+02:00 Francesco :

> Hi Jeff,
>
>
> 2018-05-18 18:54 GMT+02:00 Jeff Johnson :
>>
>> You do realize that rpm already computes a digest (usually SHA256, but
>> MD5 can be configured) for all files in a *.rpm package file? That
>> information can be retrieved using --queryformat easily (if make can use
>> digests other than MD5).
>>
>
> No, I really did not know that. I explored --queryformat and found out on
> my Centos7 system that indeed the command
>  rpm -q --qf '[%{filenames} %{FILEMD5S}\n]' coreutils | grep date
> provides:
> /usr/bin/date 0955c235d9f069b32d22830f661071a33d99425dc5eca69b8da99654c67e
> 0cc7
>
> that really looks exactly what I needed!
> Actually as you say that's not a md5 checksum (as the tag name FILEMD5S
> would suggest) but rather a sha256 checksum, but for my purpose there's no
> difference.
>
>
>
>> Meanwhile -- for general adoption/use in Makefiles -- the glue to
>> generate make dependencies should be as simple as possible. The conversions
>> involved with compression, archive format, and digest algorithm are all
>> obvious, but there really isn't good support to do the operations on a pipe
>> with an archive.
>>
>>
> Sure. I wrote a Python utility just for speed but perhaps a small C
> utility would be better (to reduce dependencies) to do the glue operations.
>
>
>> Rationale: with some simple post-processing such new option would allow
>> to e.g., find in a way transparent to the build system employed by the user
>> (automake, cmake, etc) to understand if a .rpm needs to be regenerated
>> (because some of its packaged files have been changed) or not.
>>
>>
>> The biggest issue(s) I see are:
>>
>> 1) generating another artifact (other than *.rpm) from rpmbuild would be
>> quite awkward.
>> rpmbuild (unlike gcc) is 2 layers (shell and %buildroot) from the build
>> tree, and packaging occurs long after the compilation, unlike gcc.
>>
>> 2) rpm has way too many obscure CLI options already. So instead of a CLI
>> option, the approach should be through a configuration macro, using a
>> rule-based value/pattern to generate the output location. A CLI option can
>> always be retrofitted using popt(3) aliases.
>>
>> But a separate executable, not directly integrated into rpmbuild, might
>> simplify the decompression/archive/digest handling with stdout output,
>> making it easier to automate make dependencies on *.rpm contents, since rpm
>> already has all the necessary implementations.
>>
>
> Yeah, at this point the idea of doing an rpmbuild patch is not really
> useful anymore. I was not aware that rpm was able to do all what is needed
> to achieve what I proposed (smarter rpm regeneration).
>
>
>
>>
>> hth
>>
>>
> Yes, that helped, thanks!
>
> Francesco
>
>
>
>
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Proposal for rpmbuild patch

[Francesco]

Hi Jeff,


2018-05-18 18:54 GMT+02:00 Jeff Johnson :
>
> You do realize that rpm already computes a digest (usually SHA256, but MD5
> can be configured) for all files in a *.rpm package file? That information
> can be retrieved using --queryformat easily (if make can use digests other
> than MD5).
>

No, I really did not know that. I explored --queryformat and found out on
my Centos7 system that indeed the command
 rpm -q --qf '[%{filenames} %{FILEMD5S}\n]' coreutils | grep date
provides:
/usr/bin/date
0955c235d9f069b32d22830f661071a33d99425dc5eca69b8da99654c67e0cc7

that really looks exactly what I needed!
Actually as you say that's not a md5 checksum (as the tag name FILEMD5S
would suggest) but rather a sha256 checksum, but for my purpose there's no
difference.



> Meanwhile -- for general adoption/use in Makefiles -- the glue to generate
> make dependencies should be as simple as possible. The conversions involved
> with compression, archive format, and digest algorithm are all obvious, but
> there really isn't good support to do the operations on a pipe with an
> archive.
>
>
Sure. I wrote a Python utility just for speed but perhaps a small C utility
would be better (to reduce dependencies) to do the glue operations.


> Rationale: with some simple post-processing such new option would allow to
> e.g., find in a way transparent to the build system employed by the user
> (automake, cmake, etc) to understand if a .rpm needs to be regenerated
> (because some of its packaged files have been changed) or not.
>
>
> The biggest issue(s) I see are:
>
> 1) generating another artifact (other than *.rpm) from rpmbuild would be
> quite awkward.
> rpmbuild (unlike gcc) is 2 layers (shell and %buildroot) from the build
> tree, and packaging occurs long after the compilation, unlike gcc.
>
> 2) rpm has way too many obscure CLI options already. So instead of a CLI
> option, the approach should be through a configuration macro, using a
> rule-based value/pattern to generate the output location. A CLI option can
> always be retrofitted using popt(3) aliases.
>
> But a separate executable, not directly integrated into rpmbuild, might
> simplify the decompression/archive/digest handling with stdout output,
> making it easier to automate make dependencies on *.rpm contents, since rpm
> already has all the necessary implementations.
>

Yeah, at this point the idea of doing an rpmbuild patch is not really
useful anymore. I was not aware that rpm was able to do all what is needed
to achieve what I proposed (smarter rpm regeneration).



>
> hth
>
>
Yes, that helped, thanks!

Francesco
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Proposal for rpmbuild patch

[Jeff Johnson]

> On May 17, 2018, at 1:38 PM, Francesco  wrote:
> 
> Hi all,
> [I posted a similar proposal under the thread "How to generate GNU make 
> dependency file for a SPEC file automatically?" but I'm reposting here with a 
> better subject]
> 
> Proposal: add a new option to rpmbuild, something like   
> --generate-md5sum= that writes the md5sum (or whatever other 
> hash, sha1sum, etc) of all files placed inside the RPM package. Of course the 
> hash would be that associated to the file prior to cpio compression, after 
> all other rpm processing (e.g., stripping of debug info), just before it gets 
> compressed.
> 

You do realize that rpm already computes a digest (usually SHA256, but MD5 can 
be configured) for all files in a *.rpm package file? That information can be 
retrieved using --queryformat easily (if make can use digests other than MD5).

Meanwhile -- for general adoption/use in Makefiles -- the glue to generate make 
dependencies should be as simple as possible. The conversions involved with 
compression, archive format, and digest algorithm are all obvious, but there 
really isn't good support to do the operations on a pipe with an archive.

> Rationale: with some simple post-processing such new option would allow to 
> e.g., find in a way transparent to the build system employed by the user 
> (automake, cmake, etc) to understand if a .rpm needs to be regenerated 
> (because some of its packaged files have been changed) or not.

The biggest issue(s) I see are:

1) generating another artifact (other than *.rpm) from rpmbuild would be quite 
awkward.
rpmbuild (unlike gcc) is 2 layers (shell and %buildroot) from the build tree, 
and packaging occurs long after the compilation, unlike gcc.

2) rpm has way too many obscure CLI options already. So instead of a CLI 
option, the approach should be through a configuration macro, using a 
rule-based value/pattern to generate the output location. A CLI option can 
always be retrofitted using popt(3) aliases.

But a separate executable, not directly integrated into rpmbuild, might 
simplify the decompression/archive/digest handling with stdout output, making 
it easier to automate make dependencies on *.rpm contents, since rpm already 
has all the necessary implementations.

hth

73 de Jeff

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] Proposal for rpmbuild patch

[Francesco]

 Hi all,
[I posted a similar proposal under the thread "How to generate GNU make
dependency file for a SPEC file automatically?" but I'm reposting here with
a better subject]

Proposal: add a new option to rpmbuild, something like
--generate-md5sum= that writes the md5sum (or whatever other hash, sha1sum, etc) of all
files placed inside the RPM package. Of course the hash would be that
associated to the file prior to cpio compression, after all other rpm
processing (e.g., stripping of debug info), just before it gets compressed.

Rationale: with some simple post-processing such new option would allow to
e.g., find in a way transparent to the build system employed by the user
(automake, cmake, etc) to understand if a .rpm needs to be regenerated
(because some of its packaged files have been changed) or not.

Would RPM maintainers accept such a patch?

Thanks for comments!
Francesco
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint