Re: Deltarpm xz problem with PPC generated rpms?

2009-09-15 Thread Adam Williamson
On Tue, 2009-09-15 at 01:27 -0400, Michel Alexandre Salim wrote: On Mon, Sep 14, 2009 at 1:28 PM, Seth Vidal skvi...@fedoraproject.org wrote: Boy, I'm so glad we decided to jump onto the xz ship. I take it it's too late to back out and stick to bzip2 until the situation stabilizes? I take

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-15 Thread Josh Boyer
On Tue, Sep 15, 2009 at 10:38:32AM -0700, Adam Williamson wrote: On Tue, 2009-09-15 at 01:27 -0400, Michel Alexandre Salim wrote: On Mon, Sep 14, 2009 at 1:28 PM, Seth Vidal skvi...@fedoraproject.org wrote: Boy, I'm so glad we decided to jump onto the xz ship. I take it it's too late to

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-15 Thread Bill Nottingham
Josh Boyer (jwbo...@gmail.com) said: Simple solution: Don't build the noarch RPMs on ppc. Why?: Because F12 is the last release that will have ppc be a primary arch and it is fairly arguable that you want to optimize for the future case going forward anyway. I'm not sure how 'simple' that

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-15 Thread Josh Boyer
On Tue, Sep 15, 2009 at 01:56:55PM -0400, Bill Nottingham wrote: Josh Boyer (jwbo...@gmail.com) said: Simple solution: Don't build the noarch RPMs on ppc. Why?: Because F12 is the last release that will have ppc be a primary arch and it is fairly arguable that you want to optimize for the

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-15 Thread Bill Nottingham
Josh Boyer (jwbo...@gmail.com) said: I'm not sure how 'simple' that is in the koji configuration. It will have to be done anyway, yes? Well, that would involve disabling all ppc builders for a release entirely, which is much simpler. But this isn't the right list anyway. Bill --

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Dave Airlie
On Sun, 2009-09-13 at 19:43 +0300, Jonathan Dieter wrote: Deltarpm seems to be unable to generate correct rpms for deltarpms generated from noarch rpms. The uncompressed payload is correct, but the compressed xz payload is different. To test, using Rawhide's deltarpm, try running

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, Sep 14, 2009 at 1:35 PM, Dave Airlie airl...@redhat.com wrote: On Sun, 2009-09-13 at 19:43 +0300, Jonathan Dieter wrote: Deltarpm seems to be unable to generate correct rpms for deltarpms generated from noarch rpms.  The uncompressed payload is correct, but the compressed xz payload is

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Ben Boeckel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dave Airlie wrote: On Sun, 2009-09-13 at 19:43 +0300, Jonathan Dieter wrote: Deltarpm seems to be unable to generate correct rpms for deltarpms generated from noarch rpms. The uncompressed payload is correct, but the compressed xz payload is

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Andreas Schwab
Ben Boeckel maths...@gmail.com writes: When I was playing around with xz after it came out, it detects the processor and memory available to it and defaults to a different compression quality based on that. Maybe if the compression quality and memory usage is set in the command line, you'd

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 09:25 -0400, Ben Boeckel wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dave Airlie wrote: snip [airl...@pegasus ~]$ md5sum lm93_busted.o d7174fc439c4678927725d06de4f18a2 lm93_busted.o [airl...@pegasus ~]$ xz -z -c lm93_busted.o | md5sum

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Andreas Schwab
Jonathan Dieter jdie...@gmail.com writes: 2. xz generates different compressed files when run on different architectures The problem is that the encoder uses different hash functions depending on the endianess. The hash functions are defined in liblzma/lz/lz_encoder_hash.h, and are based on

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread John Reiser
The problem is that the encoder uses different hash functions depending on the endianess. The hash functions are defined in liblzma/lz/lz_encoder_hash.h, and are based on the values in lzma_crc32_table[0]. This table is different between big end little endian. Not having looked at the

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Kevin Fenzi
On Sun, 13 Sep 2009 19:43:44 +0300 Jonathan Dieter jdie...@gmail.com wrote: ...snip... I have access to i586 and x86_64 systems, but no PPC systems. Could someone either give me access to a PPC system or verify themselves whether xz generates different files on different architectures (all

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 12:30 -0400, Bill Nottingham wrote: Andreas Schwab (sch...@redhat.com) said: 2. xz generates different compressed files when run on different architectures The problem is that the encoder uses different hash functions depending on the endianess. The hash

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 12:30 -0400, Bill Nottingham wrote: Andreas Schwab (sch...@redhat.com) said: 2. xz generates different compressed files when run on different architectures The problem is that the encoder uses different hash functions depending on the endianess. The hash

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Seth Vidal
On Mon, 14 Sep 2009, Jonathan Dieter wrote: Another alternative would be for rpm to have a private copy of the xz-lib code that stays fairly static. Not sure how that would go down. Let us never speak of that again. Thanks. So, to summarize, architecture-specific deltarpms are working

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 20:25 +0300, Jonathan Dieter wrote: Ok, I've just had a conversation on IRC with Lasse Collin, the maintainer of xz. He's now planning on changing xz so it will produce the same output independent of endianess. He hasn't committed to any timeframe, though. snip Sorry,

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Bill Nottingham
Jonathan Dieter (jdie...@gmail.com) said: He did bring up some other very good points, though. Xz's compression output hasn't been set in sand, much less stone. The file format will stay the same, but the same command-line options may result in different compressed files. ... in what way

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 13:39 -0400, Bill Nottingham wrote: Jonathan Dieter (jdie...@gmail.com) said: He did bring up some other very good points, though. Xz's compression output hasn't been set in sand, much less stone. The file format will stay the same, but the same command-line options

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Bill Nottingham
Jonathan Dieter (jdie...@gmail.com) said: ... in what way does he mean this? Obviously passing -1 ... -9 causes different output, much like it does in gzip/bzip2/etc. He means that the file generated using -5 in the future may be different than the file generated using -5 now. As long as

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Bruno Wolff III
On Mon, Sep 14, 2009 at 20:29:11 +0300, Jonathan Dieter jdie...@gmail.com wrote: Sorry, forgot to mention, another option would be to sign the *uncompressed* data in an rpm, so if the compressed data was different, it wouldn't matter. Uncompressing hostile data isn't always a good idea. It

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Kevin Kofler
Jonathan Dieter wrote: So, to summarize, architecture-specific deltarpms are working perfectly in rawhide right now, and, if you're running a PPC machine, all deltarpms are working perfectly. I don't know at what stage the deltarpms are being generated, but in Koji, noarch builds can be on

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 14:34 -0400, Bill Nottingham wrote: Jonathan Dieter (jdie...@gmail.com) said: ... in what way does he mean this? Obviously passing -1 ... -9 causes different output, much like it does in gzip/bzip2/etc. He means that the file generated using -5 in the future may

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 15:43 -0400, James Antill wrote: On Mon, 2009-09-14 at 20:29 +0300, Jonathan Dieter wrote: On Mon, 2009-09-14 at 20:25 +0300, Jonathan Dieter wrote: Ok, I've just had a conversation on IRC with Lasse Collin, the maintainer of xz. He's now planning on changing xz so

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Jonathan Dieter
On Mon, 2009-09-14 at 22:25 +0200, Kevin Kofler wrote: Jonathan Dieter wrote: So, to summarize, architecture-specific deltarpms are working perfectly in rawhide right now, and, if you're running a PPC machine, all deltarpms are working perfectly. I don't know at what stage the deltarpms

Re: Deltarpm xz problem with PPC generated rpms?

2009-09-14 Thread Michel Alexandre Salim
On Mon, Sep 14, 2009 at 1:28 PM, Seth Vidal skvi...@fedoraproject.org wrote: Boy, I'm so glad we decided to jump onto the xz ship. I take it it's too late to back out and stick to bzip2 until the situation stabilizes? I take it whatever solution ends up in F-12 is likely to be the one used by