Re: Building packages twice in a row

2007-05-22 Thread Goswin von Brederlow
Russ Allbery [EMAIL PROTECTED] writes:

 Martin Zobel-Helas [EMAIL PROTECTED] writes:
 On Wed May 16, 2007 at 10:11:55 +0200, Stefano Zacchiroli wrote:

 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

 That would surely be the better solution to catch this policy violation.

 Many, many more packages would fail this and some of us (such as myself)
 believe strongly that many of the reasons why they would fail this should
 not be policy violations.  I think the current test is excellent since it
 catches a concrete problem and isn't testing the current wording of policy
 in a vacuum.

 We at some point do need to get back to the discussion about what policy
 should say clean should actually do.

The BIG problem with this loose approach is that modifying a package
and sending in a diff becomes painfull. You build your package and run
diff and you end up with a few 100k lines of diff that are basically
all autogenerated files. The few lines actual change get easily lost
in there.

But I agree with the approach to first find and fix the cases that
FTBFS the second time around. They are the truely evil ones and
already enough work for now.

MfG
Goswin


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-22 Thread Goswin von Brederlow
Andreas Tille [EMAIL PROTECTED] writes:

 On Wed, 16 May 2007, Stefano Zacchiroli wrote:

 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

 I personally store the diff.gz from first build and compare with a
 second build - which is basically the same as your suggestion, but
 perhaps not consuming so much space.

 Kind regards

 Andreas.

Or build binary, clean, build source, debdiff old.dsc new.dsc |
diffstat. That gives a nice output for the bugreport.

MfG
Goswin


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-22 Thread Goswin von Brederlow
Tyler MacDonald [EMAIL PROTECTED] writes:

 Steve Langasek [EMAIL PROTECTED] wrote:
  granted there are things like this, but reproducible builds would be 
  fantastic and well worth the effort.
 If you're talking about byte-for-byte identical builds, then no, that
 would be a tremendous amount of effort for no practical gain.  There's no
 reason to consider it a bug for packages to not be byte-for-byte identical
 between two builds, so why should anyone waste time trying to fix it?

   We should expect that given the same source, headers, and libraries, we
 would get the same bytes out of a build every time. Any deviation from this
 would indicate something different, or erratic. If it doesn't cause
 problems, fine, but I'd raise an eyebrow over it anyway.

   I guess it depends on how anal and pedantic you want to get.

   - Tyler

I think even gcc has some optimizing code with randomness in it. The
randomness is there to avoid deadlocks or just plain guessing a few
times and picking the best.

MfG
Goswin


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-18 Thread Tino Keitel
On Wed, May 16, 2007 at 10:43:34 +0200, Martin Zobel-Helas wrote:
 Hi, 
 
 On Wed May 16, 2007 at 10:11:55 +0200, Stefano Zacchiroli wrote:

[...]

  Wouldn't it be better to unpack a package twice in two different
  directories, build and clean in one dir and then compare the obtained
  tree with the tree available in the other dir?
 
 That would surely be the better solution to catch this policy violation.
 But the way we did it now was the fastest and easiest solution for now,
 and showed already how many packages have no correct working clean
 targets.

This would also find bugs like this:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=405321

A fix for this should be trivial in most cases.

Regards,
Tino


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Stefano Zacchiroli
On Wed, May 16, 2007 at 08:10:44AM +0200, Martin Zobel-Helas wrote:
 as a QA effort the whole archive was rebuilt yesterday to catch
 build-failures, whether a package can be build twice in a row (unpack,
 build, clean, build). We found about 400 packages not having a sane
 clean target. 

Wow, thanks for the effort!

 To cite
 http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules
 
 clean
 
 This must undo any effects that the build and binary targets may
 have had, except that it should leave alone any output files created
 in the parent directory by a run of a binary target.

Isn't twice building too coarse grained to spot actual violation of
this rule?  I mean, packages that fail to build the second time have for
sure garbage left around after the former invocation of clean. But it
is not granted that packages with garbage left around will fail to
build.

Wouldn't it be better to unpack a package twice in two different
directories, build and clean in one dir and then compare the obtained
tree with the tree available in the other dir?

I know, that's cheap talk while you actually provided facts :-), just
take this as a curiosity of mine / suggestion for future tests.

Many thanks again for this!

-- 
Stefano Zacchiroli -*- PhD in Computer Science ... now what?
[EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/
(15:56:48)  Zack: e la demo dema ?/\All one has to do is hit the
(15:57:15)  Bac: no, la demo scema\/right keys at the right time


signature.asc
Description: Digital signature


Re: Building packages twice in a row

2007-05-16 Thread Martin Zobel-Helas
Hi, 

On Wed May 16, 2007 at 10:11:55 +0200, Stefano Zacchiroli wrote:
 On Wed, May 16, 2007 at 08:10:44AM +0200, Martin Zobel-Helas wrote:
  as a QA effort the whole archive was rebuilt yesterday to catch
  build-failures, whether a package can be build twice in a row (unpack,
  build, clean, build). We found about 400 packages not having a sane
  clean target. 
 
 Wow, thanks for the effort!
 
  To cite
  http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules
  
  clean
  
  This must undo any effects that the build and binary targets may
  have had, except that it should leave alone any output files created
  in the parent directory by a run of a binary target.
 
 Isn't twice building too coarse grained to spot actual violation of
 this rule?  I mean, packages that fail to build the second time have for
 sure garbage left around after the former invocation of clean. But it
 is not granted that packages with garbage left around will fail to
 build.

ack.

 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

That would surely be the better solution to catch this policy violation.
But the way we did it now was the fastest and easiest solution for now,
and showed already how many packages have no correct working clean
targets.

We will need to patch sbuild for the changes you suggested. Feel free to
send patches for that.

 I know, that's cheap talk while you actually provided facts :-), just
 take this as a curiosity of mine / suggestion for future tests.

If you provide a patch for sbuild, Lucas will surely hand you the
build-logs for all packages. Happy bug filing then.

 Many thanks again for this!
nP. QA is work everyone of the project can do.

Greetings
Martin

-- 
[EMAIL PROTECTED] /root]# man real-life
No manual entry for real-life


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Santiago Vila
On Wed, 16 May 2007, Stefano Zacchiroli wrote:

 I mean, packages that fail to build the second time have for
 sure garbage left around after the former invocation of clean.

Not always. In some cases (for example, two of my packages) the error
was to modify a .po file in place to update it. The second time
you build the package, dpkg-source complains about the .mo files,
as they are binary files and they have been modified.

How do people deal with this, BTW? Maybe by configuring the package in
another directory and using the VPATH feature of make?


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Andreas Tille

On Wed, 16 May 2007, Stefano Zacchiroli wrote:


Wouldn't it be better to unpack a package twice in two different
directories, build and clean in one dir and then compare the obtained
tree with the tree available in the other dir?


I personally store the diff.gz from first build and compare with a
second build - which is basically the same as your suggestion, but
perhaps not consuming so much space.

Kind regards

Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Adeodato Simó
* Santiago Vila [Wed, 16 May 2007 10:52:02 +0200]:

 Not always. In some cases (for example, two of my packages) the error
 was to modify a .po file in place to update it. The second time
 you build the package, dpkg-source complains about the .mo files,
 as they are binary files and they have been modified.

 How do people deal with this, BTW? Maybe by configuring the package in
 another directory and using the VPATH feature of make?

Deleting the binary files in the clean target. dpkg-source will complain
that they're missing, but will build the package just fine.

I do this also with config.{guess,sub} (and recommend it as well).

Cheers,

-- 
Adeodato Simó dato at net.com.org.es
Debian Developer  adeodato at debian.org
 
Don't ask the barber whether you need a haircut.
-- Daniel S. Greenberg


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Richard Atterer
On Wed, May 16, 2007 at 10:11:55AM +0200, Stefano Zacchiroli wrote:
 Wouldn't it be better to unpack a package twice in two different 
 directories, build and clean in one dir and then compare the obtained 
 tree with the tree available in the other dir?

IMHO, a good test would be to build the package twice and then to compare 
whether the created .debs are identical between the first and second run. 
(Of course, file timestamps would have to be ignored when comparing.)

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Guus Sliepen
On Wed, May 16, 2007 at 11:12:56AM +0200, Richard Atterer wrote:

  Wouldn't it be better to unpack a package twice in two different 
  directories, build and clean in one dir and then compare the obtained 
  tree with the tree available in the other dir?
 
 IMHO, a good test would be to build the package twice and then to compare 
 whether the created .debs are identical between the first and second run. 
 (Of course, file timestamps would have to be ignored when comparing.)

There are lots of reasons why the resulting package can differ each time
you build it, some of them perfectly valid. For example, this is not
uncommon in C programs:

printf(foo version %s (built %s %s)\n, VERSION, __DATE__, __TIME__);

Also, running update-po will always change the header of a .po file to
reflect the last time update-po was run. I don't think we can require
that building a package twice in a row produces exactly the same .deb
and/or .diff.gz.

-- 
Met vriendelijke groet / with kind regards,
  Guus Sliepen [EMAIL PROTECTED]


signature.asc
Description: Digital signature


Re: Building packages twice in a row

2007-05-16 Thread Neil Williams
On Wed, 16 May 2007 10:52:02 +0200 (CEST)
Santiago Vila [EMAIL PROTECTED] wrote:

 On Wed, 16 May 2007, Stefano Zacchiroli wrote:

  I mean, packages that fail to build the second time have for
  sure garbage left around after the former invocation of clean.

 Not always. In some cases (for example, two of my packages) the error
 was to modify a .po file in place to update it.

I'm not sure what you mean - is this autotools where you simply symlink 
po/Makefile.in.in ? What modification are you making? Native or upstream 
package?

 The second time
 you build the package, dpkg-source complains about the .mo files,
 as they are binary files and they have been modified.

But .mo files don't need to be in the upstream source - they can be
cleaned and regenerated with make -C po.

 How do people deal with this, BTW? Maybe by configuring the package in
 another directory and using the VPATH feature of make?

I haven't come across this kind of error in any package (as well as my
own, I'm currently cross-building Debian packages for Emdebian).

--


Neil Williams
=
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/



pgpiOX6CURvok.pgp
Description: PGP signature


Re: Building packages twice in a row

2007-05-16 Thread paddy
On Wed, May 16, 2007 at 11:21:51AM +0200, Guus Sliepen wrote:
 On Wed, May 16, 2007 at 11:12:56AM +0200, Richard Atterer wrote:
 
   Wouldn't it be better to unpack a package twice in two different 
   directories, build and clean in one dir and then compare the obtained 
   tree with the tree available in the other dir?
  
  IMHO, a good test would be to build the package twice and then to compare 
  whether the created .debs are identical between the first and second run. 
  (Of course, file timestamps would have to be ignored when comparing.)
 
 There are lots of reasons why the resulting package can differ each time
 you build it, some of them perfectly valid. For example, this is not
 uncommon in C programs:
 
 printf(foo version %s (built %s %s)\n, VERSION, __DATE__, __TIME__);
 
 Also, running update-po will always change the header of a .po file to
 reflect the last time update-po was run. I don't think we can require
 that building a package twice in a row produces exactly the same .deb
 and/or .diff.gz.

granted there are things like this, but reproducible builds would be 
fantastic and well worth the effort.

Regards,
Paddy



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Norbert Preining
Hi all!

On Mit, 16 Mai 2007, Santiago Vila wrote:
 Not always. In some cases (for example, two of my packages) the error
 was to modify a .po file in place to update it. The second time

I agree. In texinfo I have the following problem
- upstream ships po/*.gmo
- debian patches patch the .po files to fix some translations
- debian rules touches po/texinfo.pot
- make updates the .gmo files and the rest

Now at a second build time we have changes in the binary .gmo files
which cannot be represented.

What is the preferred solution for such a case?

On Mit, 16 Mai 2007, Adeodato Simó wrote:
 Deleting the binary files in the clean target. dpkg-source will complain
 that they're missing, but will build the package just fine.

Sounds like a hack. What do other say?

Best wishes

Norbert

---
Dr. Norbert Preining [EMAIL PROTECTED]Università di Siena
Debian Developer [EMAIL PROTECTED] Debian TeX Group
gpg DSA: 0x09C5B094  fp: 14DF 2E6C 0307 BE6D AD76  A9C0 D2BF 4AA3 09C5 B094
---
OUNDLE (vb.)
To walk along leaning sideways, with one arm hanging limp and dragging
one leg behind the other. Most commonly used by actors in amateur
production of Richard III, or by people carrying a heavy suitcase in
one hand.
--- Douglas Adams, The Meaning of Liff


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Andreas Tille

On Wed, 16 May 2007, Norbert Preining wrote:


Sounds like a hack. What do other say?


There are different opinions about orig.tar.gz should be equal
to upstream.  I tend to the opinion that no precompiled stuff
that can be builded by the source has to be in orig.tar.gz and
in such cases I would think about (mind the carful wording -
I do not explicitely suggest it) rebuilding the orig.tar.gz and
remove *.mo files.  I would definitely remove such files if
there would be other stronger reasons to change upstream
tarball.

Kind regards

  Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Adeodato Simó
* Andreas Tille [Wed, 16 May 2007 13:27:54 +0200]:

 On Wed, 16 May 2007, Norbert Preining wrote:

 Sounds like a hack. What do other say?

 There are different opinions about orig.tar.gz should be equal
 to upstream. 

In case there is confusion, my original suggestion was to remove the
files from debian/rules ('clean' target), not to remove them from the
orig tarball. I don't think this is a hack, but very standard procedure.

Cheers,

-- 
Adeodato Simó dato at net.com.org.es
Debian Developer  adeodato at debian.org
 
 Listening to: Chambao - Luz


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Andreas Tille

On Wed, 16 May 2007, Adeodato [utf-8] Simó wrote:


There are different opinions about orig.tar.gz should be equal
to upstream.


In case there is confusion, my original suggestion was to remove the
files from debian/rules ('clean' target), not to remove them from the
orig tarball. I don't think this is a hack, but very standard procedure.


Well, my wording different opinions about... was no reference to
your posting and I think I understood you right.  I just used this
careful wording because I often observed that people stick to the
upstream version at any price which I see differently.  So I'm aware
that people do not agree with me to remove *.mo files from orig.tar.gz
but I think it would be OK if it solves other problems.

Kind regards

Andreas.

--
http://fam-tille.de


Re: Building packages twice in a row

2007-05-16 Thread Josselin Mouette
Le mercredi 16 mai 2007 à 13:15 +0200, Norbert Preining a écrit :
 On Mit, 16 Mai 2007, Adeodato Simó wrote:
  Deleting the binary files in the clean target. dpkg-source will complain
  that they're missing, but will build the package just fine.
 
 Sounds like a hack. What do other say?

That's definitely the correct solution.

-- 
 .''`.
: :' :  We are debian.org. Lower your prices, surrender your code.
`. `'   We will add your hardware and software distinctiveness to
  `-our own. Resistance is futile.



Re: Building packages twice in a row

2007-05-16 Thread Frank Küster
Andreas Tille [EMAIL PROTECTED] wrote:

 On Wed, 16 May 2007, Adeodato [utf-8] Simó wrote:

 There are different opinions about orig.tar.gz should be equal
 to upstream.

 In case there is confusion, my original suggestion was to remove the
 files from debian/rules ('clean' target), not to remove them from the
 orig tarball. I don't think this is a hack, but very standard procedure.

 Well, my wording different opinions about... was no reference to
 your posting and I think I understood you right.  I just used this
 careful wording because I often observed that people stick to the
 upstream version at any price which I see differently.  So I'm aware
 that people do not agree with me to remove *.mo files from orig.tar.gz
 but I think it would be OK if it solves other problems.

It would be acceptable if it would solve a problem that can't be solved
otherwise.  Here one can just delete them in the clean target, and
therefore I think the orig.tar.gz should *not* be touched because of
this. 

Regards, Frank
-- 
Dr. Frank Küster
Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich
Debian Developer (teTeX/TeXLive)



Re: Building packages twice in a row

2007-05-16 Thread Marcus Better
Norbert Preining wrote:
 Now at a second build time we have changes in the binary .gmo files
 which cannot be represented.
 
 What is the preferred solution for such a case?

I usually save upstream's generated files somewhere in debian/rules during
build, and copy them back in the clean target. It's cumbersome but it
works. Sometimes it's easier to convince the tools to put output in some
other place and not stomp over the upstream generated files.

Marcus



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Lennart Sorensen
On Wed, May 16, 2007 at 08:10:44AM +0200, Martin Zobel-Helas wrote:
 as a QA effort the whole archive was rebuilt yesterday to catch
 build-failures, whether a package can be build twice in a row (unpack,
 build, clean, build). We found about 400 packages not having a sane
 clean target. 
 
 To cite
 http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules
 
 clean
 
 This must undo any effects that the build and binary targets may
 have had, except that it should leave alone any output files created
 in the parent directory by a run of a binary target.

What about packages that automatically pull in updated autoconf files as
part of the build, or regenerate .po files which were already there, but
due to a new version of the tools generates a different .po file from
what was already there?  The result is that doing the build caused the
sources to change and be different from the source when extracted.

Some packages also leave around .orig files due to patches applying but
with offsets or fuzz, which also don't get cleaned up and leave the
sources changed.

Neither of these cases cause build failures, but they are quite annoying
when trying to diff for any changes one may be trying to make to a
package.

I know of at least a few packages that had these issues under Sarge and
I believe also under Etch:

quagga, dhcp3, openswan: Generate changed .po files

ntp: changes autoconf files

grub: changes autoconf files and reruns automake generating new .in files.

It would certainly make life a little easier for me if these kinds of
changes were simply not permitted.  If a package can't be built and
cleaned and end up exactly like it was when extracted, then there is
something wrong with how it builds or how it cleans.

--
Len Sorensen


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Russ Allbery
Martin Zobel-Helas [EMAIL PROTECTED] writes:
 On Wed May 16, 2007 at 10:11:55 +0200, Stefano Zacchiroli wrote:

 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

 That would surely be the better solution to catch this policy violation.

Many, many more packages would fail this and some of us (such as myself)
believe strongly that many of the reasons why they would fail this should
not be policy violations.  I think the current test is excellent since it
catches a concrete problem and isn't testing the current wording of policy
in a vacuum.

We at some point do need to get back to the discussion about what policy
should say clean should actually do.

-- 
Russ Allbery ([EMAIL PROTECTED])   http://www.eyrie.org/~eagle/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Roger Leigh
Martin Zobel-Helas [EMAIL PROTECTED] writes:

 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

 That would surely be the better solution to catch this policy violation.
 But the way we did it now was the fastest and easiest solution for now,
 and showed already how many packages have no correct working clean
 targets.

 We will need to patch sbuild for the changes you suggested. Feel free to
 send patches for that.

Please could you file a wishlist bug, so I don't forget about it?

If it sounds OK, I will add an option to do that.  It should be as
simple as copying the directory after dpkg-source has been run, and
then diffing it after running debian/rules clean at the end of the
build.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?   http://gutenprint.sourceforge.net/
   `-GPG Public Key: 0x25BFB848   Please GPG sign your mail.


pgpr6wJCAhDxR.pgp
Description: PGP signature


Re: Building packages twice in a row

2007-05-16 Thread Armin Berres
On Wed, 16 May 07 11:36, Lennart Sorensen wrote:
 What about packages that automatically pull in updated autoconf files as
 part of the build, or regenerate .po files which were already there, but
 due to a new version of the tools generates a different .po file from
 what was already there?  The result is that doing the build caused the
 sources to change and be different from the source when extracted.
 
 Neither of these cases cause build failures, but they are quite annoying
 when trying to diff for any changes one may be trying to make to a
 package.

I may be wrong, but IIRC removing those generated files in the clean
target is the solution if you want a clean .diff.gz.

/Armin


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Lucas Nussbaum
On 16/05/07 at 10:11 +0200, Stefano Zacchiroli wrote:
 Isn't twice building too coarse grained to spot actual violation of
 this rule?  I mean, packages that fail to build the second time have for
 sure garbage left around after the former invocation of clean. But it
 is not granted that packages with garbage left around will fail to
 build.
 
 Wouldn't it be better to unpack a package twice in two different
 directories, build and clean in one dir and then compare the obtained
 tree with the tree available in the other dir?

As already mentioned by others, you are going to get a lot of false
positives.

An alternative could be to keep the files created by the first build
(*.{deb,diff.gz,orig.tar.gz,dsc,changes}) and compare them with those
from the second build, using debdiff. This would allow to spot the most
obvious problems (e.g missing/added files).
-- 
| Lucas Nussbaum
| [EMAIL PROTECTED]   http://www.lucas-nussbaum.net/ |
| jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F |


signature.asc
Description: Digital signature


Re: Building packages twice in a row

2007-05-16 Thread Lennart Sorensen
On Wed, May 16, 2007 at 07:57:33PM +0200, Armin Berres wrote:
 I may be wrong, but IIRC removing those generated files in the clean
 target is the solution if you want a clean .diff.gz.

But dpkg-buildpackage will then spit out lots of warnings about being
unable to store the deletion of a binary file in the diff.  So it will
look ugly, but work I guess.  diff will show the files as having
disappeared of course, versus leaving them there and dpkg-buildpackage
telling you it can't store the changes to binary files in the diff, and
diff will tell you the binary files changed.

So leaving the regenerated files there or deleting them, both result in
the same amount of noise from diff and dpkg-buildpackage for binary
files.  Saving the files, then restoring them as part of cleaning would
be probably the only way to keep diff and dpkg-buildpackage from making
any noise about changes since it is the only way there aren't changes.

--
Len Sorensen


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Steve Langasek
On Wed, May 16, 2007 at 10:00:57AM +, [EMAIL PROTECTED] wrote:
 On Wed, May 16, 2007 at 11:21:51AM +0200, Guus Sliepen wrote:
  On Wed, May 16, 2007 at 11:12:56AM +0200, Richard Atterer wrote:

Wouldn't it be better to unpack a package twice in two different 
directories, build and clean in one dir and then compare the obtained 
tree with the tree available in the other dir?

   IMHO, a good test would be to build the package twice and then to compare 
   whether the created .debs are identical between the first and second run. 
   (Of course, file timestamps would have to be ignored when comparing.)

  There are lots of reasons why the resulting package can differ each time
  you build it, some of them perfectly valid. For example, this is not
  uncommon in C programs:

  printf(foo version %s (built %s %s)\n, VERSION, __DATE__, __TIME__);

  Also, running update-po will always change the header of a .po file to
  reflect the last time update-po was run. I don't think we can require
  that building a package twice in a row produces exactly the same .deb
  and/or .diff.gz.

 granted there are things like this, but reproducible builds would be 
 fantastic and well worth the effort.

If you're talking about byte-for-byte identical builds, then no, that
would be a tremendous amount of effort for no practical gain.  There's no
reason to consider it a bug for packages to not be byte-for-byte identical
between two builds, so why should anyone waste time trying to fix it?

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
[EMAIL PROTECTED]   http://www.debian.org/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Tyler MacDonald
Steve Langasek [EMAIL PROTECTED] wrote:
  granted there are things like this, but reproducible builds would be 
  fantastic and well worth the effort.
 If you're talking about byte-for-byte identical builds, then no, that
 would be a tremendous amount of effort for no practical gain.  There's no
 reason to consider it a bug for packages to not be byte-for-byte identical
 between two builds, so why should anyone waste time trying to fix it?

  We should expect that given the same source, headers, and libraries, we
would get the same bytes out of a build every time. Any deviation from this
would indicate something different, or erratic. If it doesn't cause
problems, fine, but I'd raise an eyebrow over it anyway.

  I guess it depends on how anal and pedantic you want to get.

- Tyler


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Lars Wirzenius
On ke, 2007-05-16 at 16:26 -0700, Tyler MacDonald wrote:
   We should expect that given the same source, headers, and libraries, we
 would get the same bytes out of a build every time. Any deviation from this
 would indicate something different, or erratic. If it doesn't cause
 problems, fine, but I'd raise an eyebrow over it anyway.

printf(This program was compiled on  __DATE__ \n);

An example like the above has already been given. Build dates and other
variable information gets put into a lot of output files from
compilations.

-- 
The difference between appealing and appalling is very small.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Tyler MacDonald
Lars Wirzenius [EMAIL PROTECTED] wrote:
 printf(This program was compiled on  __DATE__ \n);
 
 An example like the above has already been given. Build dates and other
 variable information gets put into a lot of output files from
 compilations.

  Sorry, I was speaking from an overly selfish point of view, or at least
from the point of view of understanding (or wanting to understand) the code
that you're building. I don't do stuff like that in my code, so if my code
compiled differently twice in a row, I'd raise an eyebrow over that...

- Tyler



Re: Building packages twice in a row

2007-05-16 Thread Russ Allbery
Tyler MacDonald [EMAIL PROTECTED] writes:

 We should expect that given the same source, headers, and libraries, we
 would get the same bytes out of a build every time.

This just isn't how compilers work.  Timestamps are encoded in binaries,
temporary build directories are encoded in debugging information, etc.
Look at all the machinery that gcc goes through to be able to compare two
builds to make sure they're the same.  We don't want to try to maintain
that for every package.

-- 
Russ Allbery ([EMAIL PROTECTED])   http://www.eyrie.org/~eagle/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Re: Building packages twice in a row

2007-05-16 Thread Peter Samuelson

[Lennart Sorensen]
 But dpkg-buildpackage will then spit out lots of warnings about being
 unable to store the deletion of a binary file in the diff.  So it
 will look ugly, but work I guess.

dpkg-buildpackage doesn't store _any_ deletions in the diff.gz - the
warning about deletions has nothing to do with a file being binary or
not.  (It also warns about binary files being _modified_, which is
quite a different matter.)

In my opinion, dpkg-buildpackage should not warn about deleted files at
all, since it is a common and correct practice.  It is much easier and
less error-prone than saving/restoring pristine files, and as such, it
ought to be encouraged, not warned about.

I'd file a bug asking for this, but clearly the warning is intentional,
so a bug is not likely to get much more attention than #12564, which is
also related.


signature.asc
Description: Digital signature


Re: Building packages twice in a row

2007-05-16 Thread Joey Hess
Peter Samuelson wrote:
 I'd file a bug asking for this, but clearly the warning is intentional,
 so a bug is not likely to get much more attention than #12564, which is
 also related.

12564 should be fixable with wig and pen. If it does get fixed then
deleting files in clean will no longer be the simplest and best
approach.

-- 
see shy jo


signature.asc
Description: Digital signature