Your message dated Tue, 24 May 2016 18:34:31 +0000
with message-id <e1b5h9z-0007qv...@franck.debian.org>
and subject line Bug#825092: fixed in grass 7.0.4-2
has caused the Debian Bug report #825092,
regarding grass: please make the build reproducible (fileordering, randomness)
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
825092: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=825092
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Source: grass
Version: 7.0.4-1
Severity: wishlist
Tags: patch upstream
User: reproducible-bui...@lists.alioth.debian.org
Usertags: fileordering randomness
X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org
Control: block -1 by 825088
Dear Maintainer,
While working on the “reproducible builds” effort [1], we have noticed
that 'grass' could not be built reproducibly.
There are several reproducibility issues:
1) File ordering issues - the build result depends on the order of the
files listed with readdir or equivalent.
* in tools/build_modules_xml.py - see patch sort-build-modules-list
* in lib/db/dbmi_base/dbmscap.c (this affects options order in the
html/db.*.html files) - see patch sort-dbmscap that builds an ordered list.
* in include/Make/Vars.make (this affects the order in which object
files are merged) - see patch sort-obj-files
2) Randomness issue: html/colortables/random.png is built using a
pseudo-random generator seeded with build-time value. See patch
srand48_auto-from-SOURCE_DATE_EPOCH that uses the SOURCE_DATE_EPOCH [2]
environment variable (when set) to set a seed from last debian/changelog
entry date.
3) Makefile mistake: from
https://buildd.debian.org/status/fetch.php?pkg=grass&arch=i386&ver=7.0.4-1&stamp=1462121195,
it seems to me that the binary NAD files are not installed properly:
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/prvi
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/hawaii
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/alaska
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/stgeorge
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/FL
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/WO
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/TN
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/stlrnc
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/stpaul
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/conus
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/MD
/usr/bin/install -c -m 644 OBJ.i686-pc-linux-gnu/prvi
/«PKGBUILDDIR»/dist.i686-pc-linux-gnu/etc/proj/nad/WI
The single OBJ.i686-pc-linux-gnu/prvi file is here installed to *all*
/etc/proj/nad files.
See the patch binary-nad-install for a fix.
4) nad2bin issue: nad2bin has unreproducible output (see #825088)
Once these proposed patches are applied (and #825088 fixed), grass can
be built reproducibly in our current experimental framework.
Regards,
Alexis Bienvenüe.
[1] https://wiki.debian.org/ReproducibleBuilds
[2] https://reproducible-builds.org/specs/source-date-epoch/
Description: Sort build modules list
Sort modules in module_items.xml, to make the build reproducible.
Author: Alexis Bienvenüe <p...@passoire.fr>
--- grass-7.0.4.orig/gui/wxpython/tools/build_modules_xml.py
+++ grass-7.0.4/gui/wxpython/tools/build_modules_xml.py
@@ -51,7 +51,7 @@ def parse_modules(fd):
# TODO: what about ms windows? does gtask handle this?
mlist = list(gcore.get_commands()[0])
indent = 4
- for m in mlist:
+ for m in sorted(mlist):
# TODO: get rid of g.mapsets_picker.py
if m == 'g.mapsets_picker.py' or m == 'g.parser':
continue
Description: Sort dbmscap list
To get reproducible results.
Author: Alexis Bienvenüe <p...@passoire.fr>
Index: grass-7.0.4/lib/db/dbmi_base/dbmscap.c
===================================================================
--- grass-7.0.4.orig/lib/db/dbmi_base/dbmscap.c
+++ grass-7.0.4/lib/db/dbmi_base/dbmscap.c
@@ -209,19 +209,22 @@ dbDbmscap *db_read_dbmscap(void)
return list;
}
+static int cmp_entry(dbDbmscap *a, dbDbmscap *b) {
+ return( a->driverName && b->driverName ? strcmp(a->driverName,b->driverName)
: 0 );
+}
+
static void add_entry(dbDbmscap ** list, char *name, char *startup, char
*comment)
{
- dbDbmscap *head, *cur, *tail;
+ /* add an entry to the list, so that the list remains ordered (by
driverName) */
- /* add this entry to the head of a linked list */
- tail = head = *list;
- while (tail && tail->next)
- tail = tail->next;
- *list = NULL;
+ dbDbmscap *head, *cur, *tail;
cur = (dbDbmscap *) db_malloc(sizeof(dbDbmscap));
- if (cur == NULL)
- return; /* out of memory */
+ if (cur == NULL) {
+ *list = NULL;
+ return;
+ /* out of memory */
+ }
cur->next = NULL;
/* copy each item to the dbmscap structure */
@@ -229,11 +232,21 @@ static void add_entry(dbDbmscap ** list,
strcpy(cur->startup, startup);
strcpy(cur->comment, comment);
+ /* find the last entry that is less than cur */
+ tail = head = *list;
+ while (tail && tail->next && cmp_entry(tail->next,cur)<0)
+ tail = tail->next;
+
/* handle the first call (head == NULL) */
- if (tail)
- tail->next = cur;
- else
- head = cur;
+ if (tail && cmp_entry(tail,cur)<0) {
+ /* insert right after tail */
+ cur->next = tail->next;
+ tail->next = cur;
+ } else {
+ /* insert at first position */
+ cur->next = head;
+ head = cur;
+ }
*list = head;
}
Description: Sort object files
to make the build reproducible.
Author: Alexis Bienvenüe <p...@passoire.fr>
--- grass-7.0.4.orig/include/Make/Vars.make
+++ grass-7.0.4/include/Make/Vars.make
@@ -13,10 +13,10 @@ LEX_SOURCES := $(wildcard *.l)
YACC_SOURCES := $(wildcard *.y)
AUTO_OBJS := \
- $(subst .c,.o,$(C_SOURCES)) \
- $(subst .cpp,.o,$(CPP_SOURCES)) \
- $(subst .l,.yy.o,$(LEX_SOURCES)) \
- $(subst .y,.tab.o,$(YACC_SOURCES))
+ $(sort $(subst .c,.o,$(C_SOURCES))) \
+ $(sort $(subst .cpp,.o,$(CPP_SOURCES))) \
+ $(sort $(subst .l,.yy.o,$(LEX_SOURCES))) \
+ $(sort $(subst .y,.tab.o,$(YACC_SOURCES)))
ifndef MOD_OBJS
MOD_OBJS = $(AUTO_OBJS)
Description: Make srand48_auto use SOURCE_DATE_EPOCH if set
If SOURCE_DATE_EPOCH is set, use it to seed the random generator when
G_srand48_auto is called. This helps makeing the build reproducible
(html/random.png)
See https://reproducible-builds.org/specs/source-date-epoch/
Author: Alexis Bienvenüe <p...@passoire.fr>
--- grass-7.0.4.orig/lib/gis/lrand48.c
+++ grass-7.0.4/lib/gis/lrand48.c
@@ -70,7 +70,12 @@ void G_srand48(long seedval)
long G_srand48_auto(void)
{
- unsigned long seed = (unsigned long) getpid();
+ unsigned long seed;
+ char *source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+ if(source_date_epoch) {
+ seed = strtoull(source_date_epoch, NULL, 10);
+ } else {
+ seed = (unsigned long) getpid();
#ifdef HAVE_GETTIMEOFDAY
{
@@ -86,6 +91,7 @@ long G_srand48_auto(void)
seed += (unsigned long) t;
}
#endif
+ }
G_srand48((long) seed);
return (long) seed;
Description: Binary NAD files install
Corrects binary NAD files install rule.
Author: Alexis Bienvenüe <p...@passoire.fr>
--- grass-7.0.4.orig/lib/proj/Makefile
+++ grass-7.0.4/lib/proj/Makefile
@@ -43,7 +43,7 @@ $(FTOL_OBJ): $(OBJDIR)/ftol.o
$(INSTALL) $< $@
endif
-$(NAD_DSTFILES): $(NAD_DIR)/%: $(NAD_BINFILES) | $(NAD_DIR)
+$(NAD_DSTFILES): $(NAD_DIR)/%: $(OBJDIR)/% | $(NAD_DIR)
$(INSTALL_DATA) $< $@
$(NAD_BINFILES): $(OBJDIR)/%: %.lla
--- End Message ---
--- Begin Message ---
Source: grass
Source-Version: 7.0.4-2
We believe that the bug you reported is fixed in the latest version of
grass, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 825...@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Bas Couwenberg <sebas...@debian.org> (supplier of updated grass package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmas...@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Tue, 24 May 2016 20:01:10 +0200
Source: grass
Binary: grass grass-core grass-gui grass-doc grass-dev-doc grass-dev
Architecture: source amd64 all
Version: 7.0.4-2
Distribution: unstable
Urgency: medium
Maintainer: Debian GIS Project <pkg-grass-devel@lists.alioth.debian.org>
Changed-By: Bas Couwenberg <sebas...@debian.org>
Description:
grass - Geographic Resources Analysis Support System (GRASS GIS)
grass-core - GRASS GIS core components
grass-dev - GRASS GIS development files
grass-dev-doc - GRASS GIS Programmers' Manual
grass-doc - GRASS GIS user documentation
grass-gui - GRASS GIS graphical user interfaces
Closes: 825092
Changes:
grass (7.0.4-2) unstable; urgency=medium
.
* Add patches by Alexis Bienvenüe to make the build reproducible.
(closes: #825092)
Checksums-Sha1:
e7df44f8476ee26c3e9917c620e6f203356032dd 2749 grass_7.0.4-2.dsc
abc9618401c26c46d463bea2c16b817bc47f4312 26288 grass_7.0.4-2.debian.tar.xz
db1aa0a4695b30c26c64c222ebd6dcb54fe2521a 10437818
grass-core-dbgsym_7.0.4-2_amd64.deb
23e8f040302d467aab9d500010cf84b1e9c8d96e 7346968 grass-core_7.0.4-2_amd64.deb
2403ef43c4e6896ec5828e9ec7b0e7fbed99cc4e 3538
grass-dev-dbgsym_7.0.4-2_amd64.deb
1ac260f70176a947647f8abf74f212ad115250f0 7407052 grass-dev-doc_7.0.4-2_all.deb
471b15a34a73ead931af4a15ad2f743e41c5a537 196754 grass-dev_7.0.4-2_amd64.deb
5dee06eeb790e3cd68a5daffd47cdcff568f553e 19259360 grass-doc_7.0.4-2_all.deb
4157994f2b8c56cf535f7cceaf8161747a20f4a2 3012454 grass-gui_7.0.4-2_amd64.deb
87c40d72aaa4bd281a3aa905408bf7c9796a989e 18596 grass_7.0.4-2_all.deb
Checksums-Sha256:
0f22639b36d0e99b7bf71db99ae6e4737efb92bb678c5d257877dd9c53c2e529 2749
grass_7.0.4-2.dsc
b07d7a02a2b71265a357dd4025793224d5c0e3c8fb71663804620416623d4029 26288
grass_7.0.4-2.debian.tar.xz
d1d23dde8e1c0228a440267201c59e2bba05c509aa96aeb349433a2b1d22a815 10437818
grass-core-dbgsym_7.0.4-2_amd64.deb
b8d8418c1868647784b3db2d3172da9678979cfe5eac6ed9a77ce7d31458be85 7346968
grass-core_7.0.4-2_amd64.deb
545a54a6028402121da91434d9cf8e3b4263188bee0a4d540c8a6f0221d33f33 3538
grass-dev-dbgsym_7.0.4-2_amd64.deb
dd3fd36588e851be6bb0f9ca8f025099403430481e21f4e270209f4ac8d0668e 7407052
grass-dev-doc_7.0.4-2_all.deb
df3fa2c4fbcde34ff15e1bb3fb5acdf1382f9b9e04df0bd37d1ac3a690815a50 196754
grass-dev_7.0.4-2_amd64.deb
3fb2d0ee903f40a0c996f9abee9911b5ac252bed7188e70d963288218b5f6477 19259360
grass-doc_7.0.4-2_all.deb
3290bb8908a44b3056cbe3c63b1a6d1aad6f3722dc995e23a62bcb97294c0f44 3012454
grass-gui_7.0.4-2_amd64.deb
9f8d20196d69df67f1dccb977fb9f3aa998f6887c4693cfb81a63337b036ab41 18596
grass_7.0.4-2_all.deb
Files:
0ec76ce3fa5c480ea4e8deb40762fb37 2749 science optional grass_7.0.4-2.dsc
8096644e5759ffd189f2c6f723e50e7a 26288 science optional
grass_7.0.4-2.debian.tar.xz
6d96bc5068bdc2185d6ac65d242bf0ac 10437818 debug extra
grass-core-dbgsym_7.0.4-2_amd64.deb
e10483525b611ead6a3c0918ab7ca353 7346968 science optional
grass-core_7.0.4-2_amd64.deb
5c6aa0e2eb32c2734d5c6d4e335dc202 3538 debug extra
grass-dev-dbgsym_7.0.4-2_amd64.deb
1f11688560f945f390f3a33e113ffa82 7407052 doc optional
grass-dev-doc_7.0.4-2_all.deb
fee11503b7a1fc9a4356b71496fe3971 196754 devel optional
grass-dev_7.0.4-2_amd64.deb
f7638a19efea94663a6c73965b304046 19259360 doc optional
grass-doc_7.0.4-2_all.deb
a8a2472396dd0802f5ff38d7a9b60efd 3012454 science optional
grass-gui_7.0.4-2_amd64.deb
b01000d0e02fbf9a1cde48216d9b3177 18596 metapackages optional
grass_7.0.4-2_all.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCgAGBQJXRJwqAAoJEGdQ8QrojUrxzIUP/0ycL3SL5fDIot8V8vt2ps9o
BXGqTt8BB9FhvL6j4AtCX7fH/J9BXnlaSzTr7mEESo9WL06F31NbQ1wsklFk5IIl
Bfx/LyqGt8KgEnnXnaHB1/g48uKmitmOMyhMtQCUuS/Pcf1YahgNxSQmzDWI7ih8
gZbUSCJxqgwDFw1ehAep1CQdPXna+FOpJjpMJhQSMNRvvWGXxlfHMSkOnPjhGQ7B
SZIs1a9VwUuPveD9FoTgNrF1atHQNKGzrbMeq2RpVBOUwqI8GrBf/rHMoJmyl2yy
v+OR7uIdCm1rjEjCPPAUd/HVc9m9WfzGbI/ORdedv/2RetW+fRtJQ0aJsZvHZOiY
tpRhNAlzRZvPVhA4x1lxfj2MqcxcPXOtrqCNq/0lS+8HZfpkXFeBTJFUA5f0oMkJ
7dQX+9pcMlp6/xPNQ0JEBw41PeEDROdiO8V2atSt1dOmbaGbub69Hc9dwiSKZ5ZG
YU5K/qqje8oqLcDPodMYyl2RV1/DeQ8OYa89gnFgeyKo7XuFZGUQNmOcqV3Do6N0
raLeTo2pcOUmGSVNPhHctdVc42ixbxnA/JSqZzVv5U2UDa+BFp0hYGQckGMkMjXF
oM+k5VWrw1fzcmryraR2BgMQlgANIKkwyN55InExENTX+7r8UWVbJwfbI9ZvxocI
LYr5MI1KDSKdZvta+fUp
=IQzF
-----END PGP SIGNATURE-----
--- End Message ---
_______________________________________________
Pkg-grass-devel mailing list
Pkg-grass-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-grass-devel
