Source: latex2html
Version: 2015-debian1-1
Severity: wishlist
Tags: upstream
User: reproducible-builds@lists.alioth.debian.org
Usertags: toolchain timestamps username randomness
X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
Control: block -1 by 827115

Dear Maintainer,

While working on the "reproducible builds" effort [1], we have noticed
that some packages (including latex2html itself) use latex2html in their
building process, leading to the following reproducibility issues :

* keys from the perl hashes are not sorted. See
reproducible-output.patch to sort them and get a reproducible order.
* a timestamp is included in the output. See
honour-SOURCE_DATE_EPOCH.patch to use the SOURCE_DATE_EPOCH environment
variable when set [2]. This way, the timestamps correspond to the
sources date instead of the build date.
* the user name is included in the output. See
suppress-username-from-output.patch to strip it.
* the index keys are not fully ordered in the case cleaned values are
equal. See idx-sort-all.patch

Once these patches are applied, and once https://bugs.debian.org/827115
is fixed, latex2html can be built reproducibly in our current
experimental framework.

Regards,
Alexis Bienvenüe.

[1] https://wiki.debian.org/ReproducibleBuilds
[2] https://reproducible-builds.org/specs/source-date-epoch/

diff -Nru latex2html-2015-debian1/debian/changelog latex2html-2015-debian1/debian/changelog
--- latex2html-2015-debian1/debian/changelog	2016-01-19 19:24:18.000000000 +0100
+++ latex2html-2015-debian1/debian/changelog	2016-06-10 15:20:45.000000000 +0200
@@ -1,3 +1,9 @@
+latex2html (2015-debian1-1.0~reproducible1) UNRELEASED; urgency=medium
+
+  * Reproducible output.
+
+ -- Alexis Bienvenüe <p...@passoire.fr>  Fri, 10 Jun 2016 15:20:45 +0200
+
 latex2html (2015-debian1-1) unstable; urgency=medium
 
   * New upstream release (Closes: #647433)
diff -Nru latex2html-2015-debian1/debian/patches/honour-SOURCE_DATE_EPOCH.patch latex2html-2015-debian1/debian/patches/honour-SOURCE_DATE_EPOCH.patch
--- latex2html-2015-debian1/debian/patches/honour-SOURCE_DATE_EPOCH.patch	1970-01-01 01:00:00.000000000 +0100
+++ latex2html-2015-debian1/debian/patches/honour-SOURCE_DATE_EPOCH.patch	2016-06-10 15:47:57.000000000 +0200
@@ -0,0 +1,22 @@
+Description: Honour SOURCE_DATE_EPOCH
+ Honour the SOURCE_DATE_EPOCH environment variable, to make the output
+ reproducible.
+ See https://reproducible-builds.org/specs/source-date-epoch/
+Author: Alexis Bienvenüe <p...@passoire.fr>
+
+--- latex2html-2015-debian1.orig/latex2html.pin
++++ latex2html-2015-debian1/latex2html.pin
+@@ -15006,7 +15006,12 @@ sub brackets { ($OP, $CP);}
+ 
+ sub get_date {
+     local($format,$order) = @_;
+-    local(@lt) = localtime;
++    local(@lt);
++    if($ENV{SOURCE_DATE_EPOCH}) {
++        @lt = gmtime($ENV{SOURCE_DATE_EPOCH})
++    } else {
++        @lt = localtime;
++    }
+     local($d,$m,$y) = @lt[3,4,5];
+     if ($format =~ /ISO/) {
+ 	sprintf("%4d-%02d-%02d", 1900+$y, $m+1, $d);
diff -Nru latex2html-2015-debian1/debian/patches/idx-sort-all.patch latex2html-2015-debian1/debian/patches/idx-sort-all.patch
--- latex2html-2015-debian1/debian/patches/idx-sort-all.patch	1970-01-01 01:00:00.000000000 +0100
+++ latex2html-2015-debian1/debian/patches/idx-sort-all.patch	2016-06-13 14:49:30.000000000 +0200
@@ -0,0 +1,16 @@
+Description: Sort all index keys
+ Sort index keys, even if they are the same after beeing cleaned, to
+ get a reproducible output.
+Author: Alexis Bienvenüe <p...@passoire.fr>
+
+--- latex2html-2015-debian1.orig/latex2html.pin
++++ latex2html-2015-debian1/latex2html.pin
+@@ -8536,7 +8536,7 @@ sub keysort {
+     # Put alphabetic characters after symbols; already downcased
+     $x =~ s/^([a-z])/~~~$1/;
+     $y =~ s/^([a-z])/~~~$1/;
+-    $x cmp $y;
++    ($x cmp $y) || ($a cmp $b);
+ }
+ 
+ sub index_key_eq {
diff -Nru latex2html-2015-debian1/debian/patches/reproducible-output.patch latex2html-2015-debian1/debian/patches/reproducible-output.patch
--- latex2html-2015-debian1/debian/patches/reproducible-output.patch	1970-01-01 01:00:00.000000000 +0100
+++ latex2html-2015-debian1/debian/patches/reproducible-output.patch	2016-06-13 09:50:57.000000000 +0200
@@ -0,0 +1,260 @@
+Description: Make the output reproducible.
+ Sort perl hash keys to get the output reproducible.
+ See https://wiki.debian.org/ReproducibleBuilds/
+Author: Alexis Bienvenüe <p...@passoire.fr>
+
+Index: latex2html-2015-debian1/latex2html.pin
+===================================================================
+--- latex2html-2015-debian1.orig/latex2html.pin
++++ latex2html-2015-debian1/latex2html.pin
+@@ -1049,7 +1049,7 @@ sub restore_critical_variables {
+     # undef any renewed-commands...
+     # so the new defs are read from %new_command
+     local($cmd,$key,$code);
+-    foreach $key (keys %renew_command) {
++    foreach $key (sort keys %renew_command) {
+ 	$cmd = "do_cmd_$key";
+ 	$code = "undef \&$cmd"; eval($code) if (defined &$cmd);
+ 	if ($@) { print "\nundef \&do_cmd_$cmd failed"}
+@@ -1673,7 +1673,7 @@ sub make_comment {
+ 
+ sub wrap_other_environments {
+     local($key, $env, $start, $end, $opt_env, $opt_start);
+-    foreach $key (keys %other_environments) {
++    foreach $key (sort keys %other_environments) {
+ 	# skip bogus entries
+ 	next unless ($env = $other_environments{$key});
+ 	$key =~ s/:/($start,$end)=($`,$');':'/e;
+@@ -3849,7 +3849,8 @@ sub make_off_line_images {
+         print "\n\n*** LaTeXERROR\n"; return();
+     }
+ 
+-    while ( ($name, $page_num) = each %new_id_map) {
++    for $name (sort keys %new_id_map) {
++        $page_num = $new_id_map{$name};
+ 	# Extract the page, convert and save it
+ 	&extract_image($page_num,$orig_name_map{$page_num});
+     }
+@@ -3952,7 +3953,8 @@ sub make_images {
+ 	    if (s/$PREFIX$img_rx\.new/$PREFIX$1.$IMAGE_TYPE/go);
+     }
+     print "\n *** removing unnecessary images ***\n" if ($VERBOSITY > 1);
+-    while ( ($name, $page_num) = each %id_map) {
++    for $name (sort keys %id_map) {
++        $page_num = $id_map{$name};
+ 	$contents = $latex_body{$name};
+ 	if ($page_num =~ /^\d+\#\d+$/) { # If it is a page number
+ 	    do {		# Extract the page, convert and save it
+@@ -5130,8 +5132,8 @@ sub substitute_meta_cmds {
+     #
+     # Now substitute the new commands and environments:
+     # (must do them all together because of cross definitions)
+-    $new_cmd_rx = &make_new_cmd_rx(keys %new_command);
+-    $new_cmd_no_delim_rx = &make_new_cmd_no_delim_rx(keys %new_command);
++    $new_cmd_rx = &make_new_cmd_rx(sort keys %new_command);
++    $new_cmd_no_delim_rx = &make_new_cmd_no_delim_rx(sort keys %new_command);
+     $new_env_rx = &make_new_env_rx;
+     $new_end_env_rx = &make_new_end_env_rx;
+ #    $new_cnt_rx = &make_new_cnt_rx(keys %new_counter);
+@@ -5140,7 +5142,8 @@ sub substitute_meta_cmds {
+     $new_cmd_or_env_rx =~ s/^ \||\|$//;
+ 
+     print STDOUT "\nnew commands:\n" if ($VERBOSITY > 2);
+-    while (($cmd, $body) = each %new_command) {
++    for $cmd (sort keys %new_command) {
++        $body = $new_command{$cmd};
+ 	unless ($expanded{"CMD$cmd"}++) {
+ 	    print STDOUT ".$cmd " if ($VERBOSITY > 2);
+ 	    $new_command{$cmd} = &expand_body;
+@@ -5150,7 +5153,8 @@ sub substitute_meta_cmds {
+     }
+ 
+     print STDOUT "\nnew environments:\n" if ($VERBOSITY > 2);
+-    while (($cmd, $body) = each %new_environment) {
++    for $cmd (sort keys %new_environment) {
++        $body = $new_environment{$cmd};
+ 	unless ($expanded{"ENV$cmd"}++) {
+ 	    print STDOUT ".$cmd" if ($VERBOSITY > 2);
+ 	    $new_environment{$cmd} = &expand_body;
+@@ -5160,39 +5164,42 @@ sub substitute_meta_cmds {
+ 
+     print STDOUT "\nnew counters and dependencies:\n" if ($VERBOSITY > 2);
+     &clear_mydb("dependent") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %dependent) {
++    for $cmd (sort keys %dependent) {
++        $body = $dependent{$cmd};
+ 	print STDOUT ".($cmd,$body)" if ($VERBOSITY > 2);
+         &write_mydb("dependent", $cmd, $dependent{$cmd});
+     }
+     &clear_mydb("img_style") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %img_style) {
++    for $cmd (sort keys %img_style) {
+         &write_mydb("img_style", $cmd, $img_style{$cmd});
+     }
+ 
+     &clear_mydb("depends_on") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %depends_on) {
++    for $cmd (sort keys %depends_on) {
++        $body = $dependent_on{$cmd};
+ 	print STDOUT ".($cmd,$body)" if ($VERBOSITY > 2);
+         &write_mydb("depends_on", $cmd, $depends_on{$cmd});
+     }
+ 
+ 
+     &clear_mydb("styleID") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %styleID) {
++    for $cmd (sort keys %styleID) {
+         &write_mydb("styleID", $cmd, $styleID{$cmd});
+     }
+ 
+     &clear_mydb("env_style") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %env_style) {
++    for $cmd (sort keys %env_style) {
+         &write_mydb("env_style", $cmd, $env_style{$cmd});
+     }
+     &clear_mydb("txt_style") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %txt_style) {
++    for $cmd (sort keys %txt_style) {
+         &write_mydb("txt_style", $cmd, $txt_style{$cmd});
+     }
+ 
+     print STDOUT "\ntheorem counters:\n" if ($VERBOSITY > 2);
+     &clear_mydb("new_theorem") if ($DEBUG);     #avoids appending to a previous version
+-    while (($cmd, $body) = each %new_theorem) {
++    for $cmd (sort keys %new_theorem) {
++        $body = $new_theorem{$cmd};
+ 	print STDOUT ".($cmd,$body)" if ($VERBOSITY > 2);
+         &write_mydb("new_theorem", $cmd, $new_theorem{$cmd});
+     }
+@@ -6522,7 +6529,7 @@ sub parse_keyvalues {
+ #	    s/(^|,)\s*([a-zA-Z]+)\s*\=\s*(\"([^"]*)\"|\'([^\']*)\'|([#%&@;:+-\/\w\d]*))\s*/
+ 	    s/(^|,)\s*([a-zA-Z]+)\s*\=\s*(\"([^"]*)\"|\'([^\']*)\'|([^<>,=\s]*))\s*/
+ 		$attributes{$2}=($4?$4:($5?$5:$6));' '/eg;
+-	    foreach $key (keys %attributes){ 
++	    foreach $key (sort keys %attributes){ 
+ 		$KEY = $key;
+ 		$KEY =~ tr/a-z/A-Z/;
+ 		if ($taglist =~ /,$KEY,/i) {	        
+@@ -6564,7 +6571,7 @@ sub parse_keyvalues {
+ 	# with no tags provided, just list the key-value pairs
+ 	$_ = $saved;
+ 	s/\s*(\w+)\s*=\s*\"?(\w+)\"?\s*,?/$attributes{$1}=$2;''/eg;
+-	foreach $key (keys %attributes){ 
++	foreach $key (sort keys %attributes){ 
+ 	    $KEY = $key;
+ 	    $KEY =~ tr/a-z/A-Z/;
+ 	    $atts = $attributes{$key};
+@@ -6633,7 +6640,7 @@ sub extract_attributes {
+ 	if ($$name) { $taglist = $$name }
+     }
+     s/\s*(\w+)\s*=\s*\"?(\w+)\"?\s*,?/$attributes{$1}=$2;''/eg;
+-    foreach $key (keys %attributes){ 
++    foreach $key (sort keys %attributes){ 
+ 	if ($taglist =~ /\,$key\,/) {
+ 	    $attribs .= " $key=\"$attributes{$key}\"";
+ 	    &write_warnings("valid attribute $key for $tag\n");
+@@ -7197,7 +7204,8 @@ TD.eqno			{ }	/* equation-number cells *
+ EOF
+     }
+     print "\n *** Adding document-specific styles *** ";
+-    while (($env,$style) = each %env_style) {
++    for $env (sort keys %env_style) {
++        $style = $env_style{$env};
+         if ($env =~ /\./) {
+             $env =~ s/\.$//;
+             print STYLESHEET "$env\t\t{ $style }\n";
+@@ -7213,10 +7221,12 @@ EOF
+             print STYLESHEET "DIV.$env\t\t{ $style }\n";
+         }
+     }
+-    while (($env,$style) = each %txt_style) {
++    for $env (sort keys %txt_style) {
++        $style = $txt_style{$env};
+         print STYLESHEET "SPAN.$env\t\t{ $style }\n";
+     }
+-    while (($env,$style) = each %img_style) {
++    for $env (sort keys %img_style) {
++        $style = $img_style{$env};
+         print STYLESHEET "IMG.$env\t\t{ $style }\n";
+     }
+ 
+@@ -8832,8 +8842,9 @@ sub replace_cite_marks {
+     #
+     #RRM: Associate the cite_key with  $citefile , for use by other segments.
+     if ($citefile) {
+-	local($cite_key, $cite_ref);
+-	while (($cite_key, $cite_ref) = each %cite_info) {
++        local($cite_key, $cite_ref);
++        for $cite_key (sort keys %cite_info) {
++            $cite_ref = $cite_info{$cite_key};
+ 	    if ($ref_files{'cite_'."$cite_key"} ne $citefile) {
+ 		$ref_files{'cite_'."$cite_key"} = $citefile;
+ 		$changed = 1; }
+@@ -9802,7 +9813,7 @@ sub replace_word {
+ # for use in regular expressions;
+ sub get_current_sections {
+     local($_, $key);
+-    foreach $key (keys %section_commands) {
++    foreach $key (sort keys %section_commands) {
+ 	if ($key =~ /star/) {
+ 	    $_ = $key . "|" . $_}
+ 	else {
+@@ -10220,8 +10231,9 @@ sub save_array_in_file {
+ 	} else {
+ 	    print FILE "# LaTeX2HTML $TEX2HTMLVERSION\n";
+ 	    print FILE "# Associate $type original text with physical files.\n\n";
+-	}
+-	while (($uutxt,$file) = each %array) {
++        }
++        for $uutxt (sort keys %array) {
++            $file = $array{$uutxt};
+ 	    $uutxt =~ s|/|\\/|g;
+ 	    $uutxt =~ s|\\\\/|\\/|g;
+ 
+@@ -10676,7 +10688,8 @@ sub do_cmd_mbox {
+ sub generate_declaration_subs {
+     local($key, $val, $pre, $post, $code );
+     print "\n *** processing declarations ***\n";
+-    while ( ($key, $val) = each %declarations) {
++    for $key (sort keys %declarations) {
++        $val = $declarations{$key};
+ 	if ($val) {
+ 	    ($pre,$post) = ('','');
+ 	    $val =~ m|</.*$|;
+@@ -10698,7 +10711,8 @@ sub generate_declaration_subs {
+ # *Generates* subroutines to handle each of the sectioning commands.
+ sub generate_sectioning_subs {
+     local($key, $val, $cmd, $body);
+-    while ( ($key, $val) = each %standard_section_headings) {
++    for $key (sort keys %standard_section_headings) {
++        $val = $standard_section_headings{$key};
+ 	$numbered_section{$key} = 0;
+ 	eval "sub do_cmd_$key {"
+ 	    . 'local($after,$ot) = @_;'
+@@ -13329,7 +13343,7 @@ sub do_cmd_textohtmlindex {
+ # when using  makeidx.perl
+ sub make_index_labels {
+     local($key, @keys);
+-    @keys = keys %index_labels;
++    @keys = sort keys %index_labels;
+     foreach $key (@keys) {
+ 	if (($ref_files{$key}) && !($ref_files{$key} eq "$idxfile")) {
+ 	    local($tmp) = $ref_files{$key};
+@@ -13345,7 +13359,7 @@ sub make_preindex { &make_real_preindex
+ sub make_real_preindex {
+     local($key, @keys, $head, $body);
+     $head = "<HR>\n<H4>Legend:</H4>\n<DL COMPACT>";
+-    @keys = keys %index_segment;
++    @keys = sort keys %index_segment;
+     foreach $key (@keys) {
+ 	local($tmp) = "segment$key";
+ 	$tmp = $ref_files{$tmp};
+@@ -16777,7 +16791,7 @@ sub addto_languages {
+ sub make_raw_arg_cmd_rx {
+     # $1 or $2 : commands to be processed in latex (with arguments untouched)
+     # $4 : delimiter
+-    $raw_arg_cmd_rx = &make_new_cmd_rx(keys %raw_arg_cmds);
++    $raw_arg_cmd_rx = &make_new_cmd_rx(sort keys %raw_arg_cmds);
+     $raw_arg_cmd_rx;
+ }
+ 
diff -Nru latex2html-2015-debian1/debian/patches/series latex2html-2015-debian1/debian/patches/series
--- latex2html-2015-debian1/debian/patches/series	2016-01-19 19:15:15.000000000 +0100
+++ latex2html-2015-debian1/debian/patches/series	2016-06-13 14:48:36.000000000 +0200
@@ -2,3 +2,7 @@
 debian-install.patch
 perl5.22-defined-array.patch
 perl5.22-unescaped-left-braces.patch
+reproducible-output.patch
+honour-SOURCE_DATE_EPOCH.patch
+suppress-username-from-output.patch
+idx-sort-all.patch
diff -Nru latex2html-2015-debian1/debian/patches/suppress-username-from-output.patch latex2html-2015-debian1/debian/patches/suppress-username-from-output.patch
--- latex2html-2015-debian1/debian/patches/suppress-username-from-output.patch	1970-01-01 01:00:00.000000000 +0100
+++ latex2html-2015-debian1/debian/patches/suppress-username-from-output.patch	2016-06-10 15:50:45.000000000 +0200
@@ -0,0 +1,25 @@
+Description: Strip username from output,
+ to make the output reproducible.
+ See https://reproducible-builds.org/
+Author: Alexis Bienvenüe <p...@passoire.fr>
+
+--- latex2html-2015-debian1.orig/latex2html.pin
++++ latex2html-2015-debian1/latex2html.pin
+@@ -186,7 +186,7 @@ $PARTITION_PREFIX = 'part_' unless $PART
+ 
+ # Author address
+ @address_data = &address_data('ISO');
+-$ADDRESS = "$address_data[0]\n$address_data[1]";
++$ADDRESS = "$address_data[1]";
+ 
+ # ensure non-zero defaults
+ $MAX_SPLIT_DEPTH = 4 unless ($MAX_SPLIT_DEPTH);
+@@ -14088,7 +14088,7 @@ sub default_textohtmlinfopage {
+ 	, "<STRONG>latex2html</STRONG> <TT>$argv</TT>\n"
+ 	, (($SHOW_INIT_FILE && ($INIT_FILE ne ''))?
+ 	   "\n<P>with initialization from: <TT>$INIT_FILE</TT>\n$init_file_mark\n" :'')
+-	, "<P>The translation was initiated by $address_data[0] on $address_data[1]"
++	, "<P>The translation was initiated on $address_data[1]"
+ 	, $open_all, $_)
+       : join('', $close_all, "$INFO\n", $open_all, $_));
+     $_;
_______________________________________________
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Reply via email to