Bug#737389: Bug#737202: devscripts-el: fails to upgrade from 'wheezy': ERROR: install script from devscripts-el package failed (during Install devscripts-el for emacs23)

2014-05-27 Thread Agustin Martin 
On Fri, May 23, 2014 at 05:38:28PM +0200, Agustin Martin wrote:
 I have been wondering for some time if there is a simpler way to handle all
 this sorting without using explicit tsort.
 
 I have been playing for some time with a proof of concept of a perl-only
 method to handle this, and tried today to put it into emacsen-common lib.pl. 
 I am attaching current incarnation of it, both as a plain lib.pl and as a
 git diff. I have tested in my system and it seems to work well, but consider
 it highly experimental code.

Hi, Rob

Noticed some problems with my previous sort function, related to
auto-vivification in deep perl hashes of hashes when deep keys are tested
and parent key does not exist. 

I think this is fixed now, so I am attaching yet another version of lib.pl.
It also uses a simpler dependencies hash and properly names function
`generate_add_on_install_list' instead of `generate_add_on_install_list_new'.
Needs further testing, but apparently works.

Attached is also the file I am using to test that ./lib.pl.

Hope it helps,

-- 
Agustin
#!/usr/bin/perl -w

use strict;
use Cwd;

my $debug++ if $ENV{'EMACSEN_COMMON_DEBUG'};

# depends on: dpkg, tsort, perl

my $lib_dir = /usr/lib/emacsen-common;
my $var_dir = /var/lib/emacsen-common;

$::installed_package_state_dir = ${var_dir}/state/package/installed;
$::installed_flavor_state_dir = ${var_dir}/state/flavor/installed;

sub ex
{
  my(@cmd) = @_;
  if(system(@cmd) != 0)
  {
die join( , @cmd) .  failed;
  }
}

sub glob_in_dir
{
  my ($dir, $pattern) = @_;
  my $oldir = getcwd;
  chdir($dir) or die chdir $dir: $!;
  my @files = glob(*[!~]);
  chdir($oldir);
  return \@files;
}

sub validate_add_on_pkg
{
  my ($pkg, $script, $old_invocation_style) = @_;
  if($old_invocation_style)
  {
if(-e $lib_dir/packages/compat/$pkg)
{
  print STDERR ERROR: $pkg is broken - called $script as an old-style add-on, but has compat file.\n;
  #exit(1);
}
  }
  else # New invocation style.
  {
unless(-e $lib_dir/packages/compat/$pkg)
{
  print STDERR ERROR: $pkg is broken - called $script as a new-style add-on, but has no compat file.\n;
  #exit(1);
}
  }
}

sub get_installed_add_on_packages
{
  # Return all of the old format packages, plus all of the new-format
  # packages that are ready (i.e. have a state/installed file).  In
  # this case ready means ready for compilation.
  my $all_pkgs = glob_in_dir($lib_dir/packages/install, '*[!~]');
  my $new_format_pkgs = glob_in_dir($lib_dir/packages/compat, '*[!~]');
  my %ready_pkgs = map { $_ = 1 } @$all_pkgs;
  for my $p (@$new_format_pkgs)
  {
delete $ready_pkgs{$p} unless (-e $::installed_package_state_dir/$p);
  }
  return \%ready_pkgs;
}

sub get_installed_flavors
{
  my $flavors = glob_in_dir($::installed_flavor_state_dir, '*[!~]');
  return @$flavors;
}

# 
sub generate_add_on_install_list {
# 
# generate_add_on_install_list \@packages_to_sort
# generate_add_on_install_list \%packages_to_sort
# 
  my $packages_to_sort  = shift;
  my $installed_add_ons = get_installed_add_on_packages;
  my %depends_hash  = ();

  return unless $packages_to_sort;

  my $packages_to_sort_string = join(' ',@$packages_to_sort);
  my $dpkg_query_output = `dpkg-query -W -f='package:\${Package}, \${Depends}\n' $packages_to_sort_string`;
  die 'emacsen-common: dpkg-query invocation failed' unless ($? == 0);

  if ( $debug ){
print --\n;
print Packages to sort:\n$packages_to_sort_string\n;
print -\n;
print dpkg-query output:\n---\n$dpkg_query_output---\n;
print -\n;
print Installed add-ons:\n, join(', ',sort keys %{$installed_add_ons}), \n;
print -\n;
  }

  foreach my $dpkg_query_line ( split(\n, $dpkg_query_output) ){
my @package_depends = split(/[,|]/, $dpkg_query_line);
my $package = shift @package_depends;

# Remove consistency string or ignore line if missing.
next unless $package =~ s/^package://;

# Filter out all the noise (version number dependencies, etc)
@package_depends = map { /\s*(\S+)/o; $1; } @package_depends;

foreach my $dependency ( @package_depends ){
  # dpkg-query regexp above will result in empty dependency for
  # packages with no dependencies at all. Discard if so.
  next unless $dependency;

  # Filter out dependencies on non-add-on packages.
  next unless ( defined $installed_add_ons-{$dependency} );

  # Populate the dependencies hash for this package
  $depends_hash{$package}{$dependency}++;
}
  }

  if ( $debug ){
# Show packages without

Bug#737389: Bug#737202: devscripts-el: fails to upgrade from 'wheezy': ERROR: install script from devscripts-el package failed (during Install devscripts-el for emacs23)

2014-05-23 Thread Agustin Martin 
[Resending to actual emacsen-common bug, not to ancient devscripts-el bug. 
Sorry for the noise ]

On Sun, Feb 02, 2014 at 11:22:09AM +, Julian Gilbey wrote:
 
 Hi Rob,
 
 Please do have a look at this one - it's messy :-/
 
 On Fri, Jan 31, 2014 at 09:57:08PM +0900, Tatsuya Kinoshita wrote:
  On January 31, 2014 at 11:33AM +0100, anbe (at debian.org) wrote:
 Install emacsen-common for emacs23
  [...]
 install/devscripts-el: Handling emacs23, logged in /tmp/elc_tCbg9i.log
 ERROR: install script from devscripts-el package failed
  [...]
   devscripts.el:19:1:Error: Cannot open load file: mcharset
  
  It seems apel's mcharset.elc is not found when byte-compiling.
  
  The devscripts-el package depends on apel, so install/devscripts-el
  called from devscripts-el.postinst succeed, but install/devscripts-el
  called from emacsen-common or emacsen could fail as above.
 
 I've looked into this one.  I've applied a temporary patch to
 devscripts-el to avoid compiling if apel is not compiled, but it seems
 to me that the source of the breakage is in emacsen-common's handling
 of dependencies.
 
 I thought I had figured out what was going on, but now I have no idea:
 I simply don't understand why emacs-install was attempting to
 byte-compile devscripts-el when it had been unpackage earlier on
 during the upgrade (and hence run the emacs-package-remove script).
 The only thing I can think is that maybe the wheezy version of
 emacsen-common's scripts didn't correctly remove the installed marker
 file.

Hi,

I have been wondering for some time if there is a simpler way to handle all
this sorting without using explicit tsort.

I have been playing for some time with a proof of concept of a perl-only
method to handle this, and tried today to put it into emacsen-common lib.pl. 
I am attaching current incarnation of it, both as a plain lib.pl and as a
git diff. I have tested in my system and it seems to work well, but consider
it highly experimental code.

To get dependencies it relies in a single 'dpkg-query' call for all add-ons
to be processed. That dependencies are processed and info put into a
dependencies hash, used to sort after its contents by perl sort function.

To make things more compact and integrated with my tests I have merged some
tightly related functions into a single `generate_add_on_install_list'
function. In my current tests, `get_installed_add_on_packages' has been
changed to return a hash reference, and `generate_add_on_install_list'
changed to accept references to either hash or array.

If `emacsen-common' is one of the packages to be processed, it will always
be put first (new dependency should have the same result), then packages
having no other dependencies, and then packages depending on other of the
add-ons to be installed.

To enable debugging output, just set

export EMACSEN_COMMON_DEBUG=1

As said, this is to be considered highly experimental, but I hope it can be
useful. Please have mercy on my perl :-).

Regards,

-- 
Agustin
From 12c1ce80f1a86ff27c77f3a7e81cb30761c72d7f Mon Sep 17 00:00:00 2001
From: Agustin Martin Domingo agmar...@debian.org
Date: Fri, 23 May 2014 16:59:35 +0200
Subject: [PATCH] lib.pl: First cut for using a perl-only based sorting
 algorithm.

 * Make more use of hashes.
 * Use a single dpkg-query call for all add-ons to be installed.
 * Use a perl-only sorting algorithm.
 * Merged sone interrelated functions into a single function.
---
 lib.pl | 185 +++--
 1 file changed, 88 insertions(+), 97 deletions(-)
 mode change 100755 = 100644 lib.pl

diff --git a/lib.pl b/lib.pl
old mode 100755
new mode 100644
index f0aa670..2d0097e
--- a/lib.pl
+++ b/lib.pl
@@ -3,6 +3,8 @@
 use strict;
 use Cwd;
 
+my $debug++ if $ENV{'EMACSEN_COMMON_DEBUG'};
+
 # depends on: dpkg, tsort, perl
 
 my $lib_dir = /usr/lib/emacsen-common;
@@ -63,8 +65,7 @@ sub get_installed_add_on_packages
   {
 delete $ready_pkgs{$p} unless (-e $::installed_package_state_dir/$p);
   }
-  my @result = keys %ready_pkgs;
-  return \@result;
+  return \%ready_pkgs;
 }
 
 sub get_installed_flavors
@@ -73,110 +74,100 @@ sub get_installed_flavors
   return @$flavors;
 }
 
-sub get_package_status
-{
-  my($pkg) = @_;
-  my $status = `dpkg --status $pkg`;
-  die 'emacsen-common: dpkg invocation failed' if($? != 0);
-  $status =~ s/\n\s+//gmo; # handle any continuation lines...
-  return $status;
-}
-
-sub filter_depends
-{
-  my($depends_string, $installed_add_ons) = @_;
-
-  # Filter out all the noise (version number dependencies, etc)
-  # and handle or deps too, i.e. Depends: foo, bar | baz 
-  my @relevant_depends = split(/[,|]/, $depends_string);
-  @relevant_depends = map { /\s*(\S+)/o; $1; } @relevant_depends;
-
-  # Filter out all non-add-on packages.
-  @relevant_depends = grep {
-my $candidate = $_;
-grep { $_ eq $candidate } @$installed_add_ons;
-  } @relevant_depends;
-  
-  return @relevant_depends;
-}
+#

Bug#737202: devscripts-el: fails to upgrade from 'wheezy': ERROR: install script from devscripts-el package failed (during Install devscripts-el for emacs23)

2014-02-02 Thread Julian Gilbey 
clone 737202 -1
reassign -1 emacsen-common 2.0.7
retitle -1 emacsen-common: incorrectly handles dependencies during emacsXX 
upgrade
thanks

Hi Rob,

Please do have a look at this one - it's messy :-/

On Fri, Jan 31, 2014 at 09:57:08PM +0900, Tatsuya Kinoshita wrote:
 On January 31, 2014 at 11:33AM +0100, anbe (at debian.org) wrote:
Install emacsen-common for emacs23
 [...]
install/devscripts-el: Handling emacs23, logged in /tmp/elc_tCbg9i.log
ERROR: install script from devscripts-el package failed
 [...]
  devscripts.el:19:1:Error: Cannot open load file: mcharset
 
 It seems apel's mcharset.elc is not found when byte-compiling.
 
 The devscripts-el package depends on apel, so install/devscripts-el
 called from devscripts-el.postinst succeed, but install/devscripts-el
 called from emacsen-common or emacsen could fail as above.

I've looked into this one.  I've applied a temporary patch to
devscripts-el to avoid compiling if apel is not compiled, but it seems
to me that the source of the breakage is in emacsen-common's handling
of dependencies.

I thought I had figured out what was going on, but now I have no idea:
I simply don't understand why emacs-install was attempting to
byte-compile devscripts-el when it had been unpackage earlier on
during the upgrade (and hence run the emacs-package-remove script).
The only thing I can think is that maybe the wheezy version of
emacsen-common's scripts didn't correctly remove the installed marker
file.

Anyway, I'm uploading a version of devscripts-el which patches around
this bug, but it would be nice if it could be fixed properly.  This
may be by using the new code in the emacs policy (about packages
handling their own touch/rm of the installed marker file), but not
quite understanding what happened, I'm not convinced that this will
solve it.

   Julian


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#737202: devscripts-el: fails to upgrade from 'wheezy': ERROR: install script from devscripts-el package failed (during Install devscripts-el for emacs23)

2014-01-31 Thread Tatsuya Kinoshita 
On January 31, 2014 at 11:33AM +0100, anbe (at debian.org) wrote:
   Install emacsen-common for emacs23
[...]
   install/devscripts-el: Handling emacs23, logged in /tmp/elc_tCbg9i.log
   ERROR: install script from devscripts-el package failed
[...]
 devscripts.el:19:1:Error: Cannot open load file: mcharset

It seems apel's mcharset.elc is not found when byte-compiling.

The devscripts-el package depends on apel, so install/devscripts-el
called from devscripts-el.postinst succeed, but install/devscripts-el
called from emacsen-common or emacsen could fail as above.

To fix this bug, please skip byte-compilation in install/devscripts-el
if the depending file is not yet prepared.

e.g.
if [ ! -e /usr/share/$FLAVOR/site-lisp/apel/mcharset.elc ]; then exit 0; 
fi

Thanks,
--
Tatsuya Kinoshita


pgpHJuTdVRjKb.pgp
Description: PGP signature