On Sat, 10 Jun 2006, Raphael Hertzog wrote:
> > I appreciate your work, but I am not comfortable supporting two
> > competing implementations in debhelper.
>
> Please reconsider. If you don't integrate that version now, we'll most
> probably do the switch to python2.4 in the next days and we'll replace
> dh_python calls by dh_python2 calls (integrated for example in the python
> package) or we'll remove call to dh_python in favor of dh_pycentral (existing
> and in python-central) / dh_pythonsupport (not existing but quick to
> create).
After discussing with doko and vorlon, I decided to change dh_python to
respect your wish (it's understandable).
Please find a new dh_python attached. This one supports neither
python-central nor python-support. It does however all the substvars
handling required by the new Python policy and is still compatible with
the old policy.
Please integrate that one.
I have written dh_pycentral and dh_pysupport which will take care of the
byte-compilation of the modules and integration with the respective tools.
Those should be integrated in their respective packages (after review by
their maintainer).
Cheers,
--
Raphaël Hertzog
Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
#!/usr/bin/perl -w
=head1 NAME
dh_python - calculates python dependencies and adds postinst and prerm python
scripts
=cut
use strict;
use File::Find;
use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
B<dh_python> [S<I<debhelper options>>] [B<-n>] [B<-V> I<version>] [S<I<module
dirs ...>>]
=head1 DESCRIPTION
dh_python is a debhelper program that is responsible for generating the
${python:Depends} substitutions and adding them to substvars files. It
will also add a postinst and a prerm script if required.
The program will look at python scripts and modules in your package, and
will use this information to generate adequate dependencies. There is two
scenarios: if the package uses the XS-Python-Version field then
the new policy will be applied, otherwise the old policy will be used.
=head2 New policy
The XS-Python-Version field (on the source package) defines which Python
versions are supported by the package. It can be "all", "current",
"current, >= X.Y", a single version ("2.3") or a list of versions with
optional comparison operators (ex: "2.3, 2.4" or ">= 2.2, << 2.5" or
">= 2.4").
The binary packages should have a "XB-Python-Version: ${python:Versions}"
field and dh_python will generate the right substvar for that. The
resulting value can be "all" for public modules which work with all python
versions, "current" for private modules which are always byte-compiled
with the current python version or a list of of all versions for which the
extensions have been compiled (ex: "2.3, 2.4"). The dependencies are
adjusted accordingly as well.
Packages with public extensions should also have a "Provides:
${python:Provides}" field. The corresponding substvar will indicate
"pythonX.Y-foo, pythonX.Z-foo" according to all the extensions
effectively available in the package.
=head2 Old policy
It looks at scripts and modules in your package and adds a dependency on
python, with the
current major version, or on pythonX.Y if your scripts or modules need a
specific python version. The dependency will be substituted into your
package's control file wherever you place the token "${python:Depends}".
If some modules need to be byte-compiled at install time, appropriate
postinst and prerm scripts will be generated. If already byte-compiled
modules are found, they are removed.
If you use this program, your package should build-depend on python.
Note: in the old policy, /usr/lib/site-python is also scanned for modules.
=head1 OPTIONS
=over 4
=item I<module dirs>
If your package installs python modules in non-standard directories, you
can make dh_python check those directories by passing their names on the
command line. By default, it will check /usr/lib/$PACKAGE, /usr/share/$PACKAGE,
/usr/lib/games/$PACKAGE,
/usr/share/games/$PACKAGE and /usr/lib/python?.?/site-packages.
Note: only /usr/lib/python?.?/site-packages and the
extra names on the command line are searched for binary (.so) modules.
=item B<-V> I<version>
If the .py files your package ships are meant to be used by a specific
pythonX.Y version, you can use this option to specify the desired version,
such as 2.3. Do not use if you ship modules in /usr/lib/site-python.
With the new policy, this option is mostly deprecated. Use the
XS-Python-Field to indicate that you're using a specific python version.
=item B<-n>, B<--noscripts>
Do not modify postinst/postrm scripts.
=back
=head1 CONFORMS TO
Debian policy, version 3.5.7
Python policy, version 0.3.7
=cut
init();
my $python = 'python';
# The current python major version
my $python_major;
my $python_version = `$python -V 2>&1`;
if (! defined $python_version || $python_version eq "") {
error("Python is not installed, aborting. (Probably forgot to
Build-Depend on python.)");
}
elsif ($python_version =~ m/^Python\s+(\d+)\.(\d+)(\.\d+)*/) {
$python_version = "$1.$2" ;
$python_major = $1 ;
} else {
error("Unable to parse python version out of \"$python_version\".");
}
# The next python version
my $python_nextversion = next_minor_version($python_version);
my $python_nextmajor = $python_major + 1;
my @python_allversions = ('1.5','2.1','2.2','2.3','2.4','2.5');
foreach (@python_allversions) {
s/^/python/;
}
# Check for -V
my $usepython = "python$python_version";
if($dh{V_FLAG_SET}) {
$usepython = $dh{V_FLAG};
$usepython =~ s/^/python/;
if (! grep { $_ eq $usepython } @python_allversions) {
error("Unknown python version $dh{V_FLAG}");
}
}
# Cleaning the paths given on the command line
foreach (@ARGV) {
s#/$##;
s#^/##;
}
# Check if we have an XS-Python-Version and extract it from the control
# file, also check if we use python-central
my $python_header = "";
{
local $/ = ""; # Grab until empty line
open(CONTROL, "debian/control"); # Can't fail, dh_testdir has already
been called
my $source = <CONTROL>;
$python_header = $1 if ($source =~ m/^XS-Python-Version: \s*(.*)$/m);
chomp($python_header);
}
# dependency types
use constant PROGRAM => 1;
use constant PY_MODULE => 2;
use constant PY_MODULE_NONSTANDARD => 4;
use constant SO_MODULE => 8;
use constant SO_MODULE_NONSTANDARD => 16;
foreach my $package (@{$dh{DOPACKAGES}}) {
my $tmp = tmpdir($package);
my @dirs = ("usr/lib/$package", "usr/share/$package",
"usr/lib/games/$package", "usr/share/games/$package", @ARGV );
my @dirs_so = (@ARGV);
my $dep_on_python = 0;
my $strong_dep = 0;
# Fail early if the package use usr/lib/site-python
if (-d "$tmp/usr/lib/site-python") {
if ($python_header) {
# Error only on new policy
error("The package $package puts files in
/usr/lib/site-python: forbidden by policy");
} else {
# Old policy allowed that directory, so scan it
push @dirs, "usr/lib/site-python";
push @dirs_so, "usr/lib/site-python";
}
}
# First, the case of python-foo and pythonX.Y-foo
if ($package =~ /^python-/) {
$dep_on_python = 1;
$strong_dep = 1;
# This adds a dependency to python<current>-foo in python-foo
# This is old policy but is done only if pythonX.Y-foo exists
# it does no harm and assure backwards compatibility
my $pack = $package;
$pack =~ s/^python/python$python_version/;
if (grep { "$_" eq "$pack" } getpackages()) {
addsubstvar($package, "python:Depends", $pack);
}
}
@dirs = grep -d, map "$tmp/$_", @dirs;
@dirs_so = grep -d, map "$tmp/$_", @dirs_so;
my $deps = 0;
my %verdeps = ();
foreach (@python_allversions) {
$verdeps{$_} = 0;
}
# Find scripts
find sub {
return unless -f and (-x or /\.py$/);
local *F;
return unless open F, $_;
if (read F, local $_, 32 and
m%^#!\s*/usr/bin/(env\s+)?(python(\d+\.\d+)?)\s%) {
if ( "python" eq $2 ) {
$deps |= PROGRAM;
} elsif(defined $verdeps{$2}) {
$verdeps{$2} |= PROGRAM;
}
}
close F;
}, $tmp;
# Look for python modules
my $dirlist="";
if (@dirs) {
foreach my $curdir (@dirs) {
my $has_module = 0;
$curdir =~ s%^$tmp/%%;
find sub {
return unless -f;
if (/\.py$/) {
$has_module = 1;
doit(("rm","-f",$_."c",$_."o"));
}
}, "$tmp/$curdir" ;
if ($has_module) {
if ($dh{V_FLAG_SET}) {
$verdeps{$usepython} |=
PY_MODULE_NONSTANDARD;
} else {
$deps |= PY_MODULE;
}
$dirlist="$dirlist /$curdir";
}
}
}
if (@dirs_so) {
foreach my $curdir (@dirs_so) {
my $has_module = 0;
$curdir =~ s%^$tmp/%%;
find sub {
return unless -f;
$has_module = 1 if /\.so$/;
}, "$tmp/$curdir" ;
if ($has_module) {
if ($dh{V_FLAG_SET}) {
$verdeps{$usepython} |=
SO_MODULE_NONSTANDARD;
} else {
$deps |= SO_MODULE;
}
}
}
}
# Dependencies on current python
$dep_on_python = 1 if $deps;
$strong_dep = 1 if ($deps & (PY_MODULE|SO_MODULE));
if ($dep_on_python and not $python_header) {
# Generate dependencies following old policy
addsubstvar($package, "python:Depends", $python, ">=
$python_version");
if ($strong_dep) {
addsubstvar($package, "python:Depends", $python, "<<
$python_nextversion");
} else {
addsubstvar($package, "python:Depends", $python, "<<
$python_nextmajor");
}
}
# At this point we have scanned everything for executables scripts
# and directories for private modules.
my $need_prerm = 0;
# Look for specific pythonX.Y modules
my %pyversions_found;
foreach my $pyver (@python_allversions) {
my $pydir="/usr/lib/$pyver/site-packages";
if (-d "$tmp$pydir") {
find sub {
return unless -f;
if (/\.py$/) {
$verdeps{$pyver} |= PY_MODULE;
doit(("rm","-f",$_."c",$_."o"));
}
$verdeps{$pyver} |= SO_MODULE if /\.so$/;
}, "$tmp$pydir";
}
if ($verdeps{$pyver} & (PY_MODULE|SO_MODULE)) {
# Remember that files specific to this version have
# been found
$pyver =~ /^python([\d\.]+)/ and $pyversions_found{$1} = 1;
}
# Stop here when using new policy
next if $python_header;
# Go for the dependencies
addsubstvar($package, "python:Depends", $pyver) if
$verdeps{$pyver};
# And now, the postinst and prerm stuff
# It takes care of byte-compiling modules
if ($pyver eq "$usepython") {
# Current version of python, also byte-compile the
# private directories (stored in $dirlist)
if ($verdeps{$pyver} & PY_MODULE) {
$pydir = $pydir.$dirlist;
} else {
# Only private directories
$pydir = $dirlist;
}
$verdeps{$pyver} |= PY_MODULE if($deps & PY_MODULE);
} else {
# Non-current python versions: we only have to
# byte-compile stuff in /usr/lib/python2.X
# Leave $pydir as is
}
if ($verdeps{$pyver} & (PY_MODULE|PY_MODULE_NONSTANDARD) && !
$dh{NOSCRIPTS}) {
autoscript($package,"postinst","postinst-python","s%#PYVER#%$pyver%;s%#DIRLIST#%$pydir%");
$need_prerm = 1;
}
}
if ($need_prerm && ! $dh{NOSCRIPTS}) {
autoscript($package,"prerm","prerm-python","s%#PACKAGE#%$package%");
}
# Look for python2.X modules from the python package itself
foreach my $pyver (@python_allversions) {
my $pydir="/usr/lib/$pyver";
if (-d "$tmp$pydir") {
find sub {
return unless -f;
return if $File::Find::name =~
m|^$tmp$pydir/site-packages/|;
if (/\.py$/) {
$verdeps{$pyver} |= PY_MODULE;
doit(("rm","-f",$_."c",$_."o"));
}
$verdeps{$pyver} |= SO_MODULE if /\.so$/;
}, "$tmp$pydir";
}
if ($verdeps{$pyver} & (PY_MODULE|SO_MODULE)) {
# Remember that files specific to this version have
# been found
$pyver =~ /^python([\d\.]+)/ and $pyversions_found{$1} = 1;
}
}
if ($python_header) {
# New policy: generate the depends to accept all python
# versions that this package effectively provides
my $min_version = "";
my $stop_version = "";
my $versions_field = "";
# Reset again, analysis using new policy follows
$dep_on_python = 0;
# Extracting min version from XS-Python-version
if ($python_header =~ /^current(?:,\s*>=\s*([\d\.]+))?/) {
if (defined $1) {
$min_version = $1;
}
$versions_field = "current";
} elsif ($python_header =~ />=\s*([\d\.]+)/) {
$min_version = $1;
}
if ($python_header =~ /<<\s*([\d\.]+)/) {
$stop_version = $1;
}
# Private extensions, must be rebuilt for each python version
if ($deps & SO_MODULE) {
$dep_on_python++;
$stop_version = next_minor_version($python_version);
# Packages using a private extension can only
# support one version and they indicate which one
# in XS-Python-Version
if ($versions_field eq "current") {
$versions_field = $python_version;
} else {
$versions_field = $python_header;
}
}
# Private modules
if ($deps & PY_MODULE) {
# Package with private module can only support one
version at a time
# (even if the modules can be automatically
byte-compiled for any new
# python version).
unless ($versions_field) {
# If XS-Python-Version indicated "current" we
would already have a value,
# since we have not, XS-Python-Version must
contain a hardcoded version.
if ($python_header =~ /^[\d\.]+$/) {
$versions_field = $python_header;
} else {
warning("The package contains private
modules. It should indicate 'current'\n".
"or a harcoded python version (ex:
'2.4') in the XS-Python-Version field.\n")
}
}
$versions_field = "current" unless $versions_field;
}
# Python scripts
if ($deps & PROGRAM) {
$dep_on_python++;
}
# Public extensions
# XXX: if we have *.py files left in
/usr/lib/python2.X/site-packages
# they are no more byte-compiled... but that shouldn't happen
if the
# package really conforms to the policy
if (scalar keys %pyversions_found) {
# Extensions will always trigger this (as well as public
# modules not handled by python-support/python-central)
$dep_on_python++;
$min_version = min(keys %pyversions_found);
unless ($stop_version) {
my $max_version = max(keys %pyversions_found);
$stop_version = next_minor_version($max_version);
}
# Generate the Python-Version field
foreach (keys %pyversions_found) {
addsubstvar($package, "python:Versions", $_);
}
# Generate provides for the python2.X-foo packages that
we emulate
if ($package =~ /^python-/) {
foreach (keys %pyversions_found) {
my $virtual = $package;
$virtual =~ s/^python-/$python$_-/;
addsubstvar($package,
"python:Provides", $virtual);
}
}
} else {
# Still try to describe the versions that the package
support
$versions_field = "all" unless $versions_field;
addsubstvar($package, "python:Versions",
$versions_field);
}
# Private extensions compiled with non-current python
# or scripts using non-current python
foreach my $pyver (keys %verdeps) {
if ($verdeps{$pyver} & (SO_MODULE_NONSTANDARD|PROGRAM))
{
addsubstvar($package, "python:Depends", $pyver);
# Heuristic: if we have pythonX.Y program and
if we have precisely only that
# $pyversions_found, we don't need to depend on
a specific interval of python
my @found = keys %pyversions_found;
if (($verdeps{$pyver} & PROGRAM) &&
(scalar @found == 1) &&
("python$found[0]" eq $pyver)
) {
$dep_on_python--; # Cancel the
dep_on_python due to shared extensions
$min_version = "";
$stop_version = "";
}
}
}
if ($dep_on_python) {
# At least a script has been detected
addsubstvar($package, "python:Depends", $python, ">=
$min_version") if $min_version;
# If a stronger dependency is needed
addsubstvar($package, "python:Depends", $python, "<<
$stop_version") if $stop_version;
# Let's depend on python anyway
addsubstvar($package, "python:Depends", $python) unless
($min_version or $stop_version);
}
}
}
sub next_minor_version {
my $version = shift;
# Handles 2.10 -> 2.11 gracefully
my @items = split(/\./, $version);
$items[1] += 1;
$version = join(".", @items);
return $version;
}
sub compare_version {
my ($a, $b) = @_;
my @A = split(/\./, $a);
my @B = split(/\./, $b);
my $diff = 0;
for (my $i = 0; $i <= $#A; $i++) {
$diff = $A[$i] - $B[$i];
return $diff if $diff;
}
# They are the same
return 0;
}
sub max {
my $max = shift;
foreach (@_) {
$max = $_ if (compare_version($_, $max) > 0);
}
return $max;
}
sub min {
my $min = shift;
foreach (@_) {
$min = $_ if (compare_version($_, $min) < 0);
}
return $min;
}
=head1 SEE ALSO
L<debhelper(7)>
This program is a part of debhelper.
=head1 AUTHORS
Josselin Mouette <[EMAIL PROTECTED]>
Raphael Hertzog <[EMAIL PROTECTED]>
most ideas stolen from Brendan O'Dea <[EMAIL PROTECTED]>
=cut
#!/usr/bin/perl -w
=head1 NAME
dh_pycentral - use the python-central framework to handle Python modules and
extensions
=cut
use strict;
use File::Find;
use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
B<dh_pycentral> [S<I<debhelper options>>] [B<-n>]
=head1 DESCRIPTION
dh_pycentral is a debhelper program that will scan your package, detect
public Python modules and move them in /usr/share/pycentral so that
python-central can byte-compile those for all supported Python versions.
You must of have filled the XS-Python-Header to indicate the set of python
versions that are going to be supported.
=head1 OPTIONS
=over 4
=item B<-n>, B<--noscripts>
Do not modify postinst/postrm scripts.
=back
=head1 CONFORMS TO
Python policy as of 2006-06-10
=cut
init();
foreach my $package (@{$dh{DOPACKAGES}}) {
my $tmp = tmpdir($package);
# Move *.py files if needed
doit("pycentral debhelper $package $tmp >/dev/null");
# Check that we have *.py files!
my $found = 0;
find sub {
return if $File::Find::dir =~ m|^$tmp/usr/share/doc/|;
return unless -f and /\.py$/;
$found++;
}, $tmp;
if ($found) {
addsubstvar($package, "python:Depends", "python-central", ">=
0.4.7");
if (! $dh{NOSCRIPTS}) {
autoscript($package,"postinst","postinst-pycentral","s%#PACKAGE#%$package%");
autoscript($package,"prerm","prerm-pycentral","s%#PACKAGE#%$package%");
}
}
}
=head1 SEE ALSO
L<debhelper(7)>
This program is a part of python-central but is made to work with debhelper.
=head1 AUTHORS
Raphael Hertzog <[EMAIL PROTECTED]>
=cut
#!/usr/bin/perl -w
=head1 NAME
dh_pysupport - use the python-support framework to handle Python modules and
extensions
=cut
use strict;
use File::Find;
use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
B<dh_pysupport> [S<I<debhelper options>>] [B<-n>] [S<I<module dirs ...>>]
=head1 DESCRIPTION
dh_pycentral is a debhelper program that will scan your package, detect
public Python modules and move them in /usr/share/pycentral so that
python-central can byte-compile those for all supported Python versions.
You must of have filled the XS-Python-Header to indicate the set of python
versions that are going to be supported.
=head1 OPTIONS
=over 4
=item I<module dirs>
If your package installs private python modules in non-standard directories, you
can make dh_pysupport check those directories by passing their names on the
command line. By default, it will check /usr/lib/$PACKAGE,
/usr/share/$PACKAGE, /usr/lib/games/$PACKAGE and /usr/share/games/$PACKAGE
=item B<-n>, B<--noscripts>
Do not modify postinst/postrm scripts.
=back
=head1 CONFORMS TO
Python policy as of 2006-06-10
=cut
init();
foreach my $package (@{$dh{DOPACKAGES}}) {
my $tmp = tmpdir($package);
my @dirs = ("usr/lib/$package", "usr/share/$package",
"usr/lib/games/$package", "usr/share/games/$package", @ARGV
);
@dirs = grep -d, map "$tmp/$_", @dirs;
# Look for private python modules
my $dirlist="";
if (@dirs) {
foreach my $curdir (@dirs) {
my $has_module = 0;
$curdir =~ s%^$tmp/%%;
find sub {
return unless -f;
$has_module = 1 if (/\.py$/);
}, "$tmp/$curdir" ;
if ($has_module) {
$dirlist="$dirlist /$curdir";
}
}
}
if ($dirlist) {
# We have private python modules
# Use python-support to ensure that they are always
# byte-compiled for the current version
mkdir("$tmp/usr/share/python-support");
$dirlist =~ s/^\s*//;
$dirlist =~ s/\s*$//;
open(DIRLIST, "> $tmp/usr/share/python-support/$package.dirs")
||
error("Can't create
$tmp/usr/share/python-support/$package.dirs: $!");
print DIRLIST join("\n", split(/\s+/, $dirlist));
close(DIRLIST);
}
# Add scripts to postinst/prerm depending on what we found
if (-d "$tmp/usr/share/python-support") {
addsubstvar($package, "python:Depends", "python-support");
foreach my $ps_dir (glob("$tmp/usr/share/python-support/*")) {
if (-d $ps_dir) {
$ps_dir =~ s/^$tmp//;
if (! $dh{NOSCRIPTS}) {
autoscript($package, "postinst",
"postinst-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
autoscript($package, "prerm",
"prerm-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
}
# TODO: we should generate the
# /usr/share/python-support/*/.version from
# XS-Python-Version
} elsif (-f $ps_dir) {
# $ps_dir is a ".dirs" file ...
$ps_dir =~ s/^$tmp//;
autoscript($package, "postinst",
"postinst-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
autoscript($package, "prerm",
"prerm-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
}
}
}
}
=head1 SEE ALSO
L<debhelper(7)>
This program is a part of python-support but is made to work with debhelper.
=head1 AUTHORS
Josselin Mouette <[EMAIL PROTECTED]>
Raphael Hertzog <[EMAIL PROTECTED]>
=cut
