Re: python-central 0.4

[Graham Wilson]

On Fri, Oct 04, 2002 at 02:05:32AM +1000, Donovan Baarda wrote:
 The reason I said site-packages is the _right_ way, is it makes mailman
 modules usable from other applications. However, I agree that we shouldn't
 _require_ application specific modules to be in site-packages, particularly
 when upstream don't package that way. As you pointed out there are plenty of
 good reasons not to put stuff in site-packages.

just as an example, i am working on packaging spyce [1], and it does not
put any of its module files in a directory like mailman does. thus all
of the scripts using those module file would have to be changed from:

 import spyceModule

to something like:

 from spyce import spyceModule

which is not a fun change to have to make.

 Is there any cases you can think of where 'dpkg -L python-foo | grep *.py'
 will not be sufficient to identify the files to be registered?

why doesn't python-central handle the (re)compilation of python modules.
it can just get the list using dpkg -L python-foo | grep *.py.

that way, package postinst scripts dont have to worry about the details
and we wont have to run dpkg-reconfigure, which could, for example, ask
debconf questions again.

[1] http://spyce.sourceforge.net/

--
gram




pgpexbbG3Un9n.pgp
Description: PGP signature

Re: python-central 0.4

[Matthias Klose]

Donovan Baarda writes:
  I wouldn't call moving some files the packaging hell, and I have yet to
  understand why /usr/lib/mailman is so much saner or better than
  /usr/lib/python/site-packages/mailman.
  I looked into the mailman package. It should not be that much work to
  adapt it to the python-central framework (moving the Mailman module
  into .../site-packages/, removing the paths.py hack).
  And when the mailman Bug#162761 is fixed, I could even provide a patch ;)
 
 I guess this is an issue of upstream compatiblity. Sure, packages could be
 restructured to fit the scheme, but the other alternative is to masssage the
 scheme to fit the upstream packages...
 
 I think that using /usr/lib/python/site-packages/mailman properly is the
 _right_ way to do it, but I can see the argument for /usr/lib/mailman,
 particularly when upstream does it that way.

IMO this is absolutely wrong.

Bastian Kleineidam writes:
 Hi,
 
 to put the code where my mouth was I tried to make a patch for mailman
 using the python-register utility.
 It was just moving files around - so I could fire up python and
 do an 'import Mailman'.

I originally mentioned mailman as an example for an application. Sure
mailman does consist of a library, which may be worth using in
ordinary python programs. However there _are_ applications with
modules not belonging in the /usr/lib/python tree: Assume a program
foo importing baz.py and bar.py. You cannot simply put these files in
/usr/lib/python:

  - these may not be library modules.

  - they pollute the namespace (assuming you have just another
application foo2, having its own bar module.
Even putting them in a separate directory does not help, because
these directories are all added to sys.path.

So python-central _has_ to handle the case of python files laying
around somewhere (i.e. /usr/lib/foo/{bar,baz}.py), recompiling these
files on python upgrades.

Please design python-central for all kind of packages, do not require
the packages to be adapted to python-central.

  Conclusion: the python-central framework as it is now offers enough for
  everyone. If people don't want to use it, they are on their own.
 
 I think we could add support for the mailman case with minimal hassles...
 
 The biggest problem I can forsee is this introduces another option that will
 confuse developers... But this can be rectified with strongly suggests in
 the python policy.

There should be no confusion telling python-central the locations of
files to be registered.

  Hmm, reading the Debian policy, the only difference is that Depends only
  are considered on configuring a package, Conflicts are considered on
  installing. I dont know whats better, I think either will do.
  I will adapt the man pages to the Python Policy.
 
 Give this, I would prefer the Depends rather than Conflicts, as it more
 closely matches what a python version incompatability means... you can
 install the files, but you can't configure them to work.

please send a patch for the policy.

Bastian Kleineidam writes:
 python-central (0.4) unstable; urgency=low
 
   * renamed register-python-package to python-register, this way
 its prefixed with python (and its shorter ;)

as long as python-central handles libraries/modules only, it should be
called register-python-library. Don't confuse the packager ;-)


Matthias, away 'til Sunday.

Re: python-central 0.4

[Donovan Baarda]

On Thu, Oct 03, 2002 at 05:21:34PM +0200, Matthias Klose wrote:
 Donovan Baarda writes:
   I wouldn't call moving some files the packaging hell, and I have yet to
   understand why /usr/lib/mailman is so much saner or better than
   /usr/lib/python/site-packages/mailman.
   I looked into the mailman package. It should not be that much work to
   adapt it to the python-central framework (moving the Mailman module
   into .../site-packages/, removing the paths.py hack).
   And when the mailman Bug#162761 is fixed, I could even provide a patch ;)
  
  I guess this is an issue of upstream compatiblity. Sure, packages could be
  restructured to fit the scheme, but the other alternative is to masssage the
  scheme to fit the upstream packages...
  
  I think that using /usr/lib/python/site-packages/mailman properly is the
  _right_ way to do it, but I can see the argument for /usr/lib/mailman,
  particularly when upstream does it that way.
 
 IMO this is absolutely wrong.

I was agreeing with you :-)

The reason I said site-packages is the _right_ way, is it makes mailman
modules usable from other applications. However, I agree that we shouldn't
_require_ application specific modules to be in site-packages, particularly
when upstream don't package that way. As you pointed out there are plenty of
good reasons not to put stuff in site-packages.

[...]
 So python-central _has_ to handle the case of python files laying
 around somewhere (i.e. /usr/lib/foo/{bar,baz}.py), recompiling these
 files on python upgrades.
 
 Please design python-central for all kind of packages, do not require
 the packages to be adapted to python-central.

That's another vote for support of python application packages with modules
outside /usr/lib/python/site-packages :-)

   Conclusion: the python-central framework as it is now offers enough for
   everyone. If people don't want to use it, they are on their own.
  
  I think we could add support for the mailman case with minimal hassles...
  
  The biggest problem I can forsee is this introduces another option that will
  confuse developers... But this can be rectified with strongly suggests in
  the python policy.
 
 There should be no confusion telling python-central the locations of
 files to be registered.

The confusion I was refering to was the is a python-foo package
optional/encoraged/required for my pythonX.Y-foo packages type questions
that have cropped up in responce to the current policy. I think this option
will raise should I put my modules in /usr/lib/foo or
/usr/lib/python/site-packages/foo type questions unless we clarify the
circumstances for each in the policy.

Is there any cases you can think of where 'dpkg -L python-foo | grep *.py'
will not be sufficient to identify the files to be registered?

   Hmm, reading the Debian policy, the only difference is that Depends only
   are considered on configuring a package, Conflicts are considered on
   installing. I dont know whats better, I think either will do.
   I will adapt the man pages to the Python Policy.
  
  Give this, I would prefer the Depends rather than Conflicts, as it more
  closely matches what a python version incompatability means... you can
  install the files, but you can't configure them to work.
 
 please send a patch for the policy.

The Depends option is what is in the current policy. The python-central
man page was not matching the current policy.

 Bastian Kleineidam writes:
  python-central (0.4) unstable; urgency=low
  
* renamed register-python-package to python-register, this way
  its prefixed with python (and its shorter ;)
 
 as long as python-central handles libraries/modules only, it should be
 called register-python-library. Don't confuse the packager ;-)

It'll get there eventualy :-)

-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--

Re: python-central 0.4

[Carey Evans]

Donovan Baarda wrote:
But this is redundant... if the package is installed, the corresponding *.py
files _should_ be unpacked. How is testing for the existance of a file any
better indication than testing for installation of the package?
True... I think I'll try to blame my flawed argument on lack of caffeine 
in the bloodstream.

[...]
This is an interesting issue I hadn't considered. However, I can't see how
seperate compilation scripts do a better job of avoiding this problem than
dpkg-reconfigure.
Since, in the absence of debconf, all dpkg-reconfigure does is run
   /var/lib/dpkg/info/package.postinst configure current-version
/I guess editing the postinst would be fine as well.  /However, it still 
seem to me that having recompilation separate from installing 
documentation, menus, MIME types and whatever else might be worthwhile.

It might also be nice to have separate files that list the directories 
or source files to compile for each package, and have python-central 
call compileall itself, but I guess this is a separate issue.

[...]
So the postinst scripts get passed the version upgraded from? (I should know
this :-)
I should have checked too.  See section 6 of Policy for details, 
especially what can happen if an error occurs, but the postinst is the 
one script that _doesn't_ get called with the other version number.  The 
old version's prerm and postrm, and the new version's preinst, could all 
check whether an incompatible upgrade was going to or had occurred.  In 
this case, I guess they'd set a flag and delete the old *.py[co] files.

Re: python-central 0.4

[Donovan Baarda]

On Tue, Oct 01, 2002 at 12:23:38AM +1200, Carey Evans wrote:
 Donovan Baarda wrote:
[...]
 It might also be nice to have separate files that list the directories 
 or source files to compile for each package, and have python-central 
 call compileall itself, but I guess this is a separate issue.

you mean like;

dpkg -L python-foo | grep '.py$'

:-)


-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--

Re: python-central 0.4

[Donovan Baarda]

On Sun, Sep 29, 2002 at 12:26:30AM +0200, Bastian Kleineidam wrote:
 Hi,
 
 uploaded the new version 0.4 at
 http://people.debian.org/~calvin/python-central/
 
 python-central (0.4) unstable; urgency=low
 
   * renamed register-python-package to python-register, this way
 its prefixed with python (and its shorter ;)
   * add options to enable/suppress the compiling of .py[co] files
   * add a new command python-compile to compile single python
 files
   * Added a separate section in python-register.1 for python programs;
 also give an example.
 
  -- Bastian Kleineidam [EMAIL PROTECTED]  Fri, 13 Sep 2002 22:17:45 +0200

I've just had a look at this and it looks good. It perfectly meets the
requirement of allowing pure python module packages to support multiple
pythonX.Y python packages simultaniously.

The only problem is we are still missing something. We still can't easily
support pure python _program_ packages that work with multiple versions of
(the default) python.

Programs like mailman have large mailman specific modules that are used only
by the mailman program. The mailman program doesn't need to support multiple
pythonX.Y packages simultaniously, just multiple versions of the default
python. This is in many ways a simpler requirement than what python-central
solves, but we still haven't got it covered.

Programs like mailman put their modules in their own /usr/lib/mailman
directory. One way that mailman _could_ support multiple pythonX.Y packages
it to have the packager move _all_ the modules into a
/usr/lib/python/site-packages directory and then use python central.
However, this is overkill for mailman because it doesn't need to support
python2.1 and python2.2 simultanously, just the default python. It is also
hell for the packager.

The only problem programs like mailman have is if the default python
upgrades after they are installed, they need to be recompiled. The best way
to handle this is to have the default python packages run dpkg-reconfigure
on all installed packages that depend on python.

This could be added to python central. Currently it auto-detects whether a
python module or pythonX.Y package is being installed by looking at the
package name, and behaves accordingly. All that needs to be added is to have
python-central check for the package name python, and have it do the
dpkg-reconfigure for all packages depending on python. We then have the
default python packages register themselves using python-central.

Now some random thoughts;

finding all packages that depend on python is non-trivial using only dpkg.
Something like dpkg-awk, grep-dctrl, or python-apt make it much easier, but
do we want to depend on them? The old registry idea would have made this a
little easier, but I still prefer using the dpkg database to find this kind
of info.

Does dpkg-reconfigure actually run the postinst script? Does it do it in a
way (right parameters etc) that would make exising python packages
re-compile properly?

Is a dpkg-reconfigure overkill? We only need to recompile the *.py's. Do
some packages really need to go through a dpkg-reconfigure anyway? What if
they are using debconf options for things like compile modules?

We only need to reconfigure/recompile when the default version of python
upgrades to a different version of python, not minor package upgrades. Is
there any way we can detect this and avoid un-necisary recompiles?

Should python-central be providing compile scripts for use by all python
packages? Should we be making use of these scripts manditory in the
Python Policy? If it was then we should be safe using these scripts to
recompile packages instead of using dpkg-reconfigure. But would it be any
better? Is compiling in postinst scripts so trivial as to not need support
scripts?

Another thing I noticed, the man pages are now recommending using;

   Depends: python (=2.1)
   Conflicts: python (= 2.4)
  
Doesn't the policy have;

   Depends: python (=2.1), python (2.4)

I think these mean the same thing, but has the policy changed for some
reason? I actually think that Conflicts is less ambiguous...

Pending feedback/ideas whatever, I can code something up to do this. I
probably favor using python-apt (if it can actualy do it) or grep-dctrl to
do the package lookup stuff, but can do it manually if this is a problem.

-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--

Re: python-central 0.4

[Bastian Kleineidam]

On Sun, Sep 29, 2002 at 11:29:02PM +1000, Donovan Baarda wrote:
 I've just had a look at this and it looks good. It perfectly meets the
 requirement of allowing pure python module packages to support multiple
 pythonX.Y python packages simultaniously.
 
 The only problem is we are still missing something. We still can't easily
 support pure python _program_ packages that work with multiple versions of
 (the default) python.
What is wrong with
Depends: python2.2 | python2.3, python (= 2.2), python ( 2.4)
This supports the 2.2 and upcoming 2.3 default python package.

 Programs like mailman put their modules in their own /usr/lib/mailman
 directory. One way that mailman _could_ support multiple pythonX.Y packages
 it to have the packager move _all_ the modules into a
 /usr/lib/python/site-packages directory and then use python central.
 However, this is overkill for mailman because it doesn't need to support
 python2.1 and python2.2 simultanously, just the default python. It is also
 hell for the packager.
I wouldn't call moving some files the packaging hell, and I have yet to
understand why /usr/lib/mailman is so much saner or better than
/usr/lib/python/site-packages/mailman.
I looked into the mailman package. It should not be that much work to
adapt it to the python-central framework (moving the Mailman module
into .../site-packages/, removing the paths.py hack).
And when the mailman Bug#162761 is fixed, I could even provide a patch ;)

Conclusion: the python-central framework as it is now offers enough for
everyone. If people don't want to use it, they are on their own.

 Another thing I noticed, the man pages are now recommending using;
 
Depends: python (=2.1)
Conflicts: python (= 2.4)
   
 Doesn't the policy have;
 
Depends: python (=2.1), python (2.4)
 
 I think these mean the same thing, but has the policy changed for some
 reason? I actually think that Conflicts is less ambiguous...

Hmm, reading the Debian policy, the only difference is that Depends only
are considered on configuring a package, Conflicts are considered on
installing. I dont know whats better, I think either will do.
I will adapt the man pages to the Python Policy.


 Pending feedback/ideas whatever, I can code something up to do this. I
 probably favor using python-apt (if it can actualy do it) or grep-dctrl to
 do the package lookup stuff, but can do it manually if this is a problem.
Dont code anything like that. I'd rather fix those hacky'ish packages
which try to append something to sys.path.

Cheers,
Bastian
-- 
 Bastian Kleineidam

 reflexionsniveau AT web.de
   calvin AT users.sf.net
calvin AT debian.org


pgpG7b6H6swnL.pgp
Description: PGP signature

Re: python-central 0.4

[Donovan Baarda]

On Sun, Sep 29, 2002 at 05:51:30PM +0200, Bastian Kleineidam wrote:
 On Sun, Sep 29, 2002 at 11:29:02PM +1000, Donovan Baarda wrote:
  I've just had a look at this and it looks good. It perfectly meets the
  requirement of allowing pure python module packages to support multiple
  pythonX.Y python packages simultaniously.
  
  The only problem is we are still missing something. We still can't easily
  support pure python _program_ packages that work with multiple versions of
  (the default) python.
 What is wrong with
 Depends: python2.2 | python2.3, python (= 2.2), python ( 2.4)
 This supports the 2.2 and upcoming 2.3 default python package.

Yes it does. However, under python central this means you would have all the
modules compiled and symlinked for both python2.2 and python2.3 if they were
both installed. This is great if you want to use these modules from both
pythonX.Y versions, but in the case of something like mailman, you only want
to use them from the default python. Having the other pythonX.Y support is
not required.

  Programs like mailman put their modules in their own /usr/lib/mailman
  directory. One way that mailman _could_ support multiple pythonX.Y packages
  it to have the packager move _all_ the modules into a
  /usr/lib/python/site-packages directory and then use python central.
  However, this is overkill for mailman because it doesn't need to support
  python2.1 and python2.2 simultanously, just the default python. It is also
  hell for the packager.
 I wouldn't call moving some files the packaging hell, and I have yet to
 understand why /usr/lib/mailman is so much saner or better than
 /usr/lib/python/site-packages/mailman.
 I looked into the mailman package. It should not be that much work to
 adapt it to the python-central framework (moving the Mailman module
 into .../site-packages/, removing the paths.py hack).
 And when the mailman Bug#162761 is fixed, I could even provide a patch ;)

I guess this is an issue of upstream compatiblity. Sure, packages could be
restructured to fit the scheme, but the other alternative is to masssage the
scheme to fit the upstream packages...

I think that using /usr/lib/python/site-packages/mailman properly is the
_right_ way to do it, but I can see the argument for /usr/lib/mailman,
particularly when upstream does it that way.

 Conclusion: the python-central framework as it is now offers enough for
 everyone. If people don't want to use it, they are on their own.

I think we could add support for the mailman case with minimal hassles...

The biggest problem I can forsee is this introduces another option that will
confuse developers... But this can be rectified with strongly suggests in
the python policy.

  Another thing I noticed, the man pages are now recommending using;
  
 Depends: python (=2.1)
 Conflicts: python (= 2.4)

  Doesn't the policy have;
  
 Depends: python (=2.1), python (2.4)
  
  I think these mean the same thing, but has the policy changed for some
  reason? I actually think that Conflicts is less ambiguous...
 
 Hmm, reading the Debian policy, the only difference is that Depends only
 are considered on configuring a package, Conflicts are considered on
 installing. I dont know whats better, I think either will do.
 I will adapt the man pages to the Python Policy.

Give this, I would prefer the Depends rather than Conflicts, as it more
closely matches what a python version incompatability means... you can
install the files, but you can't configure them to work.

  Pending feedback/ideas whatever, I can code something up to do this. I
  probably favor using python-apt (if it can actualy do it) or grep-dctrl to
  do the package lookup stuff, but can do it manually if this is a problem.
 Dont code anything like that. I'd rather fix those hacky'ish packages
 which try to append something to sys.path.

I think this needs feedback from the people actually maintaining the
packages. If they think restructuring into /var/lib/python/site-packages is a
good idea, then fine. However, if they would prefer to structure their files
the same as upstream does...


-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--

Re: python-central 0.4

[Derrick 'dman' Hudson]

On Mon, Sep 30, 2002 at 08:19:49AM +1000, Donovan Baarda wrote:
| On Mon, Sep 30, 2002 at 08:13:23AM +1200, Carey Evans wrote:

|  It also makes it easier for users to modify if a Python package's 
|  dependencies are incorrect, and it stops compiling under a newer version 
|  of Python, making the whole dpkg run fail.
| 
| This is an interesting issue I hadn't considered. However, I can't see how
| seperate compilation scripts do a better job of avoiding this problem than
| dpkg-reconfigure.

If there's a malfunctioning script in /somewhere that I can insert
exit 0 ;
or
import sys ; sys.exit( 0 )
at the top, then as a user/admin I can sidestep the installation
error.  If the logic is tucked away in the bowels of some other
program, it isn't as trivial to hack around.

(I don't have a position on this issue, though, I'm just sorta lurking
on the discussions)

-D

-- 
But As for me and my household, we will serve the Lord.
Joshua 24:15
 
http://dman.ddts.net/~dman/


pgp0JFaZZdMpe.pgp
Description: PGP signature

python-central 0.4

[Bastian Kleineidam]

Hi,

uploaded the new version 0.4 at
http://people.debian.org/~calvin/python-central/

python-central (0.4) unstable; urgency=low

  * renamed register-python-package to python-register, this way
its prefixed with python (and its shorter ;)
  * add options to enable/suppress the compiling of .py[co] files
  * add a new command python-compile to compile single python
files
  * Added a separate section in python-register.1 for python programs;
also give an example.

 -- Bastian Kleineidam [EMAIL PROTECTED]  Fri, 13 Sep 2002 22:17:45 +0200


cu, 
Bastian
-- 
 Bastian Kleineidam

 reflexionsniveau AT web.de
   calvin AT users.sf.net
calvin AT debian.org


pgph75unJxJ81.pgp
Description: PGP signature