Re: python-central 0.4

2002-10-04 Thread Graham Wilson
On Fri, Oct 04, 2002 at 02:05:32AM +1000, Donovan Baarda wrote:
> The reason I said site-packages is the _right_ way, is it makes mailman
> modules usable from other applications. However, I agree that we shouldn't
> _require_ application specific modules to be in site-packages, particularly
> when upstream don't package that way. As you pointed out there are plenty of
> good reasons not to put stuff in site-packages.

just as an example, i am working on packaging spyce [1], and it does not
put any of its module files in a directory like mailman does. thus all
of the scripts using those module file would have to be changed from:

 import spyceModule

to something like:

 from spyce import spyceModule

which is not a fun change to have to make.

> Is there any cases you can think of where 'dpkg -L  | grep *.py'
> will not be sufficient to identify the files to be "registered"?

why doesn't python-central handle the (re)compilation of python modules.
it can just get the list using "dpkg -L  | grep *.py".

that way, package postinst scripts dont have to worry about the details
and we wont have to run dpkg-reconfigure, which could, for example, ask
debconf questions again.

[1] http://spyce.sourceforge.net/

--
gram




pgpexbbG3Un9n.pgp
Description: PGP signature


Re: python-central 0.4

2002-10-03 Thread Donovan Baarda
On Thu, Oct 03, 2002 at 05:21:34PM +0200, Matthias Klose wrote:
> Donovan Baarda writes:
> > > I wouldn't call moving some files the packaging hell, and I have yet to
> > > understand why /usr/lib/mailman is so much saner or better than
> > > /usr/lib/python/site-packages/mailman.
> > > I looked into the mailman package. It should not be that much work to
> > > adapt it to the python-central framework (moving the Mailman module
> > > into .../site-packages/, removing the paths.py hack).
> > > And when the mailman Bug#162761 is fixed, I could even provide a patch ;)
> > 
> > I guess this is an issue of "upstream compatiblity". Sure, packages could be
> > restructured to fit the scheme, but the other alternative is to masssage the
> > scheme to fit the upstream packages...
> > 
> > I think that using /usr/lib/python/site-packages/mailman properly is the
> > _right_ way to do it, but I can see the argument for /usr/lib/mailman,
> > particularly when upstream does it that way.
> 
> IMO this is absolutely wrong.

I was agreeing with you :-)

The reason I said site-packages is the _right_ way, is it makes mailman
modules usable from other applications. However, I agree that we shouldn't
_require_ application specific modules to be in site-packages, particularly
when upstream don't package that way. As you pointed out there are plenty of
good reasons not to put stuff in site-packages.

[...]
> So python-central _has_ to handle the case of python files laying
> around somewhere (i.e. /usr/lib/foo/{bar,baz}.py), recompiling these
> files on python upgrades.
> 
> Please design python-central for all kind of packages, do not require
> the packages to be adapted to python-central.

That's another vote for support of python application packages with modules
outside /usr/lib/python/site-packages :-)

> > > Conclusion: the python-central framework as it is now offers enough for
> > > everyone. If people don't want to use it, they are on their own.
> > 
> > I think we could add support for the mailman case with minimal hassles...
> > 
> > The biggest problem I can forsee is this introduces another option that will
> > confuse developers... But this can be rectified with "strongly suggests" in
> > the python policy.
> 
> There should be no confusion telling python-central the locations of
> files to be registered.

The confusion I was refering to was the "is a python-foo package
optional/encoraged/required for my pythonX.Y-foo packages" type questions
that have cropped up in responce to the current policy. I think this option
will raise "should I put my modules in /usr/lib/foo or
/usr/lib/python/site-packages/foo" type questions unless we clarify the
circumstances for each in the policy.

Is there any cases you can think of where 'dpkg -L  | grep *.py'
will not be sufficient to identify the files to be "registered"?

> > > Hmm, reading the Debian policy, the only difference is that Depends only
> > > are considered on configuring a package, Conflicts are considered on
> > > installing. I dont know whats better, I think either will do.
> > > I will adapt the man pages to the Python Policy.
> > 
> > Give this, I would prefer the "Depends" rather than "Conflicts", as it more
> > closely matches what a python version incompatability means... you can
> > install the files, but you can't configure them to work.
> 
> please send a patch for the policy.

The "Depends" option is what is in the current policy. The python-central
man page was not matching the current policy.

> Bastian Kleineidam writes:
> > python-central (0.4) unstable; urgency=low
> > 
> >   * renamed register-python-package to python-register, this way
> > its prefixed with "python" (and its shorter ;)
> 
> as long as python-central handles libraries/modules only, it should be
> called register-python-library. Don't confuse the packager ;-)

It'll get there eventualy :-)

-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--




Re: python-central 0.4

2002-10-03 Thread Matthias Klose
Donovan Baarda writes:
> > I wouldn't call moving some files the packaging hell, and I have yet to
> > understand why /usr/lib/mailman is so much saner or better than
> > /usr/lib/python/site-packages/mailman.
> > I looked into the mailman package. It should not be that much work to
> > adapt it to the python-central framework (moving the Mailman module
> > into .../site-packages/, removing the paths.py hack).
> > And when the mailman Bug#162761 is fixed, I could even provide a patch ;)
> 
> I guess this is an issue of "upstream compatiblity". Sure, packages could be
> restructured to fit the scheme, but the other alternative is to masssage the
> scheme to fit the upstream packages...
> 
> I think that using /usr/lib/python/site-packages/mailman properly is the
> _right_ way to do it, but I can see the argument for /usr/lib/mailman,
> particularly when upstream does it that way.

IMO this is absolutely wrong.

Bastian Kleineidam writes:
> Hi,
> 
> to put the code where my mouth was I tried to make a patch for mailman
> using the python-register utility.
> It was just moving files around - so I could fire up python and
> do an 'import Mailman'.

I originally mentioned mailman as an example for an application. Sure
mailman does consist of a library, which may be worth using in
ordinary python programs. However there _are_ applications with
modules not belonging in the /usr/lib/python tree: Assume a program
foo importing baz.py and bar.py. You cannot simply put these files in
/usr/lib/python:

  - these may not be library modules.

  - they pollute the namespace (assuming you have just another
application foo2, having its own bar module.
Even putting them in a separate directory does not help, because
these directories are all added to sys.path.

So python-central _has_ to handle the case of python files laying
around somewhere (i.e. /usr/lib/foo/{bar,baz}.py), recompiling these
files on python upgrades.

Please design python-central for all kind of packages, do not require
the packages to be adapted to python-central.

> > Conclusion: the python-central framework as it is now offers enough for
> > everyone. If people don't want to use it, they are on their own.
> 
> I think we could add support for the mailman case with minimal hassles...
> 
> The biggest problem I can forsee is this introduces another option that will
> confuse developers... But this can be rectified with "strongly suggests" in
> the python policy.

There should be no confusion telling python-central the locations of
files to be registered.

> > Hmm, reading the Debian policy, the only difference is that Depends only
> > are considered on configuring a package, Conflicts are considered on
> > installing. I dont know whats better, I think either will do.
> > I will adapt the man pages to the Python Policy.
> 
> Give this, I would prefer the "Depends" rather than "Conflicts", as it more
> closely matches what a python version incompatability means... you can
> install the files, but you can't configure them to work.

please send a patch for the policy.

Bastian Kleineidam writes:
> python-central (0.4) unstable; urgency=low
> 
>   * renamed register-python-package to python-register, this way
> its prefixed with "python" (and its shorter ;)

as long as python-central handles libraries/modules only, it should be
called register-python-library. Don't confuse the packager ;-)


Matthias, away 'til Sunday.




Re: python-central 0.4

2002-09-30 Thread Donovan Baarda
On Tue, Oct 01, 2002 at 12:23:38AM +1200, Carey Evans wrote:
> Donovan Baarda wrote:
[...]
> It might also be nice to have separate files that list the directories 
> or source files to compile for each package, and have python-central 
> call compileall itself, but I guess this is a separate issue.

you mean like;

dpkg -L  | grep '.py$'

:-)


-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--




Re: python-central 0.4

2002-09-30 Thread Carey Evans
Donovan Baarda wrote:
But this is redundant... if the package is installed, the corresponding *.py
files _should_ be unpacked. How is testing for the existance of a file any
better indication than testing for installation of the package?
True... I think I'll try to blame my flawed argument on lack of caffeine 
in the bloodstream.

[...]
This is an interesting issue I hadn't considered. However, I can't see how
seperate compilation scripts do a better job of avoiding this problem than
dpkg-reconfigure.
Since, in the absence of debconf, all dpkg-reconfigure does is run
   /var/lib/dpkg/info/package.postinst configure 
/I guess editing the postinst would be fine as well.  /However, it still 
seem to me that having recompilation separate from installing 
documentation, menus, MIME types and whatever else might be worthwhile.

It might also be nice to have separate files that list the directories 
or source files to compile for each package, and have python-central 
call compileall itself, but I guess this is a separate issue.

[...]
So the postinst scripts get passed the version upgraded from? (I should know
this :-)
I should have checked too.  See section 6 of Policy for details, 
especially what can happen if an error occurs, but the postinst is the 
one script that _doesn't_ get called with the other version number.  The 
old version's prerm and postrm, and the new version's preinst, could all 
check whether an incompatible upgrade was going to or had occurred.  In 
this case, I guess they'd set a flag and delete the old *.py[co] files.





Re: python-central 0.4

2002-09-29 Thread Derrick 'dman' Hudson
On Mon, Sep 30, 2002 at 08:19:49AM +1000, Donovan Baarda wrote:
| On Mon, Sep 30, 2002 at 08:13:23AM +1200, Carey Evans wrote:

| > It also makes it easier for users to modify if a Python package's 
| > dependencies are incorrect, and it stops compiling under a newer version 
| > of Python, making the whole dpkg run fail.
| 
| This is an interesting issue I hadn't considered. However, I can't see how
| seperate compilation scripts do a better job of avoiding this problem than
| dpkg-reconfigure.

If there's a malfunctioning script in /somewhere that I can insert
exit 0 ;
or
import sys ; sys.exit( 0 )
at the top, then as a user/admin I can sidestep the installation
error.  If the logic is tucked away in the bowels of some other
program, it isn't as trivial to hack around.

(I don't have a position on this issue, though, I'm just sorta lurking
on the discussions)

-D

-- 
But As for me and my household, we will serve the Lord.
Joshua 24:15
 
http://dman.ddts.net/~dman/


pgp0JFaZZdMpe.pgp
Description: PGP signature


Re: python-central 0.4

2002-09-29 Thread Donovan Baarda
On Sun, Sep 29, 2002 at 05:51:30PM +0200, Bastian Kleineidam wrote:
> On Sun, Sep 29, 2002 at 11:29:02PM +1000, Donovan Baarda wrote:
> > I've just had a look at this and it looks good. It perfectly meets the
> > requirement of allowing pure python module packages to support multiple
> > pythonX.Y python packages simultaniously.
> > 
> > The only problem is we are still missing something. We still can't easily
> > support pure python _program_ packages that work with multiple versions of
> > (the default) python.
> What is wrong with
> Depends: python2.2 | python2.3, python (>= 2.2), python (<< 2.4)
> This supports the 2.2 and upcoming 2.3 default python package.

Yes it does. However, under python central this means you would have all the
modules compiled and symlinked for both python2.2 and python2.3 if they were
both installed. This is great if you want to use these modules from both
pythonX.Y versions, but in the case of something like mailman, you only want
to use them from the default python. Having the other pythonX.Y support is
not required.

> > Programs like mailman put their modules in their own /usr/lib/mailman
> > directory. One way that mailman _could_ support multiple pythonX.Y packages
> > it to have the packager move _all_ the modules into a
> > /usr/lib/python/site-packages directory and then use python central.
> > However, this is overkill for mailman because it doesn't need to support
> > python2.1 and python2.2 simultanously, just the default python. It is also
> > hell for the packager.
> I wouldn't call moving some files the packaging hell, and I have yet to
> understand why /usr/lib/mailman is so much saner or better than
> /usr/lib/python/site-packages/mailman.
> I looked into the mailman package. It should not be that much work to
> adapt it to the python-central framework (moving the Mailman module
> into .../site-packages/, removing the paths.py hack).
> And when the mailman Bug#162761 is fixed, I could even provide a patch ;)

I guess this is an issue of "upstream compatiblity". Sure, packages could be
restructured to fit the scheme, but the other alternative is to masssage the
scheme to fit the upstream packages...

I think that using /usr/lib/python/site-packages/mailman properly is the
_right_ way to do it, but I can see the argument for /usr/lib/mailman,
particularly when upstream does it that way.

> Conclusion: the python-central framework as it is now offers enough for
> everyone. If people don't want to use it, they are on their own.

I think we could add support for the mailman case with minimal hassles...

The biggest problem I can forsee is this introduces another option that will
confuse developers... But this can be rectified with "strongly suggests" in
the python policy.

> > Another thing I noticed, the man pages are now recommending using;
> > 
> >Depends: python (>=2.1)
> >Conflicts: python (>= 2.4)
> >   
> > Doesn't the policy have;
> > 
> >Depends: python (>=2.1), python (<<2.4)
> > 
> > I think these mean the same thing, but has the policy changed for some
> > reason? I actually think that "Conflicts" is less ambiguous...
> 
> Hmm, reading the Debian policy, the only difference is that Depends only
> are considered on configuring a package, Conflicts are considered on
> installing. I dont know whats better, I think either will do.
> I will adapt the man pages to the Python Policy.

Give this, I would prefer the "Depends" rather than "Conflicts", as it more
closely matches what a python version incompatability means... you can
install the files, but you can't configure them to work.

> > Pending feedback/ideas whatever, I can code something up to do this. I
> > probably favor using python-apt (if it can actualy do it) or grep-dctrl to
> > do the package lookup stuff, but can do it manually if this is a problem.
> Dont code anything like that. I'd rather fix those hacky'ish packages
> which try to append something to sys.path.

I think this needs feedback from the people actually maintaining the
packages. If they think restructuring into /var/lib/python/site-packages is a
good idea, then fine. However, if they would prefer to structure their files
the same as upstream does...


-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--




Re: python-central 0.4

2002-09-29 Thread Donovan Baarda
On Mon, Sep 30, 2002 at 08:13:23AM +1200, Carey Evans wrote:
> Donovan Baarda wrote:
> 
> >finding all packages that depend on "python" is non-trivial using only 
> >dpkg.
> >Something like dpkg-awk, grep-dctrl, or python-apt make it much easier, but
> >do we want to depend on them? The old "registry" idea would have made this 
> >a
> >little easier, but I still prefer using the dpkg database to find this kind
> >of info.
> 
> I prefer myself the way that packages like emacsen-common, mime-support, 
> menu and I'm sure others do this: by having every package using Python 
> and wanting to be recompiled install a script in (for example) 
> /usr/share/python-central/packages/, which can be called 
> whenever .py files need recompiling.  Alternatively, it could just be a 
> configuration file with paths to the files to recompile.

python-central originally did have a "repository" where info about installed
packages was put that could then be used to recompile/whatever. However, it
was removed because there was nothing that this "repository" did that
couldn't be done by using the existing dpkg database, so it just included
redundant information that could have potentialy got out of sync.

I don't think that having a seperate compile script for each package
at /usr/share/python-central/packages/ is better than just
running "dpkg-reconfigure ". The only slightly tricky thing is
finding which packages need to be reconfigured, but all this info _is_ in
the dpkg database dependancies.

> Separate files that are part of the package would do the best job of 
> existing only when the corresponding .py files are also unpacked.

But this is redundant... if the package is installed, the corresponding *.py
files _should_ be unpacked. How is testing for the existance of a file any
better indication than testing for installation of the package?

> It also makes it easier for users to modify if a Python package's 
> dependencies are incorrect, and it stops compiling under a newer version 
> of Python, making the whole dpkg run fail.

This is an interesting issue I hadn't considered. However, I can't see how
seperate compilation scripts do a better job of avoiding this problem than
dpkg-reconfigure.

> >We only need to reconfigure/recompile when the default version of python
> >upgrades to a different version of python, not minor package upgrades. Is
> >there any way we can detect this and avoid un-necisary recompiles?
> 
> The python package itself knows from the arguments to its postinst, and 
> could pass this on to python-central.

So the postinst scripts get passed the version upgraded from? (I should know
this :-)

-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--




Re: python-central 0.4

2002-09-29 Thread Carey Evans
Donovan Baarda wrote:
finding all packages that depend on "python" is non-trivial using only dpkg.
Something like dpkg-awk, grep-dctrl, or python-apt make it much easier, but
do we want to depend on them? The old "registry" idea would have made this a
little easier, but I still prefer using the dpkg database to find this kind
of info.
I prefer myself the way that packages like emacsen-common, mime-support, 
menu and I'm sure others do this: by having every package using Python 
and wanting to be recompiled install a script in (for example) 
/usr/share/python-central/packages/, which can be called 
whenever .py files need recompiling.  Alternatively, it could just be a 
configuration file with paths to the files to recompile.

Separate files that are part of the package would do the best job of 
existing only when the corresponding .py files are also unpacked.

It also makes it easier for users to modify if a Python package's 
dependencies are incorrect, and it stops compiling under a newer version 
of Python, making the whole dpkg run fail.

We only need to reconfigure/recompile when the default version of python
upgrades to a different version of python, not minor package upgrades. Is
there any way we can detect this and avoid un-necisary recompiles?
The python package itself knows from the arguments to its postinst, and 
could pass this on to python-central.

--
Carey




Re: python-central 0.4

2002-09-29 Thread Bastian Kleineidam
On Sun, Sep 29, 2002 at 11:29:02PM +1000, Donovan Baarda wrote:
> I've just had a look at this and it looks good. It perfectly meets the
> requirement of allowing pure python module packages to support multiple
> pythonX.Y python packages simultaniously.
> 
> The only problem is we are still missing something. We still can't easily
> support pure python _program_ packages that work with multiple versions of
> (the default) python.
What is wrong with
Depends: python2.2 | python2.3, python (>= 2.2), python (<< 2.4)
This supports the 2.2 and upcoming 2.3 default python package.

> Programs like mailman put their modules in their own /usr/lib/mailman
> directory. One way that mailman _could_ support multiple pythonX.Y packages
> it to have the packager move _all_ the modules into a
> /usr/lib/python/site-packages directory and then use python central.
> However, this is overkill for mailman because it doesn't need to support
> python2.1 and python2.2 simultanously, just the default python. It is also
> hell for the packager.
I wouldn't call moving some files the packaging hell, and I have yet to
understand why /usr/lib/mailman is so much saner or better than
/usr/lib/python/site-packages/mailman.
I looked into the mailman package. It should not be that much work to
adapt it to the python-central framework (moving the Mailman module
into .../site-packages/, removing the paths.py hack).
And when the mailman Bug#162761 is fixed, I could even provide a patch ;)

Conclusion: the python-central framework as it is now offers enough for
everyone. If people don't want to use it, they are on their own.

> Another thing I noticed, the man pages are now recommending using;
> 
>Depends: python (>=2.1)
>Conflicts: python (>= 2.4)
>   
> Doesn't the policy have;
> 
>Depends: python (>=2.1), python (<<2.4)
> 
> I think these mean the same thing, but has the policy changed for some
> reason? I actually think that "Conflicts" is less ambiguous...

Hmm, reading the Debian policy, the only difference is that Depends only
are considered on configuring a package, Conflicts are considered on
installing. I dont know whats better, I think either will do.
I will adapt the man pages to the Python Policy.


> Pending feedback/ideas whatever, I can code something up to do this. I
> probably favor using python-apt (if it can actualy do it) or grep-dctrl to
> do the package lookup stuff, but can do it manually if this is a problem.
Dont code anything like that. I'd rather fix those hacky'ish packages
which try to append something to sys.path.

Cheers,
Bastian
-- 
 Bastian Kleineidam

 reflexionsniveau AT web.de
   calvin AT users.sf.net
calvin AT debian.org


pgpG7b6H6swnL.pgp
Description: PGP signature


Re: python-central 0.4

2002-09-29 Thread Donovan Baarda
On Sun, Sep 29, 2002 at 12:26:30AM +0200, Bastian Kleineidam wrote:
> Hi,
> 
> uploaded the new version 0.4 at
> http://people.debian.org/~calvin/python-central/
> 
> python-central (0.4) unstable; urgency=low
> 
>   * renamed register-python-package to python-register, this way
> its prefixed with "python" (and its shorter ;)
>   * add options to enable/suppress the compiling of .py[co] files
>   * add a new command "python-compile" to compile single python
> files
>   * Added a separate section in python-register.1 for python programs;
> also give an example.
> 
>  -- Bastian Kleineidam <[EMAIL PROTECTED]>  Fri, 13 Sep 2002 22:17:45 +0200

I've just had a look at this and it looks good. It perfectly meets the
requirement of allowing pure python module packages to support multiple
pythonX.Y python packages simultaniously.

The only problem is we are still missing something. We still can't easily
support pure python _program_ packages that work with multiple versions of
(the default) python.

Programs like mailman have large mailman specific modules that are used only
by the mailman program. The mailman program doesn't need to support multiple
pythonX.Y packages simultaniously, just multiple versions of the default
python. This is in many ways a simpler requirement than what python-central
solves, but we still haven't got it covered.

Programs like mailman put their modules in their own /usr/lib/mailman
directory. One way that mailman _could_ support multiple pythonX.Y packages
it to have the packager move _all_ the modules into a
/usr/lib/python/site-packages directory and then use python central.
However, this is overkill for mailman because it doesn't need to support
python2.1 and python2.2 simultanously, just the default python. It is also
hell for the packager.

The only problem programs like mailman have is if the default python
upgrades after they are installed, they need to be recompiled. The best way
to handle this is to have the default python packages run dpkg-reconfigure
on all installed packages that depend on python.

This could be added to python central. Currently it auto-detects whether a
python module or pythonX.Y package is being installed by looking at the
package name, and behaves accordingly. All that needs to be added is to have
python-central check for the package name "python", and have it do the
dpkg-reconfigure for all packages depending on python. We then have the
default python packages "register" themselves using python-central.

Now some random thoughts;

finding all packages that depend on "python" is non-trivial using only dpkg.
Something like dpkg-awk, grep-dctrl, or python-apt make it much easier, but
do we want to depend on them? The old "registry" idea would have made this a
little easier, but I still prefer using the dpkg database to find this kind
of info.

Does dpkg-reconfigure actually run the postinst script? Does it do it in a
way (right parameters etc) that would make exising python packages
re-compile properly?

Is a dpkg-reconfigure overkill? We only need to recompile the *.py's. Do
some packages really need to go through a dpkg-reconfigure anyway? What if
they are using debconf options for things like "compile modules"?

We only need to reconfigure/recompile when the default version of python
upgrades to a different version of python, not minor package upgrades. Is
there any way we can detect this and avoid un-necisary recompiles?

Should python-central be providing compile scripts for use by all python
packages? Should we be making use of these scripts manditory in the
Python Policy? If it was then we should be safe using these scripts to
recompile packages instead of using dpkg-reconfigure. But would it be any
better? Is compiling in postinst scripts so trivial as to not need support
scripts?

Another thing I noticed, the man pages are now recommending using;

   Depends: python (>=2.1)
   Conflicts: python (>= 2.4)
  
Doesn't the policy have;

   Depends: python (>=2.1), python (<<2.4)

I think these mean the same thing, but has the policy changed for some
reason? I actually think that "Conflicts" is less ambiguous...

Pending feedback/ideas whatever, I can code something up to do this. I
probably favor using python-apt (if it can actualy do it) or grep-dctrl to
do the package lookup stuff, but can do it manually if this is a problem.

-- 
--
ABO: finger [EMAIL PROTECTED] for more info, including pgp key
--




python-central 0.4

2002-09-28 Thread Bastian Kleineidam
Hi,

uploaded the new version 0.4 at
http://people.debian.org/~calvin/python-central/

python-central (0.4) unstable; urgency=low

  * renamed register-python-package to python-register, this way
its prefixed with "python" (and its shorter ;)
  * add options to enable/suppress the compiling of .py[co] files
  * add a new command "python-compile" to compile single python
files
  * Added a separate section in python-register.1 for python programs;
also give an example.

 -- Bastian Kleineidam <[EMAIL PROTECTED]>  Fri, 13 Sep 2002 22:17:45 +0200


cu, 
Bastian
-- 
 Bastian Kleineidam

 reflexionsniveau AT web.de
   calvin AT users.sf.net
calvin AT debian.org


pgph75unJxJ81.pgp
Description: PGP signature