Re: [Python-Dev] The docs, reloaded

[Laurent Gautier]

I would agree with the point that python core should be considered
first, but I would also only see beneficial to leave the door open
to the need of other packages.

I have (briefly but intensely) worked on a revamp of pydoc earlier on
this year, and while collecting requirements from a number of places
having maths expressions or else appeared important for a number
of cases (and a very reasonable request in a case) .

That particular point leads to something that I see important for what
a new/better documentation system should provide: a good and
modular interface to access the documentation, process it,
and navigate it.

When looking at the particular example discussed here, it could be
implemented by allowing a "pluggable" processing components for
docstrings (and let a given package developer the possibility to use as
much as the default documentation machinery as possible and
implement the processing mathml, latex, whatever, as wanted).
One can consider the possibility to have the "custom" processing of
the docstring embedded in the package itself.



Laurent



2007/5/21, Brett Cannon <[EMAIL PROTECTED]>:
>
>
> On 5/20/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> > > Georg> So, that's not really a concern of mine ;)
> > >
> > > You must realize that people will use the core tools to create
> documentation
> > > for third party packages which aren't in the core.  If you replace LaTeX
> > > with something else I think you need to keep math in mind whether it's
> used
> > > in the core documentation or not.
> >
> > I disagree. The documentation infrastructure of Python should only
> > consider the needs of Python itself. If other people can use that
> > infrastructure for other purposes, fine - if they find that it does
> > not meet their needs, they have to look elsewhere.
>
>
> Martin beat me to my comment.  =)  Python's needs should come first, period.
>  If Georg wants to add math support, fine.  But honestly I would rather he
> spend his time on Python-specific stuff then get bogged down to support
> possible third parties.
>
> -Brett
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Nick Craig-Wood]

Georg Brandl <[EMAIL PROTECTED]> wrote:
>  over the last few weeks I've hacked on a new approach to Python's 
> documentation.
>  As Python already has an excellent documentation framework, the docutils, 
> with a
>  readable yet extendable markup format, reST, I thought that it should be
>  possible to use those instead of the current LaTeX->latex2html
>  toolchain.

Good idea!

Latex is a barrier for contribution to the docs.  I imagine most
people would be much better at contributing to the docs in reST.  (Me
included: I learnt latex at university a couple of decades ago and
have now forgotten it completely!)

>  - a "quick-dispatch" function: e.g., docs.python.org/q?os.path.split would
> redirect you to the matching location.

Being a seasoned unix user, I tend to reach for pydoc as my first stab
at finding some documentation rather than (after excavating the mouse
from under a pile of paper) use a web browser.

If you've ever used pydoc you'll know it reads docstrings and for some
modules they are great and for others they are sorely lacking.

If pydoc could show all this documentation as well I'd be very happy!

Maybe your quick dispatch feature could be added to pydoc too?

>  Concluding, a small caveat: The conversion/build tools are, of course, not
>  complete yet. There are a number of XXX comments in the text, most of them
>  indicate that the converter could not handle a situation -- that would have
>  to be corrected once after conversion is done.

It is missing conversion of ``comment'' at the moment as I'm sure you
know...

You will need to make your conversion perfect before you convince the
people who wrote most of that documentation I suspect!

-- 
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Kristján Valur Jónsson]

First of all, I have put some work into pcbuild8 recently and it works
well.  I am trying to drum up momentum for work on PCBuild8
next europython.  See http://wiki.python.org/moin/EuroPython2007Sprints


> -Original Message-
> From: [EMAIL PROTECTED]
>
> I don't find the need to have separate object directories convincing:
> For building the Win32/Win64 binaries, I have separate checkouts
> *anyway*, since all the add-on libraries would have to support
> multi-arch builds, but I think they don't.

No they don't, but that doesn't mean that you need different checkouts
for python, only the others.  Anyway, this is indeed something I'd like
to see addressed.  I don't think we should ditch cross-compilation.  It
should simplify a lot of stuff, including buildbot setup and so on (if
we get the buildbot infrastructure to support it).  It is also very
cumbersome, if you are working on a project, to have the binaries all
end up in the same place.  Doing interactive work on python, I frequently
compile both the 32 bit and 64 bit versions for testing and it would
be downright silly to have to rebuild everything every time.

> I would personally like to see Python "skip" VS 2005 altogether,
> as it will be soon superceded by Orcas. Unfortunately, it's unclear
> how long Microsoft will need to release Orcas (and also, when Python
> 2.6 will be released), so I would like to defer that question by
> a few months.
I think this is a bit unrealistic.  Here we are in the middle of 2007,
VS2005 has just got SP1, and VS2003 is still the "official" compiler.
PCBuild8 is ready, it just needs a little bit of extra love and
buildbots to make us able to release PGO versions of x86 and x64.

Given the delay for getting even this far, waiting for Orcas and then
someone to create PCBuild9, and then getting it up and running and so on
will mean waiting another two years.

> The MSI packaging would need to be changed, certainly. It currently
> detects the architecture it needs to package by looking at the file
> type of python.exe; that would have to be changed to give it an
> explicit parameter what architecture to package, or have it package
> all architectures it can find.

I am not familiar with the msi packaging process at all.  But here is
something we should start to consider:  VISTA support.  This could mean
some of:
1) supplying python.dll as a Side By Side assembly
2) Changing python install locations
3) Supporting shadow libraries, where .pyc files end up in a different
   hierarchy from the .py files.  (useful for many things beside VISTA)
4) Signing the python dlls and executables
5) Providing user level manifests.

Vista adoption is going very fast.  We see 10% of our users have moved
to vista and rising.



> I'd rather make it a platform-specific definition (for platform=Windows
> API). Correct me if I'm wrong, but isn't wchar_t also available in VS
> 2003 (and even in VC6?). And doesn't it have the "right" definition in
> all these compilers?
>
> So +1 for setting Py_UNICODE to wchar_t on Windows.

Yes.  Btw, in previous visual studio versions, wchar_t was not treated
as a builtin type by default, but rather as synonymous with unsighed short.
Now the default is that it is, and this causes some semantic differences
and incompatibilities of the type seen.

Kristjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[M.-A. Lemburg]

On 2007-05-21 12:30, Kristján Valur Jónsson wrote:
>>
>> [Py_UNICODE being #defined as "unsigned short" on Windows]
>>
>> I'd rather make it a platform-specific definition (for platform=Windows
>> API). Correct me if I'm wrong, but isn't wchar_t also available in VS
>> 2003 (and even in VC6?). And doesn't it have the "right" definition in
>> all these compilers?
>>
>> So +1 for setting Py_UNICODE to wchar_t on Windows.
> 
> Yes.  Btw, in previous visual studio versions, wchar_t was not treated
> as a builtin type by default, but rather as synonymous with unsighed short.
> Now the default is that it is, and this causes some semantic differences
> and incompatibilities of the type seen.

+1 from me.

If think this is simply a bug introduced with the UCS4 patches in
Python 2.2.

unicodeobject.h already has this code:

#ifndef PY_UNICODE_TYPE

/* Windows has a usable wchar_t type (unless we're using UCS-4) */
# if defined(MS_WIN32) && Py_UNICODE_SIZE == 2
#  define HAVE_USABLE_WCHAR_T
#  define PY_UNICODE_TYPE wchar_t
# endif

# if defined(Py_UNICODE_WIDE)
#  define PY_UNICODE_TYPE Py_UCS4
# endif

#endif

But for some reason, pyconfig.h defines:

/* Define as the integral type used for Unicode representation. */
#define PY_UNICODE_TYPE unsigned short

/* Define as the size of the unicode type. */
#define Py_UNICODE_SIZE SIZEOF_SHORT

/* Define if you have a useable wchar_t type defined in wchar.h; useable
   means wchar_t must be 16-bit unsigned type. (see
   Include/unicodeobject.h). */
#if Py_UNICODE_SIZE == 2
#define HAVE_USABLE_WCHAR_T
#endif

disabling the default settings in the unicodeobject.h.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 21 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Kristján Valur Jónsson]

> -Original Message-
> This was in C++, but the problem was really WCHAR, as used by much of
> the
> win32 API.
>
> > I'd rather make it a platform-specific definition (for
> > platform=Windows
> > API). Correct me if I'm wrong, but isn't wchar_t also available in VS
> > 2003 (and even in VC6?). And doesn't it have the "right" definition
> in
> > all these compilers?
>
> hrm - as above, I'm more concerned with the definition of WCHAR - which
> means my problem is related more to the Platform SDK version rather
> than the
> compiler.  This is unfortunate - on one hand we do consider
> 'platform=Windows API', and WCHAR is very much an API concept.  I'll
> need to
> dig some more into this, but at least I know I'm not wasting my time :)

Mark, your problem may be related to a setting in the "c/c++ -> language"
tab in the settings, where "treat wchar_t as a builtin type" default has
changed.  I recommend that we do treat it as a builtin, but the VS2003
default was "no" and the 2005 is "yes".  Could this be contributing to
your problem?

Kristjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[M.-A. Lemburg]

On 2007-05-21 00:07, Talin wrote:
> Phillip J. Eby wrote:
>> I wanted to get this in before the Py3K PEP deadline, since this is a 
>> Python 2.6 PEP that would presumably impact 3.x as well.  Feedback welcome.
>>
>>
>> PEP: 365
>> Title: Adding the pkg_resources module
> 
> I'm really surprised that there hasn't been more comment on this.

True both ways, I guess: I'm still waiting for a reply to my
comments.

I'd also like to see more discussion about adding e.g.:

 * support for user packages

   (ie. having site.py add a well-defined user home directory
   based Python path entry to sys.path, e.g.
   ~/.python/user-packages, much like what MacPython already does
   now)

 * support for having the import mechanism play nice
   with namespace packages

   (ie. packages that may live in different places on the disk,
   but appear to be in the same Python package as seen by the
   import mechanism)

I think those two features would go a long way in reducing the
number of hacks setuptools currently applies to get this
functionality working with code in .pth files, monkey-patching
site.py, etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 21 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] wchar_t (was Adventures with x64, VS7 and VS8) on Windows

[Mark Hammond]

Kristján Valur Jónsson quoting me:
> > hrm - as above, I'm more concerned with the definition of 
> > WCHAR - which
> > means my problem is related more to the Platform SDK version rather
> > than the
> > compiler.  This is unfortunate - on one hand we do consider
> > 'platform=Windows API', and WCHAR is very much an API concept.  I'll
> > need to
> > dig some more into this, but at least I know I'm not 
> > wasting my time :)
> 
> Mark, your problem may be related to a setting in the "c/c++ 
> -> language"
> tab in the settings, where "treat wchar_t as a builtin type" 
> default has
> changed.  I recommend that we do treat it as a builtin, but the VS2003
> default was "no" and the 2005 is "yes".  Could this be contributing to
> your problem?

Thanks for the suggestion and for introducing me to that option - but it
made no difference.  I'm guessing its related to C++ - code such as the
following:

static PyObject *TestIBuild()
{
// obviously nonsense - the point is to test if it compiles.
WCHAR *wval = PyUnicode_AS_UNICODE(Py_None);
return PyUnicode_FromUnicode(wval, wcslen(wval));
}

works everywhere - except in a pywin32 .cpp file built on x64:)  That code
results in:

win32/src/win32apimodule.cpp(81) : error C2440: 'initializing' : cannot
convert from 'Py_UNICODE *' to 'WCHAR *'
Types pointed to are unrelated; conversion requires
reinterpret_cast, C-style cast or function-style cast
win32/src/win32apimodule.cpp(82) : error C2664: 'PyUnicodeUCS2_FromUnicode'
: cannot convert parameter 1 from 'WCHAR *' to 'const Py_UNICODE *'
Types pointed to are unrelated; conversion requires
reinterpret_cast, C-style cast or function-style cast

Unfortunately I don't have pywin32 building under vc8 on x32, but expect it
to happen there too.  pywin32 uses distutils, but I inspected the options
passed and can't find anything to make a difference.  /Zc:wchar_t and/or
/Zc:wchar_t- seem to be the command-line settings for this flag and it also
makes no difference.  I'm out of time to confirm is is simply "c++ with
vs8", but did confirm that the patch below appears to solve the problem, and
given Martin's previous +1, I decided to stop there.  I failed in a quick
attempt at replacing the literal 2 with something involving sizeof.  Does
this look reasonable?

Cheers,

Mark

Index: pyconfig.h
===
--- pyconfig.h  (revision 55487)
+++ pyconfig.h  (working copy)
@@ -492,10 +492,10 @@
 #define Py_USING_UNICODE

 /* Define as the integral type used for Unicode representation. */
-#define PY_UNICODE_TYPE unsigned short
+#define PY_UNICODE_TYPE wchar_t

 /* Define as the size of the unicode type. */
-#define Py_UNICODE_SIZE SIZEOF_SHORT
+#define Py_UNICODE_SIZE 2

 /* Define if you have a useable wchar_t type defined in wchar.h; useable
means wchar_t must be 16-bit unsigned type. (see


<>___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] wchar_t (was Adventures with x64, VS7 and VS8) on Windows

[Kristján Valur Jónsson]

> -Original Message-
> below appears to solve the problem, and given Martin's previous +1, I
> decided to stop there.  I failed in a quick attempt at replacing the
> literal 2 with something involving sizeof.  Does this look reasonable?
>
 +1.
I should add that we have this local mod in our own patched
python.  It does seem like a mistake that needs fixing.

Kristjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[skip]

Brett> Martin beat me to my comment.  =) Python's needs should come
Brett> first, period.  If Georg wants to add math support, fine.  But
Brett> honestly I would rather he spend his time on Python-specific
Brett> stuff then get bogged down to support possible third parties.

I think the people who have responded to my comment read too much into it.
Nowhere do I think I asked Georg to write an equation typesetter to include
in the Python documentation toolchain.  I asked that math capability be
considered.  I have no idea what tools he used to build his new
documentation set.  I only briefly glanced at a couple of the output pages.
I think what he has done is marvelous.  However, I don't think the door
should be shut on equation display.  Is there a route to it based on the
tools Georg is using?  If not, then I think some accommodation should be
made.  I'm being vague here on purpose because I'm unfamiliar with the
available tools.  The one thing I do know is that LaTeX provides that today
and by removing it from the toolchain you have removed a significant piece
of functionality.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[skip]

>> You must realize that people will use the core tools to create
>> documentation for third party packages which aren't in the core.  If
>> you replace LaTeX with something else I think you need to keep math
>> in mind whether it's used in the core documentation or not.

Martin> I disagree. The documentation infrastructure of Python should
Martin> only consider the needs of Python itself. If other people can
Martin> use that infrastructure for other purposes, fine - if they find
Martin> that it does not meet their needs, they have to look elsewhere.

Then I submit that you are probably removing some significant piece of
functionality from the provided documentation toolchain which some people
probably rely on.  After all, that's what LaTeX excels at.  They will be
able to continue to use the old tools, but where will they get them if they
are no longer part of Python?

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Mark Hammond]

Hi Kristján,

> First of all, I have put some work into pcbuild8 recently and it works
> well.

It does!  There are a few issues though, notably with distutils (and as
mentioned before, any other tools what may assume PCBuild - see below)

You quoting Martin:
> > I don't find the need to have separate object directories 
> > convincing:
> > For building the Win32/Win64 binaries, I have separate checkouts
> > *anyway*, since all the add-on libraries would have to support
> > multi-arch builds, but I think they don't.
> 
> No they don't, but that doesn't mean that you need different checkouts
> for python, only the others.  Anyway, this is indeed 
> something I'd like to see addressed.  I don't think we should ditch 
> cross-compilation.

While I agree with you, Martin's point about the dependant libraries and
tools is valid, and may defeat this goal.  In the short term, we should
research how some of these other projects are approaching x64 - we may also
find that this helps with any autoconf work we choose to do.  Ultimately,
the person releasing the official binaries gets to say how this works (based
on how they actually get it to work)

> > I would personally like to see Python "skip" VS 2005 altogether,
> > as it will be soon superceded by Orcas. Unfortunately, it's unclear
> > how long Microsoft will need to release Orcas (and also, when Python
> > 2.6 will be released), so I would like to defer that question by
> > a few months.
> I think this is a bit unrealistic.  Here we are in the middle of 2007,
> VS2005 has just got SP1, and VS2003 is still the "official" compiler.
> PCBuild8 is ready, it just needs a little bit of extra love and
> buildbots to make us able to release PGO versions of x86 and x64.
> 
> Given the delay for getting even this far, waiting for Orcas and then
> someone to create PCBuild9, and then getting it up and 
> running and so on
> will mean waiting another two years.

I don't believe there was any suggestion that Python 2.6 would wait for a
compiler release from Microsoft.

Before we talk about Vista and while I have your attention , some
final questions relating to PCBuild8.  Regardless of the ultimate layout for
x64, what do you think about having PCBuid8 put the binaries into the
PCBuild directory, and thus (theoretically) letting such a directory work
with distutils and otherwise as a fully functional Python installation?

> I am not familiar with the msi packaging process at all.  But here is
> something we should start to consider:  VISTA support.  This 
> could mean
> some of:
> 1) supplying python.dll as a Side By Side assembly

Yes, this is something pywin32 is going to face.  The hack of copying
python*.dll into the 'system' directory - necessary for COM - is (sensibly!)
no longer working.  One thing at a time though...

> 2) Changing python install locations
> 3) Supporting shadow libraries, where .pyc files end up in a different
>hierarchy from the .py files.  (useful for many things 
> beside VISTA)
> 4) Signing the python dlls and executables
> 5) Providing user level manifests.

And dragging distutils back op topic, having bdist_wininst supply a manifest
that indicates escalation is required appears necessary.

> Vista adoption is going very fast.  We see 10% of our users have moved
> to vista and rising.

ack - I'm yet to try a 32 bit version, but my Vista-x64 box isn't proving
very reliable yet.  It *is* very pretty and cute though :)  I'm surprised to
see 10% unless your users are skewed towards early-adopters though, but I'm
in no position to refute it!

Cheers,

Mark
<>___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Fred L. Drake, Jr.]

On Monday 21 May 2007, [EMAIL PROTECTED] wrote:
 > Then I submit that you are probably removing some significant piece of
 > functionality from the provided documentation toolchain which some people
 > probably rely on.  After all, that's what LaTeX excels at.  They will be
 > able to continue to use the old tools, but where will they get them if
 > they are no longer part of Python?

I'll be happy to pull the existing tools out into a separate distribution if 
we move to something else for Python.  There are too many users of the 
existing tools to abandon.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 01:43 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>On 2007-05-21 00:07, Talin wrote:
> > Phillip J. Eby wrote:
> >> I wanted to get this in before the Py3K PEP deadline, since this is a
> >> Python 2.6 PEP that would presumably impact 3.x as 
> well.  Feedback welcome.
> >>
> >>
> >> PEP: 365
> >> Title: Adding the pkg_resources module
> >
> > I'm really surprised that there hasn't been more comment on this.
>
>True both ways, I guess: I'm still waiting for a reply to my
>comments.

What comments are you talking about?  I must've missed them.


>I'd also like to see more discussion about adding e.g.:
>
>  * support for user packages
>
>(ie. having site.py add a well-defined user home directory
>based Python path entry to sys.path, e.g.
>~/.python/user-packages, much like what MacPython already does
>now)
>
>  * support for having the import mechanism play nice
>with namespace packages
>
>(ie. packages that may live in different places on the disk,
>but appear to be in the same Python package as seen by the
>import mechanism)
>
>I think those two features would go a long way in reducing the
>number of hacks setuptools currently applies to get this
>functionality working with code in .pth files, monkey-patching
>site.py, etc.

These items aren't directly related to the PEP, 
however.  pkg_resources doesn't monkeypatch anything or touch any 
.pth files.  It only changes sys.path at runtime if you explicitly 
ask it to locate and activate packages for you.

As for namespace packages, pkg_resources provides a more PEP 
302-compatible alternative to pkgutil.extend_path().  pkgutil doesn't 
support anything but existing filesystem directories, but the 
pkg_resources version supports zipfiles and has hooks to allow 
namespace package support to be registered for any PEP 302 importer.  See:

http://peak.telecommunity.com/DevCenter/PkgResources#supporting-custom-importers

(specifically, the register_namespace_handler() function.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[M.-A. Lemburg]

On 2007-05-21 16:05, Phillip J. Eby wrote:
> At 01:43 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>> On 2007-05-21 00:07, Talin wrote:
>>> Phillip J. Eby wrote:
 I wanted to get this in before the Py3K PEP deadline, since this is a
 Python 2.6 PEP that would presumably impact 3.x as 
>> well.  Feedback welcome.

 PEP: 365
 Title: Adding the pkg_resources module
>>> I'm really surprised that there hasn't been more comment on this.
>> True both ways, I guess: I'm still waiting for a reply to my
>> comments.
> 
> What comments are you talking about?  I must've missed them.

I've attached the email. Please see below.

>> I'd also like to see more discussion about adding e.g.:
>>
>>  * support for user packages
>>
>>(ie. having site.py add a well-defined user home directory
>>based Python path entry to sys.path, e.g.
>>~/.python/user-packages, much like what MacPython already does
>>now)
>>
>>  * support for having the import mechanism play nice
>>with namespace packages
>>
>>(ie. packages that may live in different places on the disk,
>>but appear to be in the same Python package as seen by the
>>import mechanism)
>>
>> I think those two features would go a long way in reducing the
>> number of hacks setuptools currently applies to get this
>> functionality working with code in .pth files, monkey-patching
>> site.py, etc.
> 
> These items aren't directly related to the PEP, 
> however. 

Right. I wasn't referring to this PEP. I think we should have
two more PEPs covering the above points, since they offer
benefits for all users, not just setuptools users.

> pkg_resources doesn't monkeypatch anything or touch any 
> .pth files.  It only changes sys.path at runtime if you explicitly 
> ask it to locate and activate packages for you.
>
> As for namespace packages, pkg_resources provides a more PEP 
> 302-compatible alternative to pkgutil.extend_path().  pkgutil doesn't 
> support anything but existing filesystem directories, but the 
> pkg_resources version supports zipfiles and has hooks to allow 
> namespace package support to be registered for any PEP 302 importer.  See:
> 
> http://peak.telecommunity.com/DevCenter/PkgResources#supporting-custom-importers
> 
> (specifically, the register_namespace_handler() function.)

Looking at the code it appears as if you've already formalized
an implementation for this.

However, since this is not egg-specific it should probably be
moved to pkgutil and get a separate PEP with detailed documentation
(the link you provided doesn't really explain the concepts, reading
the code helped a bit).

What I don't understand about your approach is why importers
would have to register with the namespace implementation.

This doesn't seem necessary, since the package __path__ attribute
already provides all functionality needed for redirecting
lookups to different paths.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 21 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
--- Begin Message ---
On 2007-05-01 02:29, Phillip J. Eby wrote:
> I wanted to get this in before the Py3K PEP deadline, since this is a 
> Python 2.6 PEP that would presumably impact 3.x as well.  Feedback welcome.

Could you add a section that explains the side effects of
importing pkg_resources ?

The documentation of the module doesn't mention any, but the
code suggests that you are installing (some form of) import
hooks.

Some other comments:

* Wouldn't it be better to factor out all the meta-data access
  code that's not related to eggs into pkgutil ?!

* How about then renaming the remaining module to egglib ?!

* The module needs some reorganization: imports, globals and constants
  at the top, maybe a few comments delimiting the various sections,

* The get_*_platform() should probably use the platform module
  which is a lot more flexible than distutils' get_platform()
  (which should probably use the platform module as well in the
  long run)

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 04 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,

Re: [Python-Dev] The docs, reloaded

[Bill Janssen]

> Could this be a language-independent documenting toolkit?  Could
> we document LISP or Ruby code with it?

Might want to look at "noweb", http://www.eecs.harvard.edu/~nr/noweb/:

``...noweb works ``out of the box'' with any programming language, and
supports TeX, latex, HTML, and troff back ends.''

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Bill Janssen]

> We are developing a programming language here, not a typesetting
> system.

Good point, Martin.  Are you implying that the documentation should be
kept in LaTeX, a widely-accepted widely-disseminated stable
documentation language, which someone else maintains, rather than
ReST, which elements of the the Python community maintain?

Bill




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>However, since this is not egg-specific it should probably be
>moved to pkgutil and get a separate PEP with detailed documentation
>(the link you provided doesn't really explain the concepts, reading
>the code helped a bit).

That doesn't really make sense in the context of the current PEP, 
though, which isn't to provide a general-purpose namespace package 
API; it's specifically about adding an existing piece of code to the 
stdlib, with its API intact.


>What I don't understand about your approach is why importers
>would have to register with the namespace implementation.
>
>This doesn't seem necessary, since the package __path__ attribute
>already provides all functionality needed for redirecting
>lookups to different paths.

The registration is so that when new paths are *added* to sys.path at 
runtime (e.g. by activating a plugin), the necessary __path__ lists 
are automatically updated.  Similarly, when new namespace packages 
are declared, they need their __path__ updated for everything that's 
currently on sys.path.

Finally, there is no requirement that PEP 302 importer strings use 
filesystem-path syntax; a handler has to be registered so that the 
necessary string transformation can be done according to that 
particular importer's string format.  It just happens that zipfiles 
and regular files have a common syntax.  But a URL-based importer, 
LDAP-DN based importer, SQL importer, or other exotica might require 
entirely different string transformations.  All that PEP 302 
guarantees is that sys.path and __path__ lists contain strings, not 
what the format of those strings is.


>Could you add a section that explains the side effects of
>importing pkg_resources ?

The short answer is, there are no side effects, unless 
__main__.__requires__ exists.  If that variable exists, pkg_resources 
attempts to adjust sys.path to contain the requested package 
versions, even if it requires re-ordering the current sys.path 
contents to prevent conflicting versions from being imported.


>The documentation of the module doesn't mention any, but the
>code suggests that you are installing (some form of) import
>hooks.

There are no import hooks, just a registry for registering handlers 
to support other importer types (as seen at the doc link I gave previously).


>Some other comments:
>
>* Wouldn't it be better to factor out all the meta-data access
>   code that's not related to eggs into pkgutil ?!
>
>* How about then renaming the remaining module to egglib ?!

These changes in particular would negate the primary purpose of the 
PEP: to provide a migration path for existing packages using the 
pkg_resources API, including setuptools, workingenv, zc.buildout, etc.


>* The get_*_platform() should probably use the platform module
>   which is a lot more flexible than distutils' get_platform()
>   (which should probably use the platform module as well in the
>   long run)

Please feel free to propose specific improvements to the 
distutils-sig.  But keep in mind that the platform information is 
specifically for supporting .egg filename platform tags.  Where the 
information comes from is less relevant than defining a framework for 
determining compatibility.

I first tried to get some kind of traction on this issue in 2004:

http://mail.python.org/pipermail/distutils-sig/2004-December/004355.html

And so far, the only platform for which something better has emerged 
is Mac OS X, due largely to Bob Ippolito stepping up and submitting some code.


>* The module needs some reorganization: imports, globals and constants
>   at the top, maybe a few comments delimiting the various sections,

I'm not sure I follow you.  Globals such as registries are usually 
defined close to the functions that provide the API for manipulating 
them.  And the rest of the globals (such as working_set), can't be 
defined until the class they're implemented with has been defined.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[M.-A. Lemburg]

On 2007-05-21 20:01, Phillip J. Eby wrote:
> At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>> However, since this is not egg-specific it should probably be
>> moved to pkgutil and get a separate PEP with detailed documentation
>> (the link you provided doesn't really explain the concepts, reading
>> the code helped a bit).
> 
> That doesn't really make sense in the context of the current PEP,
> though, which isn't to provide a general-purpose namespace package API;
> it's specifically about adding an existing piece of code to the stdlib,
> with its API intact.

You seem to indicate that you're not up to discussing the concepts
implemented by the module and *integrating* them with the Python
stdlib.

Please correct me if I'm wrong, but if the whole point of the PEP
is a take it or leave it decision, then I don't see the point of
discussing it. I'm -1 on adding the module in its current state;
I'd be +1 on integrating the concepts with the Python stdlib.

Hope I'm wrong,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 21 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Georg Brandl]

Nick Craig-Wood schrieb:
> Georg Brandl <[EMAIL PROTECTED]> wrote:
>>  over the last few weeks I've hacked on a new approach to Python's 
>> documentation.
>>  As Python already has an excellent documentation framework, the docutils, 
>> with a
>>  readable yet extendable markup format, reST, I thought that it should be
>>  possible to use those instead of the current LaTeX->latex2html
>>  toolchain.
> 
> Good idea!
> 
> Latex is a barrier for contribution to the docs.  I imagine most
> people would be much better at contributing to the docs in reST.  (Me
> included: I learnt latex at university a couple of decades ago and
> have now forgotten it completely!)
> 
>>  - a "quick-dispatch" function: e.g., docs.python.org/q?os.path.split would
>> redirect you to the matching location.
> 
> Being a seasoned unix user, I tend to reach for pydoc as my first stab
> at finding some documentation rather than (after excavating the mouse
> from under a pile of paper) use a web browser.
> 
> If you've ever used pydoc you'll know it reads docstrings and for some
> modules they are great and for others they are sorely lacking.
> 
> If pydoc could show all this documentation as well I'd be very happy!
> 
> Maybe your quick dispatch feature could be added to pydoc too?

It is my intention to work together with Ron Adam to make the pydoc <->
documentation integration as seamless as possible.

>>  Concluding, a small caveat: The conversion/build tools are, of course, not
>>  complete yet. There are a number of XXX comments in the text, most of them
>>  indicate that the converter could not handle a situation -- that would have
>>  to be corrected once after conversion is done.
> 
> It is missing conversion of ``comment'' at the moment as I'm sure you
> know...

Sorry, what did you mean?

> You will need to make your conversion perfect before you convince the
> people who wrote most of that documentation I suspect!

It already is as good as it gets, barring a few bugs here and there.
Which I'd like to hear about, when you find them!

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Georg Brandl]

Robert Kern schrieb:
> Neal Becker wrote:
> 
>> There is an effort as part of numpy to come up with a new system using
>> docstrings.  It seems to me it would be unfortunate if these two efforts
>> were not coordinated.
> 
> I don't think so. The issue with numpy is getting our act together and making
> parseable docstrings for auto-generated API documentation using existing tools
> or slightly modified versions thereof. No one is actually contemplating 
> building
> a new tool. AFAICT, Georg's (excellent) work doesn't address that use.

Indeed, I don't intend to do anything about docstrings. IMO, docs automatically
generated from docstrings can work, but only if there's a single consistent
style applied, and the whole thing is written in a markup language, of course,
not text only.

This is not the case for the Python standard library, so converting it is not
an option; in any case, putting all information that is available in the docs
into the docstrings would make many modules much less readable.


> I don't
> think there is anything to coordinate, here. Provided that Georg's system
> doesn't place too many restrictions on the reST it handles, we could use the
> available reST math options if we wanted to use Georg's system.

Of course, for numpy math is much more of importance than for the core.
I'm sure the docutils developers will be supportive in case someone volunteers
to create/improve reST math capabilities.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On May 21, 2007, at 12:28 AM, Martin v. Löwis wrote:

> I disagree. The documentation infrastructure of Python should only
> consider the needs of Python itself. If other people can use that
> infrastructure for other purposes, fine - if they find that it does
> not meet their needs, they have to look elsewhere.

I would take a fairly broad view of the term "for Python" though.   
Specifically, third party modules and applications written for and in  
Python should be explicitly supported.  I think we'd like to see one  
(preferred) documentation tool chain for core Python, plus the vast  
number of third party modules and apps.  I don't see any reason why  
Georg's reST-based system couldn't provide that (80/20 rule perhaps?).

I'd point to for example the howto templates, which can be easily  
used for third party applications.  Mailman uses howtos for example.

BTW, Georg excellent work.  I'm a big latex fan and long-time user,  
but I do think that using reST will open the door to a lot more  
contributions.

- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBRlHzJXEjvBPtnXfVAQLNXQP/QUuZ2gc/DpoidI9jYt7mr66Z+JYHsslv
fe4CvFSjd9OxwA3eOynd9dSOSkO6QHQPDVomW8axEkJJSHWNosnr9gmDZcC75nAD
JTt4rGImqwkcVIAzE91pZ3fmce/ltp9p1Ru3B1dDRmXbNgHxZ9njaz60MPFszC/H
19jSBo5sqSU=
=hlhi
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Georg Brandl]

[EMAIL PROTECTED] schrieb:
> Brett> Martin beat me to my comment.  =) Python's needs should come
> Brett> first, period.  If Georg wants to add math support, fine.  But
> Brett> honestly I would rather he spend his time on Python-specific
> Brett> stuff then get bogged down to support possible third parties.
> 
> I think the people who have responded to my comment read too much into it.
> Nowhere do I think I asked Georg to write an equation typesetter to include
> in the Python documentation toolchain.  I asked that math capability be
> considered.

And that is reasonable, of course.

> I have no idea what tools he used to build his new
> documentation set.  I only briefly glanced at a couple of the output pages.
> I think what he has done is marvelous.  However, I don't think the door
> should be shut on equation display.  Is there a route to it based on the
> tools Georg is using?

In the end, it all depends on what kind of support basic reST can deliver.
IMO, you still get the best math output from LaTeX, but I don't really know
many other things.

That is also something I want to convey: I'm very fond of LaTeX, and use it
regularly for all my University work. For the Python docs, however, I can
see many advantages of the docutils approach.

> If not, then I think some accommodation should be
> made.  I'm being vague here on purpose because I'm unfamiliar with the
> available tools.  The one thing I do know is that LaTeX provides that today
> and by removing it from the toolchain you have removed a significant piece
> of functionality.

That's the point I see differently: for the Python core docs, it's not
significant, and my efforts are primarily limited to that area.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Georg Brandl]

Fred L. Drake, Jr. schrieb:
> On Monday 21 May 2007, [EMAIL PROTECTED] wrote:
>  > Then I submit that you are probably removing some significant piece of
>  > functionality from the provided documentation toolchain which some people
>  > probably rely on.  After all, that's what LaTeX excels at.  They will be
>  > able to continue to use the old tools, but where will they get them if
>  > they are no longer part of Python?
> 
> I'll be happy to pull the existing tools out into a separate distribution if 
> we move to something else for Python.  There are too many users of the 
> existing tools to abandon.

That is a good idea! The converter is not likely to work with other projects
out of the box (it's been finetuned for the Doc/ sources), and it's not clear
whether they would want to switch in any case. Many of the features that the
new system would be able to provide aren't needed for them anyway, and I, as
a maintainer, would be very reluctant to put extra work in that too...


cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Jon Ribbens]

On Sat, May 19, 2007 at 07:14:09PM +0200, Georg Brandl wrote:
> For the impatient: the result can be seen at .

I think that looks great.

One comment I have, I don't know if it's relevant - it perhaps depends
on whether the "Global Module Index" is auto-generated or not. This is
the page I visit the most out of all the Python documentation, and
it's far too large and unwieldy. IMHO it would be much better if only
the top-level modules were shown here - having the single package
'distutils', for example, take up nearly 50 entries in the list is
almost certainly hindering a lot more people than it helps. It would
perhaps be better if such packages show up as one entry, which shows
the sub-modules when clicked on.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 08:56 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>On 2007-05-21 20:01, Phillip J. Eby wrote:
> > At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
> >> However, since this is not egg-specific it should probably be
> >> moved to pkgutil and get a separate PEP with detailed documentation
> >> (the link you provided doesn't really explain the concepts, reading
> >> the code helped a bit).
> >
> > That doesn't really make sense in the context of the current PEP,
> > though, which isn't to provide a general-purpose namespace package API;
> > it's specifically about adding an existing piece of code to the stdlib,
> > with its API intact.
>
>You seem to indicate that you're not up to discussing the concepts
>implemented by the module and *integrating* them with the Python
>stdlib.

No, I'm saying something else.  I'm saying it:

1. has nothing to do with the PEP,
2. isn't something I'm volunteering to do, and
3. would only make sense to do as part of Python 3 stdlib 
reorganization, if it were done at all.

Now, the code is certainly under an open license, and the concepts 
are entirely free for anyone to use.  If somebody wishes to do what 
you're describing, they're certainly welcome to take on that thankless task.

But I personally don't see the point, since by definition that new 
API would have *no current users*.  And the purpose of the PEP is to 
serve the (rather large) audience that would like to take advantage 
of existing software that uses the API.

Thus, any proposal to alter that API faces a high entry barrier to 
show how the proposed changes would provide a signficant practical 
benefit to users.

That's not even remotely similar to "take it or leave it".  It might 
*seem* that way, of course, simply because in any proposal to change 
the API, there's an implicit question of why nobody proposed the 
change via the Distutils-SIG, sometime during the last 2+ years of 
discussions around that API.

I remain open-minded and curious as to the possibility that someone 
*could* propose a meaningful change, but am also rationally skeptical 
that someone actually *will* come up with something that would 
outweigh the user benefit of keeping the already published, already 
discussed, already field-tested, already in-use API.

For that matter, I remain open-minded and curious as to the 
possibility of whether someone could propose a reasonable 
justification for *not* including the module in the stdlib.  After 
all, last year Fredrik Lundh surprised me with a convincing rationale 
for *not* including setuptools in the stdlib, which is why I backed 
off on doing so in 2.5, and am now proffering a much-reduced-in-scope 
proposal for 2.6.

So, I'm perfectly willing and able to change my mind, given 
convincing reasons to do so.  So far, though, your change suggestions 
haven't even explained why *you* want them, let alone why anybody 
else should agree.  We can hardly discuss what you haven't yet said.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Georg Brandl]

Jon Ribbens schrieb:
> On Sat, May 19, 2007 at 07:14:09PM +0200, Georg Brandl wrote:
>> For the impatient: the result can be seen at .
> 
> I think that looks great.
> 
> One comment I have, I don't know if it's relevant - it perhaps depends
> on whether the "Global Module Index" is auto-generated or not. This is
> the page I visit the most out of all the Python documentation, and
> it's far too large and unwieldy. IMHO it would be much better if only
> the top-level modules were shown here - having the single package
> 'distutils', for example, take up nearly 50 entries in the list is
> almost certainly hindering a lot more people than it helps. It would
> perhaps be better if such packages show up as one entry, which shows
> the sub-modules when clicked on.

Sure, that is certainly possible.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[A.M. Kuchling]

On Mon, May 21, 2007 at 09:23:47AM -0400, Fred L. Drake, Jr. wrote:
> I'll be happy to pull the existing tools out into a separate distribution if 
> we move to something else for Python.  There are too many users of the 
> existing tools to abandon.

That seems like a straightforward task.  

The big stumbling block in switching away from LaTeX has always been
the effort of making a good conversion; if Georg's work does 80% of
the job, we should definitely take advantage of the opportunity and 
try to switch.

Advantages of reST:

* The required tool chain shrinks (at least if you're not making printed
  output, which will probably still go through LaTeX).
* Tool chain is now more easily scriptable, and it'll be easier to 
  make use of the docs from Python code.
* We can produce XML output through the rst2xml script.

Disadvantages:

* reST markup isn't much simpler than LaTeX.

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Martin v. Löwis]

> I'm using the full-blown VS.NET 2003, as given to a number of python-dev
> people by Microsoft a number of years ago.  This appears to come with the
> SDK and a 64bit compiler.

Not sure what it makes it appear to you that way - it doesn't. VS.NET
2003 is x86 only

> I'm guessing vsextcomp doesn't use the Visual
> Studio 'ReleaseAMD64' configuration - would it be OK for me to check in
> changes to the PCBuild projects for this configuration?

Please don't. It exclusively relies on vsextcomp, and is only useful
if you have that infrastructure installed. See for yourself: it uses
the /USE_CL:MS_OPTERON command line switch, which isn't a Microsoft
invention (but instead invented by Peter Tröger).

> So is there something we can do to make distutils play better with binaries
> built from PCBuild8, even though it is considered temporary?

In what way better? It already supports them just fine, AFAICT.

I guess you are referring to the support for building extensions on
top of a source tree "installation". I doubt that is used that often
(but I understand you are using it).

> It seems the
> best thing might be to modify the PCBuild8 build process so the output
> binaries are in the ../PCBuild' directory - this way distutils and others
> continue to work fine.  Does that sound reasonable?

I think Kristjan will have to say a word here: I think he just likes
it the way it is right now. That would rather suggest that build_ext
needs to be changed.

> I've no objection to that - but I'd like to help keep the pain to a minimum
> for people who find themselves trying to build 64bit extensions in the
> meantime. 

I recommend that those people install the official binaries. Why do you
need to build the binaries from source, if all you want is to build
extensions?

>> In C or in C++? In C++, wchar_t is a builtin type, just like
>> short, int,
>> long. So there is no further formal definition.
> 
> This was in C++, but the problem was really WCHAR, as used by much of the
> win32 API.

But in C, WCHAR should not be a problem (and I would like to see
explicit source code and explicit compiler error message to be
proven wrong).

>>> * Finally, as something from left-field which may well take
>>> 12 months or
>>> more to pull off - but would there be any interest to
>>> moving the Windows
>>> build process to a cygwin environment based on the existing autoconf
>>> scripts?
>> What compiler would you use there? I very much like using the VS
>> debugger when developing on Windows, so that capability should not
>> go away.
> 
> You would use whatever compiler the autoconf toolset found.  Recent versions
> know enough about MSVC for simple projects.  Many people would need to take
> care that their environment pointed at the correct compiler - especially the
> person building releases.
> 
> But assuming MSVC was found and had the appropriate switches passed, there
> would be no impact on the ability to use Visual Studio as a debugging
> environment.

I doubt that could work in practice. You will have to rewrite everything
to make it pass the correct command line switches.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Martin v. Löwis]

>> I don't find the need to have separate object directories convincing:
>> For building the Win32/Win64 binaries, I have separate checkouts
>> *anyway*, since all the add-on libraries would have to support
>> multi-arch builds, but I think they don't.
> 
> No they don't, but that doesn't mean that you need different checkouts
> for python, only the others.  Anyway, this is indeed something I'd like
> to see addressed.  I don't think we should ditch cross-compilation.

Nobody proposed to ditch cross-compilation. I very much like
cross-compilation, I do all Itanium and AMD64 in cross-compilation.

I just want the *file structure* of the output to be the very same
for all architectures, meaning that they can't coexist in a single
source directory.

> It
> should simplify a lot of stuff, including buildbot setup and so on (if
> we get the buildbot infrastructure to support it).  It is also very
> cumbersome, if you are working on a project, to have the binaries all
> end up in the same place.  Doing interactive work on python, I frequently
> compile both the 32 bit and 64 bit versions for testing and it would
> be downright silly to have to rebuild everything every time.

No, you use two checkouts, of course.

> I think this is a bit unrealistic.  Here we are in the middle of 2007,
> VS2005 has just got SP1, and VS2003 is still the "official" compiler.
> PCBuild8 is ready, it just needs a little bit of extra love and
> buildbots to make us able to release PGO versions of x86 and x64.

No matter what the next compiler is (VS 2005 or VS 2007/2008), it's
still *a lot* of work until the VS 2005 build can be used for releasing
Python. For example, there is no support for the SxS installation of
msvcr8.dll, using the MSI merge module.

> Given the delay for getting even this far, waiting for Orcas and then
> someone to create PCBuild9, and then getting it up and running and so on
> will mean waiting another two years.

No. I would expect that either the PCbuild or PCbuild8 project files can
be used with little changes to build using VS9. I just tried, and it
works reasonably well.

> I am not familiar with the msi packaging process at all.  But here is
> something we should start to consider:  VISTA support.  This could mean
> some of:

Not sure whether anything really is needed. Python works fine on Vista.

> 1) supplying python.dll as a Side By Side assembly

What would that improve?

> 2) Changing python install locations

Why?

> 3) Supporting shadow libraries, where .pyc files end up in a different
>hierarchy from the .py files.  (useful for many things beside VISTA)

Yes, and people had written PEPs for it which failed due to lack of
follow up.

> 4) Signing the python dlls and executables

Hmm.

> 5) Providing user level manifests.
> 
> Vista adoption is going very fast.  We see 10% of our users have moved
> to vista and rising.

Sure, and have they reported problems with Python on Vista (problems
specific to Vista?)

> Yes.  Btw, in previous visual studio versions, wchar_t was not treated
> as a builtin type by default, but rather as synonymous with unsighed short.
> Now the default is that it is, and this causes some semantic differences
> and incompatibilities of the type seen.

C or C++? According to the standards, it ought to be a builtin,
primitive type in C++, and a typedef in C.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[M.-A. Lemburg]

On 2007-05-21 22:48, Phillip J. Eby wrote:
> At 08:56 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>> On 2007-05-21 20:01, Phillip J. Eby wrote:
>> > At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>> >> However, since this is not egg-specific it should probably be
>> >> moved to pkgutil and get a separate PEP with detailed documentation
>> >> (the link you provided doesn't really explain the concepts, reading
>> >> the code helped a bit).
>> >
>> > That doesn't really make sense in the context of the current PEP,
>> > though, which isn't to provide a general-purpose namespace package API;
>> > it's specifically about adding an existing piece of code to the stdlib,
>> > with its API intact.
>>
>> You seem to indicate that you're not up to discussing the concepts
>> implemented by the module and *integrating* them with the Python
>> stdlib.
> 
> No, I'm saying something else.  I'm saying it:
> 
> 1. has nothing to do with the PEP,
> 2. isn't something I'm volunteering to do, and
> 3. would only make sense to do as part of Python 3 stdlib
> reorganization, if it were done at all.

I don't understand that last part: how can adding a new module
or set of modules require waiting for reorganization of the
stdlib ?

All I'm suggesting is to reorganize the code in pkg_resources.py
a bit and move the relevant bits into pkgutil.py and into a new
eggutil.py.

> Now, the code is certainly under an open license, and the concepts are
> entirely free for anyone to use.  If somebody wishes to do what you're
> describing, they're certainly welcome to take on that thankless task.
> 
> But I personally don't see the point, since by definition that new API
> would have *no current users*.  And the purpose of the PEP is to serve
> the (rather large) audience that would like to take advantage of
> existing software that uses the API.
>
> Thus, any proposal to alter that API faces a high entry barrier to show
> how the proposed changes would provide a signficant practical benefit to
> users.

Why is that ?

You can easily provide a pkg_resource.py module with your old API
that interfaces to the new reorganized code in the stdlib.

> That's not even remotely similar to "take it or leave it".  It might
> *seem* that way, of course, simply because in any proposal to change the
> API, there's an implicit question of why nobody proposed the change via
> the Distutils-SIG, sometime during the last 2+ years of discussions
> around that API.

This doesn't have anything to do with distutils. It's entirely
about the egg distribution format.

> I remain open-minded and curious as to the possibility that someone
> *could* propose a meaningful change, but am also rationally skeptical
> that someone actually *will* come up with something that would outweigh
> the user benefit of keeping the already published, already discussed,
> already field-tested, already in-use API.
> 
> For that matter, I remain open-minded and curious as to the possibility
> of whether someone could propose a reasonable justification for *not*
> including the module in the stdlib.  After all, last year Fredrik Lundh
> surprised me with a convincing rationale for *not* including setuptools
> in the stdlib, which is why I backed off on doing so in 2.5, and am now
> proffering a much-reduced-in-scope proposal for 2.6.
>
> So, I'm perfectly willing and able to change my mind, given convincing
> reasons to do so.  So far, though, your change suggestions haven't even
> explained why *you* want them, let alone why anybody else should agree. 
> We can hardly discuss what you haven't yet said.

I'm not sure what you want to hear from me.

You asked for comments, I wrote back and gave you comments. I also
made it clear why I think that breaking up the addition into different
PEPs makes a lot of sense and why separating the code into different
modules for the same reason makes a lot of sense as well.

I also tried to stir up some discussion to make life easier
for setuptools by suggesting a user-package directory on
sys.path and adding support for namespace packages as
general Python feature instead of hiding it away in
pkg_resources.py.

You should see this as chance to introduce new concepts to Python.
Instead you seem to feel offended every time someone suggests a
change in your design. That's also the reason why I stopped
discussing things with you on the distutils list. There was simply
no way of getting through to you.

Perhaps we should just meet up for a beer in London sometime
and sort things out ;-)

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 21 2007)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::

Re: [Python-Dev] The docs, reloaded

[Martin v. Löwis]

> I think the people who have responded to my comment read too much into it.
> Nowhere do I think I asked Georg to write an equation typesetter to include
> in the Python documentation toolchain.  I asked that math capability be
> considered.  I have no idea what tools he used to build his new
> documentation set.  I only briefly glanced at a couple of the output pages.
> I think what he has done is marvelous.  However, I don't think the door
> should be shut on equation display.  Is there a route to it based on the
> tools Georg is using?

I don't think anything in the world can replace TeX for math
typesetting. So if math typesetting was a requirement (which it
should not be, for that very reason), then we could not consider
anything but TeX.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Martin v. Löwis]

> Martin> I disagree. The documentation infrastructure of Python should
> Martin> only consider the needs of Python itself. If other people can
> Martin> use that infrastructure for other purposes, fine - if they find
> Martin> that it does not meet their needs, they have to look elsewhere.
> 
> Then I submit that you are probably removing some significant piece of
> functionality from the provided documentation toolchain which some people
> probably rely on.  After all, that's what LaTeX excels at.

It may be significant for other people, but it is not significant for
the Python documentation.

> They will be
> able to continue to use the old tools, but where will they get them if they
> are no longer part of Python?

We are not going to remove old releases from the net.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Martin v. Löwis]

Bill Janssen schrieb:
>> We are developing a programming language here, not a typesetting
>> system.
> 
> Good point, Martin.  Are you implying that the documentation should be
> kept in LaTeX, a widely-accepted widely-disseminated stable
> documentation language, which someone else maintains, rather than
> ReST, which elements of the the Python community maintain?

No - I have no particular preference wrt. to the markup language.
I can personally live with all of them, and I like none of them.

I hear that contributors complain about having to use TeX, and
I hear other people say that they were more happy if they could
use ReST instead of TeX. Making contributors happy is really what
the objective should be (if the quality of the typesetting output
is adequate - and most people use the HTML output these days,
where latex2html may not have adequate quality).

That docutils happens to be written in Python should make little
difference - it's *not* part of the Python language project,
and is just a tool for us, very much like latex and latex2html.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[skip]

>> One comment I have, I don't know if it's relevant - it perhaps
>> depends on whether the "Global Module Index" is auto-generated or
>> not. This is the page I visit the most out of all the Python
>> documentation, and it's far too large and unwieldy. IMHO it would be
>> much better if only the top-level modules were shown here - having
>> the single package 'distutils', for example, take up nearly 50
>> entries in the list is almost certainly hindering a lot more people
>> than it helps. It would perhaps be better if such packages show up as
>> one entry, which shows the sub-modules when clicked on.

Georg> Sure, that is certainly possible.

Take a look at .  It records request
counts for the various pages and presents the most frequently requested
pages in a section at the top of the page.  I can make the script available
if anyone wants it (it uses Myghty - Mason in Python.)

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 11:44 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>On 2007-05-21 22:48, Phillip J. Eby wrote:
> > At 08:56 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
> >> On 2007-05-21 20:01, Phillip J. Eby wrote:
> >> > At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
> >> >> However, since this is not egg-specific it should probably be
> >> >> moved to pkgutil and get a separate PEP with detailed documentation
> >> >> (the link you provided doesn't really explain the concepts, reading
> >> >> the code helped a bit).
> >> >
> >> > That doesn't really make sense in the context of the current PEP,
> >> > though, which isn't to provide a general-purpose namespace package API;
> >> > it's specifically about adding an existing piece of code to the stdlib,
> >> > with its API intact.
> >>
> >> You seem to indicate that you're not up to discussing the concepts
> >> implemented by the module and *integrating* them with the Python
> >> stdlib.
> >
> > No, I'm saying something else.  I'm saying it:
> >
> > 1. has nothing to do with the PEP,
> > 2. isn't something I'm volunteering to do, and
> > 3. would only make sense to do as part of Python 3 stdlib
> > reorganization, if it were done at all.
>
>I don't understand that last part: how can adding a new module
>or set of modules require waiting for reorganization of the
>stdlib ?

I'm saying that an API reorganization that split up the pkg_resources 
API might be appropriate at that time, in much the same way that say, 
merging "dl" and "ctypes" (or dropping "dl") might take place during 
such a reorganization.  (And would thus go along with the 2to3 
conversion or whatever other mechanism will be used for porting 
Python 2 code to Python 3 code.)


>All I'm suggesting is to reorganize the code in pkg_resources.py
>a bit and move the relevant bits into pkgutil.py and into a new
>eggutil.py.
...
>You can easily provide a pkg_resource.py module with your old API
>that interfaces to the new reorganized code in the stdlib.

...and are you proposing that this other module *also* be included in 
the stdlib?  If so, that was not clear from your previous messages.


> > That's not even remotely similar to "take it or leave it".  It might
> > *seem* that way, of course, simply because in any proposal to change the
> > API, there's an implicit question of why nobody proposed the change via
> > the Distutils-SIG, sometime during the last 2+ years of discussions
> > around that API.
>
>This doesn't have anything to do with distutils. It's entirely
>about the egg distribution format.

Huh?  I'm saying that the pkg_resources API has been being discussed 
on the Distutils-SIG for a good 2 years now.  If there was something 
that users *wanted* to change in the API, surely it would've been 
discussed by now?


>I'm not sure what you want to hear from me.

A good place to start would be the rationale for your proposed API changes.


>You asked for comments, I wrote back and gave you comments.

Yep, and I explained my take on them.  You then brought up this whole 
"take it or leave it" thing, in response to which I attempted to 
provide further clarification of my reasoning -- none of which 
involves any "take it or leave it"-ness, from my perspective.


>  I also
>made it clear why I think that breaking up the addition into different
>PEPs makes a lot of sense and why separating the code into different
>modules for the same reason makes a lot of sense as well.

Actually, no, you didn't.  You simply asserted that certain things 
would be "better" (without any mention of *how* they would be 
better), and that other things "should probably" be done (without any 
explanation of *why*).

So, I simply responded with information of why I did *not* see these 
changes to be useful in any self-evident way, and why I'd want to see 
some justification that would weigh against the PEP's raison d'etre 
-- supporting the existing user base and making it easier for more 
people to join that user base.


>I also tried to stir up some discussion to make life easier
>for setuptools by suggesting a user-package directory on
>sys.path and adding support for namespace packages as
>general Python feature instead of hiding it away in
>pkg_resources.py.

The first is a great discussion topic, but unrelated to the 
PEP.  I'll be happy to participate in that discussion, if I have any 
input to offer.

The second, I don't see as "hiding away"; I would actually suggest 
that pkgutil.extend_path simply be deprecated in favor of the 
pkg_resources API for namespace packages.

I'm not aware of there being a particularly large user base for the 
pkgutil.extend_path, so I don't see a need to change an API that's 
used a lot, to match one that's not used as much.  AFAIK, extend_path 
was originally created for Zope Corp., and also AFAIK, they are now 
using pkg_resources instead.  See also this previous distutils-sig 
discussion (as I said, the API *does* get discussed there):

  http://mail.python.org/pipermail/distutils-sig/2006-Au

Re: [Python-Dev] The docs, reloaded

[Neal Becker]

[EMAIL PROTECTED] wrote:

> 
> >>> What would be my choices for add math to the documentation?
> 
> >> Where in the current documentation is there any math notation /at
> >> all/?
> 
> Georg> There is exactly one instance of LaTeX math in the whole docs,
> Georg> it's in the description of audioop, AFAIR, an contains a sum
> over Georg> square roots...
> 
> Georg> So, that's not really a concern of mine ;)
> 
> You must realize that people will use the core tools to create
> documentation
> for third party packages which aren't in the core.  If you replace LaTeX
> with something else I think you need to keep math in mind whether it's
> used in the core documentation or not.
> 
Perhaps my comment was misunderstood.  I have no objection to a new system,
and it does not have to be based on latex.  I just hope there will be some
escape mechanism that allows math.  It happens that for math markup, there
isn't really anything better (or more familiar) than latex.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Fred L. Drake, Jr.]

On Monday 21 May 2007, A.M. Kuchling wrote:
 > Disadvantages:
 >
 > * reST markup isn't much simpler than LaTeX.

* reST doesn't support nested markup, which is used in the current
  documentation.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Fred L. Drake, Jr.]

On Monday 21 May 2007, [EMAIL PROTECTED] wrote:
 > Take a look at .  It records request
 > counts for the various pages and presents the most frequently requested
 > pages in a section at the top of the page.  I can make the script
 > available if anyone wants it (it uses Myghty - Mason in Python.)

This is very cool.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Module cleanup improvement

[Alan McIntyre]

Hi all,

Bug #1717900 has an example of a script that causes a (cryptic, IMO)
error during module cleanup since instances of a class just happen to
get destroyed after their class is destroyed, and the __del__ method
manipulates a class attribute.  As I understand it this is expected
under the behavior outlined here:
http://www.python.org/doc/essays/cleanup/

Adding a step C1.5 which removes only objects that return true for
PyInstance_Check seems to prevent the problem exhibited by this bug (I
tried it out locally on the trunk and it doesn't cause any problems
with the regression test suite).  Is there any reason that adding such
a step to module cleanup would be a bad idea?

Alan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Aahz]

On Mon, May 21, 2007, Jon Ribbens wrote:
> 
> One comment I have, I don't know if it's relevant - it perhaps depends
> on whether the "Global Module Index" is auto-generated or not. This is
> the page I visit the most out of all the Python documentation, and
> it's far too large and unwieldy. IMHO it would be much better if only
> the top-level modules were shown here - having the single package
> 'distutils', for example, take up nearly 50 entries in the list is
> almost certainly hindering a lot more people than it helps. It would
> perhaps be better if such packages show up as one entry, which shows
> the sub-modules when clicked on.

That's a good point in general, but I think we want to manually label
some submodules as having entries in the global module index (notably
os.path).
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"Look, it's your affair if you want to play with five people, but don't
go calling it doubles."  --John Cleese anticipates Usenet
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[skip]

Neal> It happens that for math markup, there isn't really anything
Neal> better (or more familiar) than latex.

True enough.  There is MathML and its offspring, ASCIIMathML, which are
probably worth looking at.

http://www.w3.org/Math/
http://www1.chapman.edu/~jipsen/asciimath.html

I have no idea if either one has backend support for presentation outside
the web, but if people are interested in this (probably within the docutils
scope) they probably should be considered.  ASCIIMathML in particular is
probably worth using now within even if you can't convert it to any other
format.  It's about as readable as LaTeX source.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Steven Bethard]

On 5/21/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> >> One comment I have, I don't know if it's relevant - it perhaps
> >> depends on whether the "Global Module Index" is auto-generated or
> >> not. This is the page I visit the most out of all the Python
> >> documentation, and it's far too large and unwieldy. IMHO it would be
> >> much better if only the top-level modules were shown here - having
> >> the single package 'distutils', for example, take up nearly 50
> >> entries in the list is almost certainly hindering a lot more people
> >> than it helps. It would perhaps be better if such packages show up as
> >> one entry, which shows the sub-modules when clicked on.
>
> Georg> Sure, that is certainly possible.
>
> Take a look at .  It records request
> counts for the various pages and presents the most frequently requested
> pages in a section at the top of the page.  I can make the script available
> if anyone wants it (it uses Myghty - Mason in Python.)

+1 for integrating this with the official docs. I loved this the last
time you posted it too.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Bob Ippolito]

On 5/21/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> > I think the people who have responded to my comment read too much into it.
> > Nowhere do I think I asked Georg to write an equation typesetter to include
> > in the Python documentation toolchain.  I asked that math capability be
> > considered.  I have no idea what tools he used to build his new
> > documentation set.  I only briefly glanced at a couple of the output pages.
> > I think what he has done is marvelous.  However, I don't think the door
> > should be shut on equation display.  Is there a route to it based on the
> > tools Georg is using?
>
> I don't think anything in the world can replace TeX for math
> typesetting. So if math typesetting was a requirement (which it
> should not be, for that very reason), then we could not consider
> anything but TeX.

You can use docutils to generate LaTeX output from reST, and you can
put raw LaTeX into the output with ".. raw:: latex". I would imagine
this is sufficient for now.

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Mark Hammond]

Hi Marc-Andre,

> +1 from me.
>
> If think this is simply a bug introduced with the UCS4 patches in
> Python 2.2.
>
> unicodeobject.h already has this code:
>
> #ifndef PY_UNICODE_TYPE
>
> /* Windows has a usable wchar_t type (unless we're using UCS-4) */
> # if defined(MS_WIN32) && Py_UNICODE_SIZE == 2
> #  define HAVE_USABLE_WCHAR_T
> #  define PY_UNICODE_TYPE wchar_t
> # endif
>
> # if defined(Py_UNICODE_WIDE)
> #  define PY_UNICODE_TYPE Py_UCS4
> # endif
>
> #endif
>
> But for some reason, pyconfig.h defines:
>
> /* Define as the integral type used for Unicode representation. */
> #define PY_UNICODE_TYPE unsigned short
>
> /* Define as the size of the unicode type. */
> #define Py_UNICODE_SIZE SIZEOF_SHORT
>
> /* Define if you have a useable wchar_t type defined in
> wchar.h; useable
>means wchar_t must be 16-bit unsigned type. (see
>Include/unicodeobject.h). */
> #if Py_UNICODE_SIZE == 2
> #define HAVE_USABLE_WCHAR_T
> #endif
>
> disabling the default settings in the unicodeobject.h.

Yes, that does appear strange.  The following patch works for me, keeps
Python building and appears to solve my problem.  Any objections?

Mark


Index: pyconfig.h
===
--- pyconfig.h  (revision 55487)
+++ pyconfig.h  (working copy)
@@ -491,22 +491,13 @@
 /* Define if you want to have a Unicode type. */
 #define Py_USING_UNICODE

-/* Define as the integral type used for Unicode representation. */
-#define PY_UNICODE_TYPE unsigned short
-
 /* Define as the size of the unicode type. */
-#define Py_UNICODE_SIZE SIZEOF_SHORT
+/* This is enough for unicodeobject.h to do the "right thing" on Windows.
*/
+#define Py_UNICODE_SIZE 2

-/* Define if you have a useable wchar_t type defined in wchar.h; useable
-   means wchar_t must be 16-bit unsigned type. (see
-   Include/unicodeobject.h). */
-#if Py_UNICODE_SIZE == 2
-#define HAVE_USABLE_WCHAR_T
-
 /* Define to indicate that the Python Unicode representation can be passed
as-is to Win32 Wide API.  */
 #define Py_WIN_WIDE_FILENAMES
-#endif

 /* Use Python's own small-block memory-allocator. */
 #define WITH_PYMALLOC 1

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Mark Hammond]

Hi Martin,

> > I'm using the full-blown VS.NET 2003, as given to a number
> of python-dev
> > people by Microsoft a number of years ago.  This appears to
> come with the
> > SDK and a 64bit compiler.
>
> Not sure what it makes it appear to you that way - it doesn't. VS.NET
> 2003 is x86 only

Yes - I was confused by finding an x64 configuration option, and this
looking very similar to VC8.  My apologies for the confusion.

> > So is there something we can do to make distutils play
> better with binaries
> > built from PCBuild8, even though it is considered temporary?
>
> In what way better? It already supports them just fine, AFAICT.
>
> I guess you are referring to the support for building extensions on
> top of a source tree "installation". I doubt that is used that often
> (but I understand you are using it).

Yes, that is correct.  I agree it will be rarely used by Python user's, but
believe it is a common scenario for people who maintain extensions or
libraries, particularly those who want debugging builds.

> > It seems the
> > best thing might be to modify the PCBuild8 build process so the output
> > binaries are in the ../PCBuild' directory - this way distutils and
others
> > continue to work fine.  Does that sound reasonable?

> I think Kristjan will have to say a word here: I think he just likes
> it the way it is right now. That would rather suggest that build_ext
> needs to be changed.

So this means PCBuild8 does indeed get formalized into distutils?  I'm happy
to live with it if it lets me work, but it seems contrary to our emails
yesterday.  It would also mean the PCBuild8 environment will not work with
external build processes that assume the standard layout, but that really
isn't something I'm willing to run the pydev gauntlet for at the current
time.

> > I've no objection to that - but I'd like to help keep the
> pain to a minimum
> > for people who find themselves trying to build 64bit
> extensions in the
> > meantime.
>
> I recommend that those people install the official binaries.
> Why do you
> need to build the binaries from source, if all you want is to build
> extensions?

Let's assume that people have a valid reason to build from source - it
really is quite common in the open source world.  The meta-question then
becomes "is it reasonable to expect people be able to build extensions from
a tree built with VC8, in the same way they can with VS7?".  I think you are
suggesting we do want to support this, but I want to be clear.

>
> >> In C or in C++? In C++, wchar_t is a builtin type, just like
> >> short, int,
> >> long. So there is no further formal definition.
> >
> > This was in C++, but the problem was really WCHAR, as used
> by much of the
> > win32 API.
>
> But in C, WCHAR should not be a problem (and I would like to see
> explicit source code and explicit compiler error message to be
> proven wrong).

Please see my other mail to Kristjan - at this stage I can only reproduce it
in C++ on VC8.  C++ on VC7 and C on VC8 both work fine.  Please let me know
if thde code snippet I pasted or the compiler error aren't suitable.  I've
stopped further investigation since there seems support for a change that
makes the problem go away.

> >>> * Finally, as something from left-field which may well take
> >>> 12 months or
> >>> more to pull off - but would there be any interest to
> >>> moving the Windows
> >>> build process to a cygwin environment based on the
> existing autoconf
> >>> scripts?
> >> What compiler would you use there? I very much like using the VS
> >> debugger when developing on Windows, so that capability should not
> >> go away.
> >
> > You would use whatever compiler the autoconf toolset found.
>  Recent versions
> > know enough about MSVC for simple projects.  Many people
> would need to take
> > care that their environment pointed at the correct compiler
> - especially the
> > person building releases.
> >
> > But assuming MSVC was found and had the appropriate
> switches passed, there
> > would be no impact on the ability to use Visual Studio as a
> debugging
> > environment.
>
> I doubt that could work in practice. You will have to rewrite
> everything
> to make it pass the correct command line switches.

'have to rewrite everything' sounds a little pessimistic, but that's fine
with me - consider this idea dropped.

Cheers,

Mark

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Steve Holden]

Kristján Valur Jónsson wrote:
> First of all, I have put some work into pcbuild8 recently and it works
> well.  I am trying to drum up momentum for work on PCBuild8
> next europython.  See http://wiki.python.org/moin/EuroPython2007Sprints
> 
> 
>> -Original Message-
>> From: [EMAIL PROTECTED]
>>
>> I don't find the need to have separate object directories convincing:
>> For building the Win32/Win64 binaries, I have separate checkouts
>> *anyway*, since all the add-on libraries would have to support
>> multi-arch builds, but I think they don't.
> 
> No they don't, but that doesn't mean that you need different checkouts
> for python, only the others.  Anyway, this is indeed something I'd like
> to see addressed.  I don't think we should ditch cross-compilation.  It
> should simplify a lot of stuff, including buildbot setup and so on (if
> we get the buildbot infrastructure to support it).  It is also very
> cumbersome, if you are working on a project, to have the binaries all
> end up in the same place.  Doing interactive work on python, I frequently
> compile both the 32 bit and 64 bit versions for testing and it would
> be downright silly to have to rebuild everything every time.
> 
>> I would personally like to see Python "skip" VS 2005 altogether,
>> as it will be soon superceded by Orcas. Unfortunately, it's unclear
>> how long Microsoft will need to release Orcas (and also, when Python
>> 2.6 will be released), so I would like to defer that question by
>> a few months.
> I think this is a bit unrealistic.  Here we are in the middle of 2007,
> VS2005 has just got SP1, and VS2003 is still the "official" compiler.
> PCBuild8 is ready, it just needs a little bit of extra love and
> buildbots to make us able to release PGO versions of x86 and x64.
> 
> Given the delay for getting even this far, waiting for Orcas and then
> someone to create PCBuild9, and then getting it up and running and so on
> will mean waiting another two years.
> 
Addressing only the issues of PCBuild8 and 64-bit architectures, I have 
tried to establish "free" buildbot support for the 64-bit architectures 
without any real success.

Should the PSF be considering paying for infrastructure that will 
support 64-bit build reporting?

 > [...]

regards
  Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd   http://www.holdenweb.com
Skype: holdenweb  http://del.icio.us/steve.holden
-- Asciimercial -
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.comsquidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-- Thank You for Reading 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Martin v. Löwis]

> Yes, that is correct.  I agree it will be rarely used by Python user's, but
> believe it is a common scenario for people who maintain extensions or
> libraries, particularly those who want debugging builds.

Ah, debugging builds. It's true that PCbuild does not support them for
AMD64, and it's also true that you need such a build if you want the
debug CRT.

However, I think people ask much too often for a debugging build;
in many cases, they could work happily with a release build that
supports debugging. Depending on the problem you try to solve, you
may or may not need debug information for pythonxy.dll as well,
or just for your extension module.

I'd like to repeat an offer that I have made several times in the
past: if somebody contributes a patch to msi.py which packs all
PDB files (and whatever you else you need) into a ZIP file (or
whatever else works) from the release build, then I could happily
release PDB files along with the actual installer files. (I would
not like to release the PDB files *in* the installer files, as
I expect they would blow up the msi size significantly).

>> I think Kristjan will have to say a word here: I think he just likes
>> it the way it is right now. That would rather suggest that build_ext
>> needs to be changed.
> 
> So this means PCBuild8 does indeed get formalized into distutils?  I'm happy
> to live with it if it lets me work, but it seems contrary to our emails
> yesterday.

It would mean that - I'm willing to compromise to make Kristjan happy
(he contributed PCbuild8, so he has to decide what changes to it are
acceptable and which aren't).

A middle ground might be an addition build step (e.g. as a .bat file)
which copies all result files also into PCbuild.

> Let's assume that people have a valid reason to build from source - it
> really is quite common in the open source world.  The meta-question then
> becomes "is it reasonable to expect people be able to build extensions from
> a tree built with VC8, in the same way they can with VS7?".  I think you are
> suggesting we do want to support this, but I want to be clear.

As long as people are willing to maintain it - why not? It's not
"official" in the sense that if it breaks, nobody might fix it,
but it doesn't hurt to have it (AFAICT). Also, it not being official
might mean that we are not obliged to provide backwards compatibility
for it, so it can go away without notice (e.g. when/if PCbuild8
is dropped).

The same could be done for PC/VC6, if anybody cared to contribute
and maintain it.

I think there are several other cases in distutils to support
special cases (e.g. the DISTUTILS_USE_SDK environment variable);
if people want to see their setup supported, AND ARE WILLING TO
CONTRIBUTE: more power to them.

> Please see my other mail to Kristjan - at this stage I can only reproduce it
> in C++ on VC8.  C++ on VC7 and C on VC8 both work fine.  Please let me know
> if thde code snippet I pasted or the compiler error aren't suitable.

It's fine. I readily believe that it causes problems in C++.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Stephen J. Turnbull]

Neal Becker writes:

 > Perhaps my comment was misunderstood.  I have no objection to a new system,
 > and it does not have to be based on latex.  I just hope there will be some
 > escape mechanism that allows math.

Docutils already provides the "raw" directive.  I don't know if the
latex backend supports it, but adding support shouldn't be hard.

I don't think even thinking about that is Georg's responsibility, of
course.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Martin v. Löwis]

> Addressing only the issues of PCBuild8 and 64-bit architectures, I
> have tried to establish "free" buildbot support for the 64-bit
> architectures without any real success.
> 
> Should the PSF be considering paying for infrastructure that will 
> support 64-bit build reporting?

You can bring it up to the board, of course. However, given that
all other buildbot machines were contributed by volunteers, the
fact that nobody volunteers to contribute buildbot machines for
PCbuild8 indicates that there is not a lot of interest in that
build infrastructure.

Note that there *are* 64-bit buildbot slaves, e.g. for AMD64
(contributed by Neal Norwitz), Alpha (contributed by Neal
Norwitz), Itanium (contributed by Matthias Klose, offering
a machine from the Debian project), and PPC64 (likewise).

These machines all run Linux or Tru64, and (to my understanding)
serve other purposes as well, making the buildbot slave
just a minor detail.

It's an unfortunate fact of life that Microsoft Windows does not
support multi-tasking multi-user workloads that well, so the
Windows buildbot slave are (to my knowledge) typically dedicated
machines (except for Tim's machine, which is just disconnected
from the master when he doesn't feel like running buildbot
jobs).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Titus Brown]

On Tue, May 22, 2007 at 07:32:42AM +0200, "Martin v. L?wis" wrote:
-> > Addressing only the issues of PCBuild8 and 64-bit architectures, I
-> > have tried to establish "free" buildbot support for the 64-bit
-> > architectures without any real success.
-> > 
-> > Should the PSF be considering paying for infrastructure that will 
-> > support 64-bit build reporting?
-> 
-> You can bring it up to the board, of course. However, given that
-> all other buildbot machines were contributed by volunteers, the
-> fact that nobody volunteers to contribute buildbot machines for
-> PCbuild8 indicates that there is not a lot of interest in that
-> build infrastructure.
-> 
-> Note that there *are* 64-bit buildbot slaves, e.g. for AMD64
-> (contributed by Neal Norwitz), Alpha (contributed by Neal
-> Norwitz), Itanium (contributed by Matthias Klose, offering
-> a machine from the Debian project), and PPC64 (likewise).
-> 
-> These machines all run Linux or Tru64, and (to my understanding)
-> serve other purposes as well, making the buildbot slave
-> just a minor detail.
-> 
-> It's an unfortunate fact of life that Microsoft Windows does not
-> support multi-tasking multi-user workloads that well, so the
-> Windows buildbot slave are (to my knowledge) typically dedicated
-> machines (except for Tim's machine, which is just disconnected
-> from the master when he doesn't feel like running buildbot
-> jobs).

I haven't really been listening to this conversation, so forgive me if
this isn't relevant, but:

I'd be happy to install Windows and the latest VisualStudio on a 64-bit
VMware image.  I just can't be responsible for day-to-day administration
of the buildslave; Windows requires too much attention for me to do
that.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Mark Hammond]

> However, I think people ask much too often for a debugging build;
> in many cases, they could work happily with a release build that
> supports debugging. Depending on the problem you try to solve, you
> may or may not need debug information for pythonxy.dll as well,
> or just for your extension module.

While that is true in theory, I often find it is not the case in practice,
generally due to the optimizer.  Depending on what magic the compiler has
done, it can be very difficult to set breakpoints (conditional or
otherwise), inspect variable values, etc.  It is useful in some cases, but
very often I find myself falling back to a debug build after attempting to
debug a local release build.

> I think there are several other cases in distutils to support
> special cases (e.g. the DISTUTILS_USE_SDK environment variable);
> if people want to see their setup supported, AND ARE WILLING TO
> CONTRIBUTE: more power to them.

Yes, but I believe its also important to solicit feedback on the best way to
achieve their aims.  In this particular case, I believe it would have been
misguided for me to simply check in whatever was necessary to have distutils
work from the PCBuild8 directory.  I hope it is clear that I am willing to
contribute the outcome of these discussions...

Cheers,

Mark

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com