Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
> I've been making changes to py.exe in hg.p.o, so I hope the standalone one > is tracking. The msi for it as part of the official build can also > standalone, so maybe we should merge the two? > Hmm, sadly I don't think it is. Originally the standalone one was (I > believe) provided by Vinay for people using Pythons that didn't have > it bundled, and to add features (such as the new file extensions) on a > quicker timescale than Python releases. But since he passed it to the > pypa umbrella I don't think he's been keeping the two in sync. I've > copied him in case I'm completely wrong on this. Thanks for copying me in - I haven't been tracking changes to the launcher in hg.p.o purely because I have been really busy with consultancy work for the past few months, leading to reduced time available for open-source contributions. I intend to try and keep things synced up where feasible and when time permits, and your email has given me the heads-up to look and see what's changed in the hg.p.o version. > Personally if we now have a standalone launcher MSI, I'd like to > discontinue the external one completely, and officially publish the > standalone launcher MSI on python.org as a service for users of older > Pythons. The standalone repo/installer is just there for older Pythons where it's not bundled (and also to allow trying out features independently of the Python release cycle, as you mentioned). If python.org will offer an unbundled MSI just to install the launcher on Pythons < 3.3, then I suppose my installer could be retired. I don't know if the hg.p.o. installer would be equivalent to the standalone one (e.g. when the standalone launcher is uninstalled, the user gets a chance to associate file extensions with one of the installed Pythons found on the system). Off to read the whole thread on python-dev now ... Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
- Original Message - From: Steve Dower > I believe are the purpose of those extensions. If "py.exe spam.pyz" > already works, I don't see any need for a PEP to add the association in > the installer. That would be my view, too. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
> Actually, I've just remembered, it's already in PEP 441, > which hasn't been approved but which goes further and > includes a stdlib tool to create pyz files. I'm not sure > if that changes things at all... Actually, I developed the pyzzer tool (which Paul may remember - he made some helpful suggestions to improve it) which does a similar job to PEP 441's pyzaa and provides improved Windows support (some of which might be supplanted by py.exe support for .pyz/.pyzw, but I have cases where I can't rely on py.exe being present). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
- Original Message -
From: Paul Moore
> I'd also appreciate your views on the spin-off thread and PEP 486
> ("Make the Python Launcher aware of virtual environments").
On an initial reading it seems quite reasonable. I'll sleep on it and see if
any other things spring to mind.
By the way, although I turned over the standalone launcher to pypa, I forked
that repo, and do development on my fork [1]. I remembered that I added a
feature (with the help of Pawel Jasinski) to allow e.g. py -ipy to pick a
command 'ipy' configured in the .ini file (this is used to launch IronPython,
but it could be used for other things too). In addition, my version has some
bugfixes (e.g. use of iswspace instead of isspace, which is wrong). Steve's
changes seem to just be related to adding IP_VERSION - I'm not sure what that's
about and I haven't looked into it yet; I just skimmed the differences, and
didn't see anything major in launcher.c (I compared with the version in the
default branch of cpython).
Regards,
Vinay Sajip
[1] https://bitbucket.org/vinay.sajip/pylauncher
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
- Original Message -
From: Paul Moore
> I'd also appreciate your views on the spin-off thread and PEP 486
> ("Make the Python Launcher aware of virtual environments").
I just thought of something ... as far as I know, we've enabled searching for a
command on the path - since when a venv is active in the sense of VIRTUAL_ENV
being defined, PATH is also set up so that the venv's Scripts folder is on it -
so we would just need to find "python.exe" on the PATH and invoke that - no
need to even use the value of VIRTUAL_ENV, making implementation simpler. What
do you think?
Regards,
Vinay
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
Paul Moore gmail.com> writes: > 2. It would be a nice, although extremely obscure, feature to be able > to allow people who want to keep Python off their path to set > VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing > but there's *still* no python.exe on PATH. Limit the dependency on a > single environment variable rather than two, in other words. If this is the case, then your patch may not work if PATH isn't set. If a shebang is found in a script, it is processed in maybe_handle_shebang(), which means that your code to look in a venv won't be hit (though if you rely on the PATH being correctly set, then it will work as expected because the handling for e.g. /usr/bin/env python searches the PATH). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows
- Original Message - From: Paul Moore > The original PEP said nothing about a PATH search for #!/usr/bin/env > python, so I mistakenly assumed there was no way currently to write a > script that worked with an activated virtualenv. Actually the path search was added later, by you! See [1] and [2]. Regards, Vinay Sajip [1] http://bugs.python.org/issue17903 [2] https://bitbucket.org/pmoore/pylauncher/commits/87d41c5a8f0fef5ef22f09570903e7aa4c850e96 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore.
Sorry, my mistake - I should have checked more carefully. Thank you for pointing me to the commit where they were removed - I'll install a venv with Sphinx, revert the change and remove those files from my repo. Regards, Vinay Sajip From: Benjamin Peterson To: python-dev@python.org; python-check...@python.org; vinay_sa...@yahoo.co.uk Sent: Friday, 14 March 2014, 15:10 Subject: Re: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. On Fri, Mar 14, 2014, at 07:20 AM, vinay.sajip wrote: > http://hg.python.org/cpython/rev/edb12dad7bf6 > changeset: 89654:edb12dad7bf6 > user: Vinay Sajip > date: Fri Mar 14 14:20:09 2014 + > summary: > Added Doc/tools/ subdirs to .hgignore. > > files: > .hgignore | 5 + > 1 files changed, 5 insertions(+), 0 deletions(-) > > > diff --git a/.hgignore b/.hgignore > --- a/.hgignore > +++ b/.hgignore > @@ -24,6 +24,11 @@ > reflog.txt$ > tags$ > Lib/plat-mac/errors.rsrc.df.rsrc > +Doc/tools/sphinx/ > +Doc/tools/docutils/ > +Doc/tools/jinja/ > +Doc/tools/jinja2/ > +Doc/tools/pygments/ Why are you readding these when they were apparently purposely removed in eef7899ea7ab?___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0
M.-A. Lemburg egenix.com> writes: > The suggestion to have the metadata available on PyPI doesn't > have anything to do with security. > > It's about being able to determine compatibility and select the > right distribution file for download. The metadata also helps in > creating dependency graphs, which are useful for a lot of things. FYI, distlib already uses metadata in this way. Note that: 1. Dependency information is locked away in setup.py files, e.g in kwargs to the setup() call. 2. PyPI does not provide access to this metadata. So I have had to write code to extract what information I could out of setup.py setup() and into metadata that can be used. I have, for now, hosted it on my own server, but it can be used for e.g. determining complete dependency graphs without ever actually downloading any distributions. Distlib provides access to it: $ python finddeps.py pyramid 12 dists, elapsed time: 4.35 secs Download order: download setuptools (0.6c11) [for pyramid, zope.interface, zope.deprecation] download PasteDeploy (1.5.0) [for pyramid] download repoze.lru (0.6) [for pyramid] download venusian (1.0a7) [for pyramid] download MarkupSafe (0.15) [for Mako] download translationstring (1.1) [for pyramid] download Chameleon (2.11) [for pyramid] download WebOb (1.2.3) [for pyramid] download zope.interface (4.0.3) [for pyramid] download Mako (0.7.3) [for pyramid] download zope.deprecation (4.0.2) [for pyramid] download pyramid (1.4) In the above, nothing was actually downloaded except the metadata. It's just indicating an intuitive ordering for download based on a topological sort of the dependency graph - for pyramid, in this case. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0
Chris Jerdonek gmail.com> writes: > Maybe this is already stated somewhere, but is there a plan for when > distlib will be brought into the repository? Is there a reason not to > do it now? It seems it would have more visibility that way (e.g. > people could see it as part of the development version of the online > docs and it would be in check-ins as are PEP edits), and its status > relative to Python would be clearer. There's no firm plan other than it being on the 3.4 wishlist, but IMO there's no rush to do it right now. AFAICT interested parties can be in the loop even though the project is actively developed on BitBucket rather than hg.python.org: remember that the current stakeholders in terms of packaging tools are not all in python-dev and may find it easier to contribute on BitBucket. (I often get comments from Daniel Holth in response to distlib commits I push to BitBucket.) I periodically sync the BitBucket repo with hg.python.org/distlib. The docs are also readily accessible on readthedocs.org (the first two Google results for distlib are those docs and the project on BitBucket). While not integrated with docs.python.org, I don't think that the distlib docs suffer inordinately from being on readthedocs.org - for one thing, I've enabled Disqus comments, allowing feedback on the docs - not that I've had any ;-) Closer integration should perhaps happen, but not until the project has matured a little. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0
Paul Moore gmail.com> writes: > Understood - that's why I suggested that distlib reach a point where > it's stable as an external package and supported on (some) older > versions. I'm hoping for an experience more like unittest2 than > packaging/distutils2. Currently, distlib runs on Python 2.6, 2.7 and 3.2+, so there shouldn't be any concerns about support on those older Python versions. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: PEP 426: replace implied 'version starts with' with new ~= operator
Nick Coghlan gmail.com> writes: > Daniel is a fan of this syntax, but I think it is inferior to the > implied approach, so don't expect it to survive to any accepted > version of the PEP :) Another thing against ~= is that it isn't valid Python syntax. It's not a deal- breaker, but it does mean that you can't e.g. use the ast module in the implementation. This might be a factor if the mini-language ever grows (as it recently did, adding parentheses). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on Windows
Paul Moore gmail.com> writes: > I don't understand what this is saying - can someone clarify the > reason behind this statement? What is different about a > "non-system-wide installation" that causes this issue (I assume > "non-system-wide" means "not All Users")? The reason I ask is that > virtualenv doesn't do this, and I'm not clear if this is because of a > potential bug lurking in virtualenv (in which case, I'd like to find > out how to reproduce it) or because virtualenv takes a different > approach which avoids this issue somehow. One example of a non-system-wide installation is a source build of Python. PEP 405 venvs created from a source build should work in the same way as venvs created using an installed Python. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (2.7): Issue #17508: Handled out-of-order handler configuration correctly.
Antoine Pitrou pitrou.net> writes: >> Issue #17508: Handled out-of-order handler configuration correctly. > Could you explain what "out-of-order handler configuration" means? In logging, a MemoryHandler buffers records and has a reference to another handler - the "target" - which does the actual output of the buffered records. It can happen when using dictConfig() that the MemoryHandler is configured before the target handler, and in this case the reference to the target wasn't set up correctly. The commit rectifies this. > Also, could you add a Misc/NEWS entry for the change / bugfix? Ok, will do. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Safely importing zip files with C extensions
> This quote is here to stop GMane complaining that I'm top-posting. Ignore.
I've already posted this to distutils-sig, but thought that it might be of
interest to readers here as it relates to importing C extensions ...
zipimport is great, but there can be issues importing software that contains
C extensions. But the new wheel format (PEP 427) may give us a better way of
importing zip files containing C extensions. Since wheels are .zip files, they
can sometimes be used to provide functionality without needing to be installed.
But whereas .zip files contain no convention for indicating compatibility with
a particular Python, wheels do contain this compatibility information. Thus, it
is possible to check if a wheel can be directly imported from, and the wheel
support in distlib allows you to take advantage of this using the mount() and
unmount() methods. When you mount a wheel, its absolute path name is added to
sys.path, allowing the Python code in it to be imported. (A DistlibException is
raised if the wheel isn't compatible with the Python which calls the mount()
method.)
You don't need mount() just to add the wheel's name to sys.path, or to import
pure-Python wheels, of course. But the mount() method goes further than just
enabling Python imports - any C extensions in the wheel are also made available
for import. For this to be possible, the wheel has to be built with additional
metadata about extensions - a JSON file called EXTENSIONS which serialises an
extension mapping dictionary. This maps extension module names to the names in
the wheel of the shared libraries which implement those modules.
Running unmount() on the wheel removes its absolute pathname from sys.path and
makes its C extensions, if any, also unavailable for import.
Wheels built with the new "distil" tool contain the EXTENSIONS metadata, so can
be mounted complete with C extensions:
$ distil download -d /tmp simplejson
Downloading simplejson-3.1.2.tar.gz to /tmp/simplejson-3.1.2
63KB @ 73 KB/s 100 % Done: 00:00:00
Unpacking ... done.
$ distil package --fo=wh -d /tmp /tmp/simplejson-3.1.2/
The following packages were built:
/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl
$ python
Python 2.7.2+ (default, Jul 20 2012, 22:15:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from distlib.wheel import Wheel
>>> w = Wheel('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
>>> w.mount()
>>> import simplejson._speedups
>>> dir(simplejson._speedups)
['__doc__', '__file__', '__loader__', '__name__', '__package__',
'encode_basestring_ascii', 'make_encoder', 'make_scanner', 'scanstring']
>>> simplejson._speedups.__file__
'/home/vinay/.distlib/dylib-cache/simplejson/_speedups.so'
>>>
Does anyone see any problems with this approach to importing C extensions from
zip files?
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Safely importing zip files with C extensions
Amaury Forgeot d'Arc gmail.com> writes:
> Better: just put the wheel path to sys.path
sys.path.append('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
> and let a sys.path_hook entry do the job.
That's what the mount() actually does - adds the wheel to a registry that an
import hook uses. You also need a place to check that the wheel being mounted
is compatible with the Python doing the mounting - I'm not sure whether what
the import hook should do if e.g. there is a compatibility problem with the
wheel (e.g. is it clear that it should always raise an ImportError? Or ignore
the wheel - seems wrong? Or do something else?)
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Safely importing zip files with C extensions
Stefan Behnel behnel.de> writes: > I've always hated this setuptools misfeature of copying C extensions from > an installed archive into a user directory, one for each user. At least > during normal installation, they should be properly unpacked into normal > shared library files in the file system. The user directory location is not a key part of the functionality, it could just as well be a shared location across all users. And this is an option for specific scenarios, not a general substitute for installing the wheel (which unpacks everything into FHS-style locations). A lot of people use virtual envs, which are per-user anyway. I'm not suggesting this is a good idea for system-wide deployments of software. > Whether it then makes sense to special case one-shot trial imports like the > above without installation is a bit of a different question, but I don't > see a compelling reason for adding complexity here. It's not really an > important use case. Well, my post was to elicit some comment about the usefulness of the feature, so fair enough. It doesn't seem especially complex though, unless I've missed something. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Safely importing zip files with C extensions
Daniel Holth gmail.com> writes: > zipping from ever happening. This is an important reason why wheel is > billed as an installation format -- fewer users with pitchforks. It's > very cool that it works though. Debugging is slightly easier than it > was in the old days because pdb can now read the source code from the > zip. Well, it's just an experiment, and I was soliciting comments because I'm not as familiar with the issues as some others are. Distlib is still only at version 0.1.1, and the mount()/unmount() functionality is not set in stone :-) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong
Paul Moore gmail.com> writes: > This will mean that scripts written with #!/usr/bin/env python will > behave the same on Unix and Windows in the presence of activated > virtualenvs. Overall I think it's the right result. There's one other compatibility clarification: at the moment, as allowed by the PEP, you can have launcher flags in a line that starts with #!/usr/bin/env python, for example #!/usr/bin/env python3.2-32 -u In such a case the launcher would use the 3.2-32 suffix to indicate that 32-bit Python 3.2 is wanted, and pass the -u to the launched executable. I assume that this behaviour should continue, and the Posix-compatible behaviour being proposed should only apply for lines that contain "#!/usr/bin/env python" followed by whitespace. Also, since we're making a backwards incompatible change, do people feel that it needs to be switched on only in the presence of e.g. an environment variable such as PYLAUNCH_SEARCHPATH, or should we just change the default behaviour now and risk breaking user scripts which may rely on the current behaviour? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong
Paul Moore gmail.com> writes:
> PS Vinay - from this post, I assume you're already looking at this
> code? I was considering trying to put together a patch, but I don't want to
> duplicate effort if you're working on it.
I've taken a quick look at it, but I probably won't be able to make any
changes until the near the end of the coming week. Feel free to have a go;
the place to make changes will be near the call
is_virt = parse_shebang(...)
...
if (!is_virt) {
...
}
else {
/* In here is where the new logic will probably go. */
}
Also, the #define SEARCH_PATH needs to be uncommented to include the
find_on_path function. It also enables searching the path for customised
commands.
Regards,
Vinay
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong
> From: Paul Moore >Did you get a chance to have a look at this? I didn't manage to create a pull >request against your copy of pylauncher as my repo > is a fork of the pypa one - I'm not sure if that's a limitation of bitbucket > or if I just don't know how to do it... I've created a pull request > against the pypa version in case that's of use... Hi Paul, Sorry I haven't had a chance yet - real life is very busy at the moment. A pull request against the pypa version is fine, and I will get to it soon - thanks for your patience. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Problem with pydoc script in PEP 405 venvs
Issue #18224 (http://bugs.python.org/issue18224) highlights a problem on Windows with the pydoc script provided with venvs created by pyvenv. On POSIX, the script is named pydoc and causes no problems: on Windows, it is called pydoc.py and this causes problems because it shadows the stdlib pydoc module. Possible solutions: 1. Remove the pydoc script altogether from created venvs, on Windows but also on POSIX (for consistency). 2. Rename the pydoc script on both Windows and POSIX (e.g. to pydocs.py and pydocs respectively). 3. Rename the pydoc.py script to pydoc-script.py and introduce a simple .exe launcher pydoc.exe adjacent to it (which is how setuptools and distlib handle installed scripts). The first two approaches are backwards-incompatible, while the third is less likely to lead to breakage, but involves adding a Windows script launcher to Python. While this is a bigger change, I think any built-in Python installer functionality should include such a launcher (as setuptools and distlib do). Still, that's probably a discussion for another day. Does anyone have any comments? Approach #2 seems the most appropriate. I assume it would be reasonable to implement this in both 3.3 and 3.4, as it's not a change in core Python APIs. In the absence of adverse feedback here, I propose to implement approach #2 on both 3.3 and 3.4. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problem with pydoc script in PEP 405 venvs
Richard Oudkerk gmail.com> writes: > > Can't a batch file pydoc.bat be used? > I generally find .bat files so limiting that I never thought of that. In this case, it makes perfect sense to use one. Thanks! @echo off __VENV_PYTHON__ -c "import sys, pydoc; sys.exit(pydoc.cli())" should do it. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problem with pydoc script in PEP 405 venvs
Oleg Broytman phdru.name> writes: > > On Mon, Jun 17, 2013 at 06:37:49PM +0000, Vinay Sajip yahoo.co.uk> wrote: > > echo off > > __VENV_PYTHON__ -c "import sys, pydoc; sys.exit(pydoc.cli())" > >I think you want to pass command line arguments: > > echo off > __VENV_PYTHON__ -c "import sys, pydoc; sys.exit(pydoc.cli())" %1 %2 %3 %4 %5 %6 %7 %8 %9 > That was off-the-cuff, what I actually implemented (in my sandbox repo) was @echo off __VENV_PYTHON__ -c "import sys, pydoc; sys.argv[0] = 'pydoc'; sys.exit(pydoc.cli())" %* Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python logging and 1.5.2 compatibility
I'm planning to "officially" drop support for Python 1.5.2 in the logging package. When the logging package was introduced in Python 2.3, many Linux distros were shipping 1.5.2 as the system's Python, even though 2.2 had been out for a while. So it seemed important to support 1.5.2 for those sysadmins who wanted to use logging with their system Python. The Linux landscape has changed a bit since then. Most Linux distros ship with much more recent versions of Python, and so I no longer see 1.5.2 support as important. Dropping support for 1.5.2 means that future changes to logging will not be concerned with 1.5.2 compatibility. For example, boolean values which were 0/1 in the logging package will at some point be replaced by False/True, and newer language features will start to be used when changes are made. There are no plans for a specific "cleanup" exercise at the moment. In fact some changes made a while ago inadvertently broke 1.5.2 compatibility, but no-one's complained. So I'm assuming the whole thing is really a non-issue, and this post is just to keep everyone in the picture. A 1.5.2-compatible version of the package is still available via http://www.red-dove.com/python_logging.html if anyone needs it. This version is not actively maintained, but that shouldn't be an issue. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python logging and 1.5.2 compatibility
Antoine Pitrou pitrou.net> writes: > So, what is the minimum supported version now? Well, each version is tested with the release of Python which it's part of, and in theory it would be possible for the code in logging for that release to use features only available in that release of Python. So, I'm not sure if it now makes sense to support a minimum version. When changing the package I intend to be conservative in terms of using the simplest, rather than using the latest language features (say, coroutines) just because they're available. The reason it was done for 1.5.2 was that there was already a community of users of the "standalone" package (which was 1.5.2-compatible) before it got accepted into Python 2.3, and it didn't make sense at the time to maintain two different codebases - one 1.5.2-compatible and one not. Keeping it compatible meant that 1.5.2-dependent users/sysadmins could just copy the whole package and have it work, and take advantage of bug-fixes and enhancements as they appeared. PEP 291 (Backward Compatibility for Standard Library) mentions logging as having 1.5.2 compatibility, and I'm thinking that it can be removed from that PEP altogether. I don't think it's still necessary to hold back from using new features just to support 1.5.2, given that a working implementation is available if any 1.5.2 user should need it, and that the 1.5.2-dependent community will be pretty minimal. A lot of water has passed under the bridge. There are some new features which have been added in recent versions (say, WatchedFileHandler in 2.6, NullHandler in 2.7) and these should still work if copied and pasted into a codebase which runs with an older version of Python. Certain projects which need to support multiple Python versions - Django, say - can take advantage of this by e.g. offering a copy of these handlers in their own codebase for use when constrained to work with an older Python version. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 389: argparse - new command line parsing module
Yuvgoog Greenle gmail.com> writes: > +1 for adding argparse and eventually deprecating optparse, -0 for deprecating getopt. > 2. Big modules - 1 uber-module for each subject that does everything. Maybe logging is an example of this. I'm not sure that it is, if you mean code size. In Python 2.5, logging is 1300 SLOC, less than say tarfile, pickletools, pydoc and decimal. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 389: argparse - new command line parsing module
Brett Cannon python.org> writes: > Obviously if one of the getopt supporters has a better way of doing > this then please speak up. I'm not a getopt supporter per se - I'm +1 for argparse - but Opster provides some nice syntax sugar on top of getopt, and can be seen here: http://blogg.ingspree.net/blog/2009/09/14/opster/ I've contacted Steven about a similar approach for argparse, and we're waiting for Yuvgoog's work on argparse(func) to be done to see if there's a feasible similar approach. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 389: argparse - new command line parsing module
Brett Cannon python.org> writes: > I am reluctant to jump on these argument parsing decorators until they > have existed outside the standard library for a while and have settled > down as I suspect some people want to use annotations to do type > conversion/checking, others don't, etc. > I agree - it's just an approach which shows some promise and is worth exploring. I'm not suggesting jumping in with both feet. But it does show about getopt that perhaps there's life in the old dog yet ;-) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Steven Bethard gmail.com> writes:
> There's a lot of code already out there (in the standard library and
> other places) that uses %-style formatting, when in Python 3.0 we
> should be encouraging {}-style formatting. We should really provide
> some sort of transition plan. Consider an example from the logging
> docs:
>
> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
>
> We'd like to support both this style as well as the following style:
>
> logging.Formatter("{asctime} - {name} - {levelname} - {message}")
>
In logging at least, there are two different places where the formatting issue
crops up.
The first is creating the "message" part of the the logging event, which is
made up of a format string and arguments.
The second is the one Steven's mentioned: formatting the message along with
other event data such as time of occurrence, level, logger name etc. into the
final text which is output.
Support for both % and {} forms in logging would need to be considered in
these two places. I sort of liked Martin's proposal about using different
keyword arguments, but apart from the ugliness of "dicttemplate" and the fact
that "fmt" is already used in Formatter.__init__ as a keyword argument, it's
possible that two different keyword arguments "fmt" and "format" both referring
to format strings might be confusing to some users.
Benjamin's suggestion of providing a flag to Formatter seems slightly better,
as it doesn't change what existing positional or keyword parameters do, and
just adds an additional, optional parameter which can start off with a default
of False and transition to a default of True.
However, AFAICT these approaches only cover the second area where formatting
options are chosen - not the creation of the message from the parameters passed
to the logging call itself.
Of course one can pass arbitrary objects as messages which contain their own
formatting logic. This has been possible since the very first release but I'm
not sure that it's widely used, as it's usually easier to pass strings. So
instead of passing a string and arguments such as
logger.debug("The %s is %d", "answer", 42)
one can currently pass, for a fictitious class PercentMessage,
logger.debug(PercentMessage("The %s is %d", "answer", 42))
and when the time comes to obtain the formatted message, LogRecord.getMessage
calls str() on the PercentMessage instance, whose __str__ will use %-formatting
to get the actual message.
Of course, one can also do for example
logger.debug(BraceMessage("The {} is {}", "answer", 42))
where the __str__() method on the BraceMessage will do {} formatting.
Of course, I'm not suggesting we actually use the names PercentMessage and
BraceMessage, I've just used them there for clarity.
Also, although Raymond has pointed out that it seems likely that no one ever
needs *both* types of format string, what about the case where application A
depends on libraries B and C, and they don't all share the same preferences
regarding which format style to use? ISTM no-one's brought this up yet, but it
seems to me like a real issue. It would certainly appear to preclude any
approach that configured a logging-wide or logger-wide flag to determine how to
interpret the format string.
Another potential issue is where logging events are pickled and sent over
sockets to be finally formatted and output on different machines. What if a
sending machine has a recent version of Python, which supports {} formatting,
but a receiving machine doesn't? It seems that at the very least, it would
require a change to SocketHandler and DatagramHandler to format the "message"
part into the LogRecord before pickling and sending. While making this change
is simple, it represents a potential backwards-incompatible problem for users
who have defined their own handlers for doing something similar.
Apart from thinking through the above issues, the actual formatting only
happens in two locations - LogRecord.getMessage and Formatter.format - so
making the code do either %- or {} formatting would be simple, as long as it
knows which of % and {} to pick.
Does it seems too onerous to expect people to pass an additional "use_format"
keyword argument with every logging call to indicate how to interpret the
message format string? Or does the PercentMessage/BraceMessage type approach
have any mileage? What do y'all think?
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Antoine Pitrou pitrou.net> writes:
> Why not allow logging.Formatter to take a callable, which would in turn call
> the callable with keyword arguments?
>
> Therefore, you could write:
>logging.Formatter("{asctime} - {name} - {level} - {msg}".format)
>
> and then:
>logging.critical(name="Python", msg="Buildbots are down")
This seems perhaps usable for a Formatter instantiation (infrequent) but a
problem for the case where you want to convert format_str + args -> message
(potentially frequent, and less readable). Another problem is that logging
calls already use keyword arguments (extra, exc_info) and so backward
compatibility might be compromised. It also feels like passing a callable could
encourage patterns of usage which restrict our flexibility for future changes:
we want for now to just allow choosing between % and {}, but a callable can do
anything. That's more flexible, to be sure, but more specialized formatting
requirements are already catered for using e.g. the PercentMessage/BraceMessage
approach.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Barry Warsaw python.org> writes:
>
> This is a very interesting idea.
>
> Note that one of the reasons to /at least/ support {}-strings also is
> that %-strings are simply too error prone in many situations. For
> example, if I decide to support internationalization of log format
> strings, and all I can use is %-strings, it's almost guaranteed that I
> will have bugs because a translator forgot the trailing 's'. This
> exactly the motivation that led to PEP 292 $-strings.
>
> In fact, while we're at it, it would be kind of cool if I could use $-
> strings in log templates. Antoine's idea of accepting a callable
> might fit that bill nicely.
You're already covered if you use the PercentMessage/BraceMessage approach I
mentioned elsewhere in this thread. Suppose:
#Just typing this in, it's not tested or anything
class DollarMessage:
def __init__(self, fmt, *args, **kwargs):
self.fmt = fmt
self.args = args
self.kwargs = kwargs
def __str__(self):
return string.Template(self.fmt).substitute(*args, **kwargs)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Vinay Sajip yahoo.co.uk> writes: > #Just typing this in, it's not tested or anything > class DollarMessage: > def __init__(self, fmt, *args, **kwargs): > self.fmt = fmt > self.args = args > self.kwargs = kwargs > > def __str__(self): > return string.Template(self.fmt).substitute(*args, **kwargs) > Whoops, sorry, pressed the "post" button by accident on my previous post. The above substitute call should of course say string.Template(self.fmt).substitute(*self.args, **self.kwargs) and you can alias DollarMessage (or whatever name you choose) as _ or __, say. As far as the Formatter formatting goes, it's easy enough to subclass Formatter to format using whatever approach you want. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Brett Cannon python.org> writes:
> As for the PercentMessage/BraceMessage, I would make sure that you
> just simply take the string format and simply apply the arguments
> later to cut down on the amount of parentheses butting up against each
> other: ``logger.debug(BraceMessage("The {} is {}"), "answer", 42)``.
The problem with that is that BraceMessage.__str__() wouldn't know what
arguments to use to produce the message.
> cost of wrapping all strings for a while. I suspect most people will
> simply import the wrapping class and give it some short name like
> people do with gettext.
>
Yes,
logger.debug(__("The {} is {}", "answer", 42))
isn't ideal but perhaps liveable with. And hopefully with a decent editor, the
paren-butting annoyance will be minimized.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Antoine Pitrou pitrou.net> writes:
>
> Why is it a problem? I don't understand. It certainly is less pleasant to
> write "{foo}".format or "{0} {1}".format than it is to write "{0} {1}" alone,
> but it's still prettier and easier to remember than the special wrappers
> people are proposing here.
Well, it's less readable, as I said in parentheses. It would work, of course.
And the special wrappers needn't be too intrusive:
__ = BraceMessage
logger.debug(__("Message with {0} {1}", 1, "argument"))
> Then logging can just keep recognizing those special keyword arguments, and
> forward the others to the formatting function.
It just means that you can't pass those values through, and what if some of
them are used somewhere in existing code?
> > It also feels like passing a callable could
> > encourage patterns of usage which restrict our flexibility for future
> > changes:
>
> Which future changes are you thinking about? AFAIK, there hasn't been a
It's the Rumsfeldian "We don't know what we don't know" ;-)
> single change in logging output formatting in years. Rejecting a present
> change on the basis that it "restricts our flexibility for future changes"
> sounds like the worst kind of argument to me
Now don't get upset and take it as a rejection, as we're still in the
kicking-ideas-around stage ;-) I'm just saying how it feels to me.
I agree that logging output formatting hasn't changed in years, and that's
because there's been no particular need for it to change (some changes *were*
made in the very early days to support a single dict argument). Now that time
for change has perhaps come.
I'm just trying to think ahead, and can't claim to have got a definitive answer
up my sleeve. Passing a callable has upsides and downsides, and ISTM it's
always worth focusing on the downsides to make sure they don't come back and
bite you later. I don't foresee any specific problem - I'm just uneasy about
it.
> Except that having to wrap format strings with "PercentMessage" or
> "BraceMessage" is horrible. Python is not Java.
Amen. I'd say "Yeccchh!" too, if it literally had to be like that. And I also
note that there are voices here saying that support for %-formatting shouldn't,
or doesn't need to, change, at least until Python 4.0.
So consider the following tentative suggestion, which is off the top of my head
and offered as a discussion point:
Suppose that if you want to use %-formatting, everything stays as is. No
backward-compatibility headaches.
To support {}-formatting, add an extra class which I've called BraceMessage.
Consider this name a "working title", as no doubt a better name will suggest
itself, but for now this name makes it clear what we're talking about.
If any module wants to use {} formatting for their logging, they can add the
line
from logging import BraceMessage as __
I've used two underscores, since _ might be being used for gettext, but
obviously the importer can use whatever name they want.
and then they can use
logger.debug(__("The {0} is {1}", "answer", 42))
which I think is more readable than putting in ".format" following the string
literal. It's not a *huge* point, perhaps, but "Readability counts".
This has the side benefit that if e.g. Barry wanted to use string.Template for
formatting, he's just got to replace the above import with something like
from logging import DollarMessage as __
Another "working title", please note. And while I've shown these classes being
imported from logging, it doesn't make sense to put them there if this idea
were to fly in a more general context. Then, perhaps string would be a better
home for these classes.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Paul Moore gmail.com> writes:
> This seems to me to be almost the same as the previous suggestion of
> having a string subclass:
>
> class BraceFormatter(str):
> def __mod__(self, other):
> # Needs more magic here to cope with dict argument
> return self.format(*other)
>
> __ = BraceFormatter
>
> logger.debug(__("The {0} is {1}"), "answer", 42)
>
> The only real differences are
>
> 1. The positioning of the closing parenthesis
> 2. The internal implementation of logger.debug needs to preserve
> string subclasses properly
>
The other difference is that my suggestion supports Barry's desire to use
string.Template with no muss, no fuss ;-) Plus, very little additional work is
required compared to your items 1 and 2. ISTM BraceMessage would be something
like this,
clsss BraceMessage:
def __init__(self, fmt, *args, **kwargs):
self.fmt = fmt
self.args = args
self.kwargs = kwargs
def __str__(self):
return self.fmt.format(*self.args, **self.kwargs)
Regards,
Vinay
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Paul Moore gmail.com> writes:
> 2. The internal implementation of logger.debug needs to preserve
> string subclasses properly
>
> But the benefit is that the approach allows anyone to use brace
> formatting in any API that currently accepts % format (assuming string
> subclasses don't get mangled).
>
> On the one hand, I'd prefer a more general solution. On the other, I'm
> nervous about that "assuming string subclasses..." proviso.
>
You're right to be nervous: it's not hard to mangle subtypes.
>>> class mystr(str): pass
...
>>> s = mystr("Abc")
>>> s
'Abc'
>>> type(s)
>>> s2 = s.replace("A", "a")
>>> s2
'abc'
>>> type(s2)
>>>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Antoine Pitrou pitrou.net> writes:
> There's another possibility, which is to build the wrapping directly around
> the
> logger. That is, if I want a %-style logger, I do:
>
>logger = logging.getLogger("smtp")
>logger.debug("incoming email from %s", sender_address)
>
> and I want a {}-style logger, I do:
>
>logger = logging.getLogger("smtp", style="{}")
>logger.debug("incoming email from {addr}", addr=sender_address)
>
> (of course, different users of the "smtp" logger can request different
> formatting styles when calling getLogger().)
There's a LoggerAdapter class already in the system which is used to wrap
loggers so that additional contextual information (e.g. network or database
connection information) can be added to logs. The LoggerAdapter could fulfill
this "wrapping" function.
> We could combine the various proposals to give users flexible APIs. Of course,
> it generally smells of "there's more than one way to do it".
Yeah, that bothers me too.
> > It's the Rumsfeldian "We don't know what we don't know"
>
> Is this guy in the Python community?
Not sure, but I believe he's a piece of work and not a guy to get on the wrong
side of ;-)
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Raymond Hettinger rcn.com> writes:
> It looks like the BraceMessage would have to re-instantiate on every
> invocation.
True, because the arguments to the instantiation are kept around as a
BraceMessage instance until the time comes to actually format the message
(which might be never). Since typically in performance-sensitive code, the
isEnabledFor pattern is used to avoid doing unnecessary work, as in
if logger.isEnabledFor(logging.DEBUG):
logger.debug(__("The {0} is {1}", "answer", 42))
The BraceMessage invocation overhead is only incurred when needed, as is the
cost of computing the additional arguments.
As I understand it {}-formatting is slower than %-formatting anyway, and if
this pattern is used only for {}-formatting, then there will be no additional
overhead for %-formatting and some additional overhead for {}-formatting. I'm
not sure what that instantiation cost will be relative to the overall time for
an "average" call - whatever that is ;-) - though.
Other approaches to avoid instantiation could be considered: for example,
making __ a callable which remembers previous calls and caches instances keyed
by the call arguments. But this will incur memory overhead and some processing
overhead and I'm not sure if it really buys you enough to warrant doing it.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
> So I created this last night:
>
> import collections
>
> class braces_fmt(str):
>
> def __mod__(self, stuff):
> if isinstance(stuff, tuple):
> return self.__class__(self.format(*stuff))
> elif isinstance(stuff, collections.Mapping):
> return self.__class__(self.format(**stuff))
> else:
> return self.__class__(self.format(stuff))
>
> The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
> instead of throwing an error that str.format() would do with the code
> above. But what's nice about this is I think I can use this now w/ any
> library that expects % interpolation and it should basically work.
So there's no need to change modules like logging to explicitly provide support
for {}-formatting? What's not to like? ;-) Something like this perhaps should
have been added in at the same time as str.format went in.
> I don't think Paul's suggestion requires much more work to support
> string.Template, simply a subclass that implements __mod__
True.
> I guess my question is what's the point of the class if you are simply
> converting it before you pass it in to the logger? To be lazy about
> the formatting call? Otherwise you could simply call str.format() with
> your arguments before you pass the string into the logger and not have
> to wrap anything.
That's exactly the reason - to defer the formatting until it's needed.
Otherwise you can always format the string yourself,as you say, and pass it as
the single argument in the logging call - logging won't know or care if it was
passed in as a literal, or was computed by %-, {}-, $- or any other formatting
approach.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
James Y Knight fuhm.net> writes: > Using containment instead of inheritance makes sure none of the > *other* operations people do on strings will appear to work, at least > (substring, contains, etc). I bet explicitly calling str() on a format > string is even more rare than attempting to do those things. Actually, logging calls str() on the object passed as the first argument in a logging call such as logger.debug(), which can either be a format string or an arbitrary object whose __str__() returns the format string. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Nick Coghlan gmail.com> writes:
> I believe classes like fmt_braces/fmt_dollar/fmt_percent will be part of
> a solution, but they aren't a complete solution on their own. (Naming
> the three major string formatting techniques by the key symbols involved
> is a really good idea though)
>
> The two major problems with them:
>
> 1. It's easy to inadvertently convert them back to normal strings. If a
> formatting API even calls "str" on the format string then we end up with
> a problem (and switching to containment instead of inheritance doesn't
> really help, since all objects implement __str__).
>
> 2. They don't help with APIs that expect a percent-formatted string and
> do more with it than just pass it to str.__mod__ (e.g. inspecting it for
> particular values such as '%(asctime)s')
Good point as far as the general case is concerned, though it's perhaps not that
critical for logging. By which I mean, it's not unreasonable for
Formatter.__init__ to grow a "style" keyword parameter which determines whether
it uses %-, {}- or $-formatting. Then the formatter can look for '%(asctime)s',
'{asctime}' or '$asctime' according to the style.
Just to clarify - LogRecord.getMessage *will* call str() on a message object if
it's not a string or Unicode object. For 2.x the logic is
if type(msg) not in (unicode, str):
msg = str(msg)
and for 3.x the check is for isinstance(msg, str).
> Still, it's worth considering adding the three fmt_* classes to the
> string module to see how far they can get us in adapting the formats for
> different APIs.
>
> Note that I don't think these concepts are fully baked yet, so we
> shouldn't do anything in a hurry - and anything that does happen should
> be via a PEP so we can flush out more issues.
Yes, we're just "kicking the tires" on the various ideas. There are things still
a bit up in the air such as what happens when pickling and sending to an older
version of Python, etc. which still need to be resolved for logging, at least.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Paul Moore gmail.com> writes:
> Still tangential, but it seems to me that this discussion has exposed
> a couple of areas where the logging interface is less than ideal:
>
> - The introspection of the format string to delay computing certain
> items (Eric's suggestion may be an improvement here).
Yes, but that's an implementation detail and not part of the logging interface.
It can be changed without any particular additional impact on user code - when
I say "additional" I mean apart from the need to change the format strings to
{} format, which they would have to do anyway at some point.
> - The "call str() on any non-string object to get a format string" API
> (which precludes string subclasses).
It doesn't preclude string subclasses: it just calls str() on an arbitrary
message object to get the string representation for that object. The return
value is used to interpolate into the formatted output, and that's all. So
I don't understand what's being precluded and how - please elaborate.
Thanks & regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Brett Cannon python.org> writes:
> Why don't we start something in the sandbox and see how far we can
> get. If no one beats me to it I will add the directory some time today
> and we can start hashing out the solution there.
>
I've done a first cut of a converter from %-format to {}-format strings. I'm not
sure where you want to put it in the sandbox, I've created a gist on GitHub:
http://gist.github.com/200936
Not thoroughly tested, but runs in interactive mode so you can try things out.
All feedback appreciated!
Regards,
Vinay
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Nick Coghlan gmail.com> writes:
> I'm starting to think that a converter between the two format
> mini-languages may be the way to go though.
>
> fmt_braces is meant to provide a superset of the capabilites of
> fmt_percent, so a forward converter shouldn't be too hard. A reverse
> converter may have to punt with ValueError when it finds things that
> cannot be expressed in the fmt_percent mini language though.
I've done a first cut of a forward (% -> {}) converter:
http://gist.github.com/200936
but I'm not sure there's a case for a converter in the reverse direction, if
we're encouraging movement in one particular direction.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Raymond Hettinger rcn.com> writes:
> We should get one written. ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.
>
I've made a start, but I'm not sure how best to handle the '#' and ' '
conversion flags.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] summary of transitioning from % to { } formatting
Brett Cannon python.org> writes:
> No one is saying we should deprecate % any time soon on strings
> themselves or anywhere. This discussion is purely in regards to
> argparse and logging to transition *their* APIs over to {} formatting
> which would most likely involve some deprecation for *using* %
> formatting in those APIs. But % formatting on strings themselves is
> not directly being discussed here.
While I have no problem with supporting {}-formatting in logging if we find a
good way of doing it, one thing that bothers me about transitioning the logging
API to deprecate or otherwise de-emphasise %-formatting is the question of
performance. Now, str.format is more flexible than str.__mod__ and so some
performance loss may be tolerable in many scenarios. However, in the feedback I
regularly get about logging, people are concerned about performance. No-one ever
comes up with any hard numbers, but logging is often bashed as slow (see e.g.
Andrii Mishkovskyi's LoggingPackage page on the Python wiki). I don't especially
want to add fuel to the fire, as any performance degradation caused by
supporting {}-formatting will likely just result in more finger-pointing at
logging in general.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
MRAB mrabarnett.plus.com> writes:
> Interesting that you're using the %-format to translate %-formats to
> {}-formats!
Yes, ironic, isn't it? ;-)
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Raymond Hettinger rcn.com> writes:
> We should get one written. ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.
I'm not sure you can always get equivalent output from the formatting, though.
For example:
>>> "%0#8x" % 0x1234
'0x001234'
>>> "{0:0>#8x}".format(0x1234)
'000x1234'
>>>
Someone please tell me if there's a better {}-format string which I've missed!
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
MRAB mrabarnett.plus.com> writes:
> >>> "{0:#08x}".format(0x1234)
> '0x001234'
Good call, but here's another case:
>>> "%#o" % 0x1234
'011064'
I don't see how to automatically convert the "%#o" spec, though of course there
are ways of fudging it. The obvious conversion doesn't give the same value:
>>> "{0:#o}".format(0x1234)
'0o11064'
I couldn't see a backward-compatibility mode for str.format generating just a
leading "0" for octal alternative format (the C style) as opposed to "0o".
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] summary of transitioning from % to { } formatting
Nick Coghlan gmail.com> writes:
> Oh, I see what you meant now - you were pointing out that lazy
> formatting APIs (such as logging) already don't work properly for
> alternative formatting mechanisms (such as string.Template).
>
Logging doesn't work automatically with string.Template as it pre-dates
string.Template, but it can be made to work without too much trouble:
(a) Subclass Formatter to use $-formatting instead of %-formatting for e.g. the
lines written to log files, and
(b) Use a class like DollarMessage which I mentioned elsewhere in this thread,
to format logging call format strings and arguments.
Both of these approaches will also work for {}-formatting. The present thread
really started out with a view to suggesting that the stdlib start adopting
{}-format as "native", rather than %-format.
Would it be helpful if I added a section to the Python docs about how to use
alternative formatting systems with logging?
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] comments vs spam in PyPI [was: eggs now mandatory for pypi?]
P.J. Eby telecommunity.com> writes: > For example, having a packages reddit (nb: open source, written in > Python), where people can upvote or downvote packages and leave > comments. That's probably the minimum amount of checks and balances > required to avoid problems of the sort the PyPI commenting feature is > already having, since others will be able to reply to the comments, > and downvote nonsense into oblivion. Seems like a reasonable idea, given that there's already a Python reddit which seems popular. There was also www.cheeserater.com (which appears down at the moment) which was a Django demo site (source available at http://github.com/jacobian/cheeserater) for rating PyPI packages. I just created a "cheeseshop" sub-reddit to play around with. > (Alternatively, shutting off the comment system would also > work. Nothing stops people from using Google to search for "foo > sucks" or "problems using foo" if they want to research what's been > said about a package.) Yes, but it's good to have all the feedback in one place, if possible. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] A new way to configure logging
At present, configuration of Python's logging package can be done in one of two ways: 1. Create a ConfigParser-readable configuration file and use logging.config.fileConfig() to read and implement the configuration therein. 2. Use the logging API to programmatically configure logging using getLogger(), addHandler() etc. The first of these works for simple cases but does not cover all of the logging API (e.g. Filters). The second of these provides maximal control, but besides requiring users to write the configuration code, it fixes the configuration in Python code and does not facilitate changing it easily at runtime. In addition, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the stdlib at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python of course support an additional battery-included format which can be used for configuration - namely, JSON. Other options, such as YAML, are also possible ways of configuring systems, Google App Engine-style, and PyYAML has matured nicely. There has also been talk on the django-dev mailing list about providing better support for using Python logging in Django. When it happens (as of course I hope it does) this has the consequence that many new users who use Django but are relatively inexperienced in Python (e.g. in PHP shops which are moving to Django) will become exposed to Python logging. As Django is configured using a Python module and use of ConfigParser-style files is not a common approach in that ecosystem, users will find either of the two approaches outlined above a particular pain point when configuring logging for their Django applications and websites, unless something is done to avoid it. All three of the contenders for the title of "commonly found configuration mechanism" - JSON, YAML and Python code - will be expressible, in Python, as Python dicts. So it seems to make sense to add, to logging.config, a new callable bound to "dictConfig" which will take a single dictionary argument and configure logging from that dictionary. An important facet of implementing such a scheme will be the format or schema which the dictionary has to adhere to. I have started working on what such a schema would look like, and if people here think it's a good idea to go ahead with this, I'll provide the details of the schema on a separate post which I'll also cross-post on comp.lang.python so that I can get feedback from there, too. In outline, the scheme I have in mind will look like this, in terms of the new public API: class DictConfigurator: def __init__(self, config): #config is a dict-like object (duck-typed) import copy self.config = copy.deepcopy(config) def configure(self): # actually do the configuration here using self.config dictConfigClass = DictConfigurator def dictConfig(config): dictConfigClass(config).configure() This allows easy replacement of DictConfigurator with a suitable subclass where needed. What's the general feeling here about this proposal? All comments and suggestions will be gratefully received. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
Olemis Lang gmail.com> writes: > This kind of problems is similar to the one mentioned in another > thread about modifying config options after executing commands. In > that case I mentioned that the same dict-like interface also holds for > WinReg and so on ... > > So thinking big (yes ! I have a big head ! ) I think that the best > approach in this case is to build an adaptation (simple) layer on top > of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific > extensions for these formats . Perhaps the proper interfaces are > already there (e.g. `dict`, `shelve` ) and I'm just blind and looking > somewhere else ;o) Sorry, you've lost me :-) > > import copy > > self.config = copy.deepcopy(config) > > Why ? So I'm free to mutate self.config as I see fit. > extension is cool ... what's the point about adding the new method > instead of using `DictConfigurator` directly ? When you say "the new method", if you mean "configure" - the pattern is so that a subclass can override __init__ and do additional setup before configure() is called. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
Paul Rudin writes:
> How about the global logging configuration being available as an object
> supporting the usual dictionary interface? So you could, for example, do
> something like: "logging.dictconfig.update(partialconfig)"
A "partial configuration" only makes sense under certain limited conditions. For
example, a dict used for configuration will map to a graph of objects - filters,
formatters, handlers and loggers - which by necessity need to be referred to via
a string key in the dict - because you can't easily encode a graph in a dict
which comes from, say, a YAML or JSON file. These keys are temporary and just
used to indicate e.g. which handlers to attach to which loggers for that
particular configuration call:
{
"handlers": {
"h1": {
configuration for a handler
},
"h2": {
configuration for another handler
},
"h3": {
configuration for yet another handler
},
},
"loggers": {
"foo": {
"level": "DEBUG",
"handlers": ["h1", "h3"],
},
"bar.baz": {
"level": "INFO",
"handlers": ["h2", "h3"],
},
"spam.eggs": {
"level": "ERROR",
"handlers": ["h1", "h3"],
},
}
}
Here, the keys "h1", "h2" etc. are just used to encode connections in the graph,
and are useless outside the context of the specific configuration call. If you
pass a partial configuration, to be implemented incrementally, there's no way of
knowing if (in general) it's going to be consistent with the existing
configuration. Nor can you refer to e.g. handlers, formatters or filters you
passed in earlier configuration calls.
Since logger names such as "foo", "bar.baz" in the above example *are*
meaningful across multiple configuration calls, it *would be* possible to make
certain changes in the configuration - such as changing levels of loggers - by
passing in a partial configuration. However, if this "partial" functionality is
provided without restriction, it's hard to ensure that a user doesn't
inadvertently misconfigure logging. And while that is equally true with a
one-shot configuration, it's at least easier to debug. Things can get pretty
murky from a diagnostic point of view if code can make arbitrary changes to the
logging configuration during application execution, so that's not a good
practice to encourage :-)
Also, providing a readable dict-like interface to the internal configuration
will almost certainly lead to backward-compatibility headaches in the future.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Antoine Pitrou pitrou.net> writes:
>
> Vinay Sajip yahoo.co.uk> writes:
> >
> > >>> "%0#8x" % 0x1234
> > '0x001234'
> > >>> "{0:0>#8x}".format(0x1234)
> > '000x1234'
>
> Apart from the sheer unreadability of the {}-style format string, the result
> looks rather unexpected from a human being's point of view.
>
> (in those situations, I would output the 0x manually anyway, such as:
> >>> "0x%06x" % 0x1234
> '0x001234'
> >>> "0x{:06x}".format(0x1234)
> '0x001234'
> )
>
Well of course, but I asked the question in the context of providing an
*automatic* converter from %-format strings to {}-format. At the moment, it
doesn't seem like a 100%-faithful automatic conversion is feasible.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a new setuptools release?
P.J. Eby telecommunity.com> writes: > * Distribute 0.6.x is a stable maintenance branch, much like setuptools 0.6 I'm new to this particular discussion so feel free to correct me if I'm wrong, but ISTM that Distribute 0.6.x differs markedly from setuptools 0.6 in that the former has an active maintainer and the latter does not, or am I wrong about that? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
Paul Moore gmail.com> writes: > One option I would have found useful in some code I wrote would be to > extend the configuration - > > class DictConfigurator: >... >def extend(self, moreconfig): > import copy > more = copy.deepcopy(moreconfig) # Not sure if this is needed? > self.config.update(more) See my response to Paul Rudin's post about incremental configuration. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
Glenn Linderman g.nevcal.com> writes: > But DictConfigurator the name seems misleading... like you are > configuring how dicts work, rather than how logs work. Maybe with more > context this is not a problem, but if it is a standalone class, it is > confusing what it does, by name alone. Of course the fully qualified name would be logging.config.DictConfigurator which does provide the requisite context, but if I think of/someone suggests a better name I'll certainly use it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Logging, Unicode and sockets
> Thanks to http://bugs.python.org/issue7077 I've noticed that the socket-based logging handlers - SocketHandler, DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the presence of Unicode messages. I'd like to fix this by giving these handlers an optional (encoding=None) parameter in their __init__, and then using this to encode on output. If no encoding is specified, is it best to use locale.getpreferredencoding(), sys.getdefaultencoding(), sys.getfilesystemencoding(), 'utf-8' or something else? On my system: >>> sys.getdefaultencoding() 'ascii' >>> sys.getfilesystemencoding() 'mbcs' >>> locale.getpreferredencoding() 'cp1252' which suggests to me that the locale.getpreferredencoding() should be the default. However, as I'm not a Unicode maven, any suggestions would be welcome. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Nick Coghlan gmail.com> writes:
> Vinay's problem above is due to using the wrong alignment flag: ">",
> which says to right align everything, instead of "=", which says to left
> align the sign character and the numeric prefix with the fill character
> inserted in the middle. In this particular case he could also use the
> zero-padding shortcut which leaves out the alignment flag altogether
> (and implies a "0=" alignment format).
>
[snip]
> Note that, since percent formatting doesn't allow specification of the
> fill characters or the field alignment, translations should probably
> rely on the simple field width specifier, optionally selecting zero
> padding by preceding it with a zero. It should never be necessary to use
> the full alignment spec for translated formats.
>
> The other thing to keep in mind is that brace formatting is fussier
> about the order of things - items *must* appear in the order they are
> listed in PEP 3101 (i.e. if wanting a zero padded field with leading
> sign and numeric prefix, you must write "+#0"). Percent format, on the
> other hand, allows the "#", "+" and "0" to be placed in any order you
> like (although they must appear before the field width definition,
> precision specifier and type code).
>
> As far as I can see, that leaves the prefixing of octal numbers ("0o" vs
> "0") as the only true incompatibility between percent formatting and
> brace formatting, and even for those the incompatibility is limited to
> cases where a field width is specified without leading zeroes or a sign
> character is specified. In other cases, the translation can just stick a
> leading literal "0" in front of the field in the brace formatting string.
Helpful analysis there, Nick, thanks. Bonzer ;-)
There's also the corner case of things like
%#.0f
which, when asked to format 3e100, will print
3.e+100
whereas the translated format {0:.0f}, will print
3e+100
for the same value.
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
worth raising an enhancement request on the bug tracker using "O" to generate
compatible rendering for octals.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
> I've had bad experiences in the past with dictionary-based APIs. They seem > "simpler" in the short run, because the user "only needs to create some > dictionaries". Once the complexity of that nested dictionary grows to a > certain > point, though, one has to refer back to documentation constantly to make sure Fair point, and agreed that the schema needs some care, here's roughly what I'm thinking of as an example configuration (YAML): formatters: brief: format: '%(levelname)-8s: %(name)-15s: %(message)s' precise: format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s' filters: allow_foo: name: foo handlers: console: class : logging.StreamHandler formatter: brief level : INFO stream : sys.stdout filters: [allow_foo] file: class : logging.handlers.RotatingFileHandler formatter: precise filename: logconfig.log maxBytes: 1024 backupCount: 3 debugfile: class : logging.FileHandler formatter: precise filename: logconfig-detail.log mode: a loggers: foo: level : ERROR handlers: [debugfile] spam: level : CRITICAL handlers: [debugfile] propagate: no bar.baz: level: WARNING root: level : DEBUG handlers : [console, file] It's not too deeply nested, and I can't see any need for it being more deeply nested. It more or less mirrors the way you'd have to write the corresponding ConfigParser-compatible configuration, but if you use a dict as a common format, you have the benefits which I mentioned of supporting JSON, YAML or Python code in a Django settings.py. > the structure conforms to the "schema". Building a simple config tree using > light-weight classes with documented APIs tends to be more sustainable in the > long run. When you say 'config tree using light-weight classes', I presume you mean a parallel set of classes to the actual logging classes which will be instantiated? ISTM logging classes are already reasonably usable for programmatic configuration, but this doesn't address e.g. updating configurations easily without program code changes, or the ability to configure from say a YAML node, where YAML may be being used for application configuration as a whole. (Unless of course I've misunderstood what you're getting at.) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
> I didn't get your message, could you resend?.
Resent, it may have been stopped by your spam filters since it came from my
vinay-underscore-sajip-at-red-hyphen-dove-dot-com address. The subject was
"Python str.format() and octal formatting compatibility".
> I was thinking the same thing, but it seems like a transition step. I'd
> rather
> not keep such backward compatibility hacks (if you will) around for the long
> haul. How about a flag (maybe '*') at the start of the format specification
> which says "operate in backward compatibility mode"? We could document it as
> being only useful for the % to {} translator, and promise to remove it at
> some
> point in the future. Either actually deprecate it or just promise to
> deprecate
> it in the future.
I don't much mind matter exactly which mechanism we use to distinguish between
0o and 0 prefixes, as long as it's one most people are happy with :-)
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Benjamin Peterson python.org> writes:
> It's also really easy to just write 0{:o} like my translator does.
How does that cope when handed a negative number to format?
>>> "%#0o" % -1234
'-02322'
>>> "0{0:o}".format(-1234)
'0-2322'
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging, Unicode and sockets
Martin v. Löwis v.loewis.de> writes: > I can't understand what the problem with SocketHandler/DatagramHandler > is. As they use pickle, they should surely be able to send records with > Unicode strings in them, no? Of course you are right. When I posted that it was a knee-jerk reaction to the issue that was raised for SysLogHandler configured to use UDP. I did realise a bit later that the issue didn't apply to the other two handlers but I was hoping nobody would notice ;-) > OTOH, why is SMTPHandler not in your list? I assumed smtp.sendmail() would deal with it, as it deals with the wire protocol, but perhaps I was wrong to do so. I noticed that Issue 521270 (SMTP does not handle Unicode) was closed, but I didn't look at it closely. I now see it was perhaps only a partial solution. I did a bit of searching and found this post by Marius Gedminas: http://mg.pov.lt/blog/unicode-emails-in-python.html Now if that's the right approach, shouldn't it be catered for in a more general part of the stdlib than logging - perhaps in smtplib itself? Or, seeing that Marius' post is five years old, is there a better way of doing it using the stdlib as it is now? > For syslog, I don't think that's appropriate. I presume this is meant to > follow RFC 5424? If so, it SHOULD send the data in UTF-8, in which case > it MUST include a BOM also. A.8 then says that if you are not certain > that it is UTF-8 (which you wouldn't be if the application passes a byte > string), you MAY omit the BOM. So ISTM that the right thing to do on 2.x would be: if str to be sent, send as is; if unicode to be sent, encode using utf-8 and send with a BOM. For 3.x, just encode using utf-8 and send with a BOM. Does that seem right? Thanks and regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Initialization of __builtins__
>I'm not top-posting, but gmane is giving me a hard time :-( In the py3k branch, logging has the line _unicode = 'unicode' in dir(__builtins__) to determine the existence of Unicode support. The code in trunk, being 1.5.2 compatible, used hasattr(types, 'UnicodeType') I wanted to eliminate the types import and modernise the trunk code a bit, so I copied the py3k line to the trunk version of logging. It didn't work as expected! So I added the line print dir(__builtins__) in logging fairly early on (though not the very first line - just after the __date__ = ... line. Here's what I got with 2.7a0 and 2.6.1: Python 2.7a0 (trunk:75292M, Oct 9 2009, 09:21:05) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import logging ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] >>> ActivePython 2.6.1.1 (ActiveState Software Inc.) based on Python 2.6.1 (r261:67515, Dec 5 2008, 13:58:38) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import logging ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] >>> However, if I just do dir(__builtins__) in the interactive prompt, I get the whole shebang. Excuse my ignorance, but how come? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Initialization of __builtins__
Antoine Pitrou pitrou.net> writes: > Why do you do this? > In py3k, unicode is always enabled but it's called "str" and the name > "unicode" > doesn't exist. > That wasn't done by me but by GvR (according to svn annotate) in r55818, presumably during the stdlib porting to py3k. I just copied the line over, not thinking it wouldn't work for 2.7. > Why don't you simply write: > > unicode_support = True > try: > unicode > except NameError: > unicode_support = False > That's just about what I've actually done: I was just curious about the difference between py3k and trunk :-) If __builtins__ is an implementation detail which can't be relied on, should the py3k code be changed to the try: form? Or shall I just remove the checks altogether, since Unicode should always be there in 3.x? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] A proposal: configuring logging using dictionaries
A little while ago, I posted here a suggestion about a new way to configure logging, using dictionaries. This received some positive and no negative feedback, so I have thought some more about the details of how it might work. I present below the results of that thinking, in a PEP-style format. I don't know if an actual PEP is required for a change of this type, but I felt that it's still worth going through the exercise to try and achieve a reasonable level of rigour. (I hope I've succeeded.) I would welcome all your feedback on this proposal. If I hear no negative feedback, I propose to implement this feature as suggested. I thought about posting this on comp.lang.python as well, but possibly it's a little too much information for most of the folks there. I think it would be useful to get feedback from the wider community, though, and welcome any suggestions on how best to achieve this. Thanks and regards, Vinay Sajip --- PEP: XXX Title: Dictionary-Based Configuration For Logging Version: $Revision$ Last-Modified: $Date$ Author: Vinay Sajip Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Oct-2009 Python-Version: 2.7 and 3.2 Post-History: Abstract This PEP describes a new way of configuring logging using a dictionary to hold configuration information. Rationale = The present means for configuring Python's logging package is either by using the logging API to configure logging programmatically, or else by means of ConfigParser-based configuration files. Programmatic configuration, while offering maximal control, fixes the configuration in Python code. This does not facilitate changing it easily at runtime, and, as a result, the ability to flexibly turn the verbosity of logging up and down for different parts of a using application is lost. This limits the usability of logging as an aid to diagnosing problems - and sometimes, logging is the only diagnostic aid available in production environments. The ConfigParser-based configuration system is usable, but does not allow its users to configure all aspects of the logging package. For example, Filters cannot be configured using this system. Furthermore, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the Python standard at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python inlude JSON support in the standard library, and this is also usable as a configuration format. In other environments, such as Google App Engine, YAML is used to configure applications, and usually the configuration of logging would be considered an integral part of the application configuration. Although the standard library does not contain YAML support at present, support for both JSON and YAML can be provided in a common way because both of these serialization formats allow deserialization of Python dictionaries. By providing a way to configure logging by passing the configuration in a dictionary, logging will be easier to configure not only for users of JSON and/or YAML, but also for users of bespoke configuration methods, by providing a common format in which to describe the desired configuration. Another drawback of the current ConfigParser-based configuration system is that it does not support incremental configuration: a new configuration completely replaces the existing configuration. Although full flexibility for incremental configuration is difficult to provide in a multi-threaded environment, the new configuration mechanism will allow the provision of limited support for incremental configuration. Specification = The specification consists of two parts: the API and the format of the dictionary used to convey configuration information (i.e. the schema to which it must conform). Naming -- Historically, the logging package has not been PEP-8 conformant. At some future time, this will be corrected by changing method and function names in the package in order to conform with PEP-8. However, in the interests of uniformity, the proposed additions to the API use a naming scheme which is consistent with the present scheme used by logging. API --- The logging.config module will have the following additions: * A class, called ``DictConfigurator``, whose constructor is passed the dictionary used for configuration, and which has a ``configure()`` method. * A callable, called ``dictConfigClass``, which will (by default) be set to ``DictConfigurator``. This is provided so that if desired, ``DictConfigurator`` can be replaced with a suitable user-defined implementation. * A function, called ``dictConfig()``, which takes a single argument - the diction
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Nick Coghlan gmail.com> writes: > Since part of the purpose of this new configuration option is to allow > other projects to use it as a target for specifying a logging > configuration, maintaining it as a formal PEP seems like a good idea to > me. This is already done for a few other standard-ish interfaces > (distutils metadata, DB-API, WSGI). I'm happy with this. I've subscribed to the PEPs list and once the subscription comes through, will post there linking to the PEP draft. > My only real objection is to the "incremental" flag in the dictionary. I > would suggest having two different API methods instead - initDictConfig > and updateDictConfig. Or is there some specific reason for having the > choice of incremental update vs replacement configuration in the hands > of the config author rather than the application developer? I was thinking that in some environments, configuration changes might be sent as pickled dicts over the wire. In such cases it might be useful to have the "incremental" flag in the payload. But I haven't thought that through in detail, as it's somewhat peripheral to this feature. > My other question is whether or not it would be worth having the logging > module able to export it's *current* configuration in dictionary form. I > don't have a use case other than that it might be useful for debugging > logging configuration errors when attempting to combine multiple sources > of logging data, but it's worth considering. Another fair point - perhaps a getConfig() which returns a dict can be added. > You may want to allow yourself AttributeError and ImportError here. > Alternatively, you could commit to converting those to an appropriate > ValueError that provides more detail on the config setting that > contained the problematic reference. Perhaps TypeError, too. The idea is to definitely pinpoint where in the configuration the error occurred. I'll think about the detail of this a bit further down the line, but I just wanted to establish that I wasn't planning on introducing any logging-related exception classes. Thanks for the quick response and detailed feedback. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Vinay Sajip yahoo.co.uk> writes: > Perhaps TypeError, too. Sorry, brainfade there. I already mentioned TypeError in the original post :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Nick Coghlan gmail.com> writes: > My other question is whether or not it would be worth having the logging > module able to export it's *current* configuration in dictionary form. I > don't have a use case other than that it might be useful for debugging > logging configuration errors when attempting to combine multiple sources > of logging data, but it's worth considering. > One restriction on any output of the current configuration is that it may not be suitable for round-tripping, i.e. you may not be able to use the output dict to (re)configure the system. That's because in general, the configurator wants to know how to instantiate handlers, filters, formatters and so it essentially needs the constructor arguments in an incoming dict. However, there's no requirement for the handlers, filters and formatters to keep those constructor arguments around in the original form; and so, in the general case, they are not available to output in a dict describing the current configuration. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.6.4rc2
Barry Warsaw python.org> writes: > http://www.python.org/download/releases/2.6.4/ Good news, but just one little nit: the page above appears to link to the NEWS file for 2.6.4rc1. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making loggerClass an attribute of the logger manager?
Georg Brandl gmx.net> writes: > Making the loggerClass configurable per manager would solve the > problem for me, and AFAICS since most applications don't use > different managers anyway, there should not be any detrimental > effects. What do you think? Seems reasonable. Apart from the API to set/get, _loggerClass is only used by the manager when instantiating a new logger. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 391 ready for review
Full View PEP 391 is, I believe, ready for review. http://www.python.org/dev/peps/pep-0391/ I've also worked up an implementation, though not yet checked it in: it's available as a module "dictconfig.py" with accompanying unit tests in "test_dictconfig.py", at: http://bitbucket.org/vinay.sajip/dictconfig I believe I have incorporated into the PEP comments from people both here and on python-list. There have been comments in favour of the proposal, no objections to the proposal as a whole, and some questions and objections about specific details. I believe I have addressed these with changes to the PEP. Discussion threads on python-dev: http://mail.python.org/pipermail/python-dev/2009-October/092695.html http://mail.python.org/pipermail/python-dev/2009-October/092782.html http://mail.python.org/pipermail/python-dev/2009-October/093062.html And on python-list: http://mail.python.org/pipermail/python-list/2009-October/1223658.html http://mail.python.org/pipermail/python-list/2009-October/1224228.html There hasn't been a *huge* amount of discussion, but as it's a niche topic I suppose that's fair enough. I'm ready to actually integrate these changes in the core logging package. As I understand it, the next step is to put the PEP forward for review, which is why I'm sending this mail. I've already sent an equivalent mail to the PEP editors: David Goodger suggested I incorporate the above links (done) and Brett Cannon suggested I send this to python-dev - also, now, done. Looking forward to the feedback! Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 391 ready for review
Terry Reedy udel.edu> writes: > [snip] Terry, Thanks for your helpful comments. I have checked in some changes to the PEP (r76533) which take into account the comments you made. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 391 ready for review
Dj Gilcrease gmail.com> writes: > > I would recommend removing the class keyword and replacing it with the > () as you have in the custom examples or replacing () with class so it > is uniform across all config options [snip] > > This just strikes me as odd to have to remember for built in handlers > I need to use the class keyword and for my custom handlers I need to > use (). My preference would be the class keyword vs the odd (), as > that is what I am defining, the class to be used for this handler. If you are using the built in handlers, you will be using a class - hence the key 'class' is used. If you are using a custom factory, then the system does not force you to use a class - you can use any callable (e.g. a function which constructs an instance and sets some attributes of it before returning it), and the use of '()' indicates that you're not being forced to use a class. Of course you can, since a class is a callable, but you're not restricted to classes here. > The ext:// I think should be py:// when defining objects to be access > via pythons normal import mechanisms and ext:// for resolving external > processes or system calls > The mechanism I proposed can be extended or changed as you suggest according to the norms of a specific developer environment (e.g. organization policies), but the basic system as I see it doesn't (and shouldn't) care about whether a specific value resolves to an internal (e.g. provided by stdlib) value or an external (e.g. provided by 3rd-party lib) value. The resolution process would be exactly the same in either case. Or perhaps I misunderstood what you meant? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] topics I plan to discuss at the language summit
Brett Cannon python.org> writes: > If there is something missing from the stdlib discussion that you think should be brought up at the summit let me know. And if there is something here you want to discuss before the summit let me know and I can start a separate thread on it. I'm sorry I won't be able to come to the language summit, but I would like if possible to expedite a pronouncement on PEP 391 (configuring logging using dictionaries). I believe I addressed all the comments made on the discussion threads mentioned in the PEP and so I'm not sure what more I need to do to get a pronouncement. I guess the stdlib slot gives an opportunity for people to air their views and so I'd be grateful if you added it to the agenda. Thanks & regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 391 - Please Vote!
In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries: http://www.python.org/dev/peps/pep-0391/ In November 2009 I posted to this list that the PEP was ready for review. I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP. So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things. Thanks & regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 391 - Please Vote!
> From: Jesse Noller > I'm generally +1 - but given I know that Django 1.2 is slated to > implement something somewhat similar, I'm interested to hear how this > proposal meshes with their plan(s).. Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post: http://groups.google.com/group/django-developers/msg/4ef81a2160257221 They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so). Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Mailing List archive corruption?
Hi, When I look at the mailing list archive for python-dev, I see some odd stuff at the bottom of the page: http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232 Anyone know what's happened? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mailing List archive corruption?
Barry Warsaw python.org> writes: > WTF? I think the archives were recently regenerated, so there's probably a > fubar there. CC'ing the postmasters. > Is someone still working on this? I see no updates coming in to the Python-dev archive on mail.python.org, though I do see them on Gmane (for example, the PEP 3146 thread from yesterday). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyCon Keynote
Barry Warsaw python.org> writes: > > On Jan 22, 2010, at 10:06 AM, Chris McDonough wrote: > > >Can you tell us where Uncle Timmy has been and when he'll be back? > > He's given up bags of ham for walls of chocolate. > In the Mountain View Chocolate Factory? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 391 Acceptance - Thanks!
- Original Message > From: Guido van Rossum > To: Jesse Noller > Cc: Vinay Sajip ; python-dev@python.org > >> Assuming PEP 391 gets the nod, then after implementing the changes into > Python, I plan to work with the Django community to get improved logging > support > in Django for 1.3. > > Having read the PEP I heartily approve. The PEP can be moved to > Accepted state. This can be implemented in Python 2.7 and 3.2, right? > --Guido van Rossum (python.org/~guido) Guido, That's great - thanks very much! I'll get on with integrating the PEP 391 implementation into trunk right away, and it should be in good time for 2.7/3.2. Regards, Vinay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] contributor to committer
Florent Xicluna gmail.com> writes: > I am a semi-regular contributor for Python: I have contributed many patches > since end of last year, some of them were reviewed by Antoine. > Lately, he suggested that I should apply for commit rights. +1 Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
Steve Holden holdenweb.com> writes:
> I'm sure we wouldn't want to go so far as to inhibit this. (Py 3.1)
>
> >>> def f(**kwargs):
> ... kwargs[1] = "dummy"
> ... print(kwargs)
> ...
> >>> f(this="Guido", that="Raymond", the_other="Steve")
> {'this': 'Guido', 1: 'dummy', 'the_other': 'Steve', 'that': 'Raymond'}
I think that according to the proposal, the above snippet would be OK, but
def f(**kwargs):
kwargs[1] = 'dummy'
g(**kwargs)
would fail at the call of g.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] urllib2/urllib incompatibility?
> Not top-posting, but gmane seems too finicky in this area.
I encountered what seems like an incompatibility between urllib and urllib2 in
the way they handle file:// URLs, is this a bug? I had a look on the bug tracker
and via Google to see if there were prior reports, but perhaps my search-fu is
deficient. Here's a console session to illustrate:
vi...@eta-karmic:/tmp$ echo Hello, world! >hello.txt
vi...@eta-karmic:/tmp$ cat hello.txt
Hello, world!
vi...@eta-karmic:/tmp$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib,urllib2
>>> s = 'file:tmp/hello.txt'
>>> f1 = urllib.urlopen(s)
>>> f1.read()
'Hello, world!\n'
>>> f2 = urllib2.urlopen(s)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.6/urllib2.py", line 124, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.6/urllib2.py", line 389, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 407, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 1240, in file_open
return self.parent.open(req)
File "/usr/lib/python2.6/urllib2.py", line 389, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 407, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 1287, in ftp_open
raise URLError('ftp error: no host given')
urllib2.URLError:
>>>
Anyone seen this before? Should I file an issue on the tracker? If I've missed
something obvious, sorry for the noise, but please do tell!
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib2/urllib incompatibility?
Senthil Kumaran gmail.com> writes: > > The actual (and Valid) file:// url in your case is 'file:///tmp/hello.txt' > And this was fine and consistent. > > The extra '/' is making it in invalid url in urllib2, I think that be > tracked as bug at least to show a consistent behaviour. The local > file open methods of urllib2 and urllib are different. > Yes, it seems to be a bug in urllib which permits an invalid URL to pass. I came across it by accident, I normally use urls like file://localhost/tmp/hello.txt so I don't trip over the /// hurdles :-) I'll raise an issue on the tracker. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3148 ready for pronouncement
Brian Quinlan sweetapp.com> writes: > "future" is a computing science term of art, like "thread". Anyway, > this has been discussed in the past and Guido was happy with the name. I've not seen this mentioned, but on such a long thread I might have missed it: we already have a "__future__" module, as in from __future__ import with_statement and to my mind, this is a potential point of confusion with the term "future". > > * It seems unnecessarily verbose to tack "Executor" > > onto the end of every Executor subclass. They could > > simply be called ThreadPool and ProcessPool without > > losing anything. > > You could have general thread pools that aren't related to executors > (actually, it would be great if Python had a good built-in thread pool > implementation) and I'd like to avoid using an overly generic name. > Aren't executors and pools both housekeepers around a bunch of threads which execute code as a service for other threads? A thread is useless unless it executes code, isn't it? I'm not sure I understand the fine distinction between an executor and a pool. Having Executor as a suffix will give a point of familiarity to those who already know java.util.concurrent. And ProcessPool might cause confusion with multiprocessing.Pool because at first glance they seem to be the same thing. > > * I don't see a strong reason to put this module > > inside a newly-created namespace. If there were a > > namespace called "concurrent", I would expect to find > > other existing concurrency-related modules there as > > well, such as threading and multiprocessing. But we > > can't move them there without breaking existing code. > > I think that Jesse was planning to add some functionality to this > namespace. I don't really have an opinion on this. > I'm not sure of the benefit of a "concurrent" namespace, since it wouldn't contain the existing concurrency stuff. I think it's something to consider only for a big reorg which would break backward compatibility. IMO it would make more sense to leave this module as a top-level module for now (a sibling to "threading", "multiprocessing"). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle security and remote logging
anatoly techtonik gmail.com> writes: > insecure. SocketHandler and DatagramHandler docs should at least > contain a warning about danger of exposing unpickling interfaces to > insecure networks. I've updated the documentation of SocketHandler.makePickle to mention security concerns, and that the method can be overridden to use a more secure implementation (e.g. HMAC-signed pickles). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle security and remote logging
Guido van Rossum python.org> writes: > As for protocol buffers, assuming its absence (so far from the > stdlib is the only objection, how hard would it be to make the logging > package "prepared" so that if one *did* have protocol buffers > installed, it would be a one-line config setting to use them? I envisage that if protocol buffers were available, and if support for them in logging was to be added, this could be done via an optional keyword arg to the SocketHandler which sets a handler attribute, which would then be used in makePickle to make the required serialized form. @anatoly: The documentation just mentions HMAC as an example; the levels of paranoia to be applied are different for different people, different times and different situations ;-) I assume that someone reading the docs could readily see that they could substitute "sign the pickle" with some alternative strategy in makePickle. You could implement marshal, protocol buffers etc. right now just by overriding SocketHandler.makePickle in your custom class. An alternative strategy would be to provide an optional serializer=None callable in the SocketHandler constructor. If specified, then makePickle would call this serializer with the LogRecord instance as the only argument, and use the return value as the serialized form, instead of calling pickle.dumps. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python needs a standard asynchronous return object
James Yonan openvpn.net> writes: > I'd like to propose that the Python community standardize on a > "deferred" object for asynchronous return values Have a look at PEP 3148: http://www.python.org/dev/peps/pep-3148/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python needs a standard asynchronous return object
James Yonan openvpn.net> writes: > > I'd like to propose that the Python community standardize on a > "deferred" object for asynchronous return values, modeled after the > well-thought-out Twisted Deferred class. Ummm, sorry, I seem to need new glasses :-( ignore my last post. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Some changes to logging for 3.2
Hi all, I'm planning to make some smallish changes to logging in Python 3.2, please see http://plumberjack.blogspot.com/2010/09/improved-queuehandler-queuelistener.html If you're interested, I'd be grateful for any feedback you can give. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Some changes to logging for 3.2
Nick Coghlan gmail.com> writes: > To further reduce overhead, would it make sense for the signature of > the QueueListener constructor to be (queue, *handlers)? Good suggestion - thanks. Regards, Vinay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Some changes to logging for 3.2
Chris Withers simplistix.co.uk> writes: > > Cool, how can I use it in Python 2.6? > > Chris Hi Chris, 1. Copy the top part (imports, QueueHandler and QueueListener classes) from the Gist linked to in the article - http://gist.github.com/591699 - into a utility module you can use again and again. 2. In your code, try to import them from logging.handlers and in an except ImportError: clause, import them from your utility module. 3. Profit ;-) Note that there's a slight change in the Gist from when the post was published - but it should still work as per the post. Until 3.2 is out, there may be other small changes: but you can copy the code over from the 3.2 branch in the Python SVN repository later and it should work fine under Python 2.6 and 2.7. Of course if you do find any problems (or have any other questions), please let me know asap :-) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WSGI is now Python 3-friendly
Martin v. Löwis v.loewis.de> writes: > > I'm still with Guido here: I'd accept PEP 333 as final in the state it > had last week, give PJE's edits a new PEP number, and accept that as > final right away also. > This sounds like it should make everyone happy - no rewriting of history, and no procedural roadblocks or delays to the amendments PJE wants to make. Are we done here, or am I missing something? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Change to logging Formatters: support for alternative format styles
I've just checked in a change to logging into the py3k branch (r85835),
including doc changes and tests, for providing slightly more flexibility in
alternative format styles for logging.
Basically, Formatter.__init__ gets an extra optional keyword arg style=. This is then used to merge the format string with
the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or
string.Template(fmt).substitute(**record.dict). Backward compatibility is
maintained (unless I've missed something).
This does not cater for how you combine the logging message + its args when you
make a logging call, but should work with any existing library which uses
%-formatting in its logging calls.
We discussed this here around a year ago in the context of generally encouraging
a move from %-formatting to {}-formatting, but there seemed to be no obvious
easy path forward.
As of now, and even without this change I've just checked in, developers can use
{}- or $-formatting in their logging messages using e.g.
logger.debug(__('Message with {} {}', 2, 'place-holders'))
with a suitable class standing in for __. A bit ugly, I know, with extra
parentheses and what not, but hopefully for those who must have {} now it
shouldn't be too much of a price to pay and there's no major performance
implication, as actual rendering to string is done as late as possible, just as
it is now.
Comments welcome. Assuming there are no strong objections asking for reversion
of this change, I'll publicise to the wider community in a few days.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Change to logging Formatters: support for alternative format styles
Nick Coghlan gmail.com> writes: > Looking at your checkin though, I wonder if it might be worth > implementing some little formatting style classes to get rid of the > if/elif chains from the Formatter code. Something like: Fair comment: I did think about the messiness of that if/elif, but considered it OK as it was likely that Python would not grow any more formatting approaches. However, I hadn't thought about user-defined approaches; what you're suggesting is more extensible. I'll incorporate your and Boštjan's comments in the next checkin. Thanks, Vinay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
Kristján Valur Jónsson ccpgames.com> writes: > Let’s move the current ‘trunk’ into /branches/afterlife-27. Open it for > submissions from people such as myself that use 2.7 on a regular basis and are > willing to give it some extra love. Just curious - what specific new features or backwards-incompatible fixes do you need to add, i.e. things which cannot be catered for by release27-maint? Or is this just about the *principle* of having a 2.8? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
Vinay Sajip yahoo.co.uk> writes: > need to add, i.e. things which cannot be catered for by release27-maint? Or is > this just about the *principle* of having a 2.8? Never mind - I've just picked up the extra posts on this thread, which for some reason didn't show up in my reader before. Sorry for the noise. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
