ANN: python-blosc 1.2.0 released
= Announcing python-blosc 1.2.0 = What is new? This release adds support for the multiple compressors added in Blosc 1.3 series. The new compressors are: * lz4 (http://code.google.com/p/lz4/): A very fast compressor/decompressor. Could be thought as a replacement of the original BloscLZ, but it can behave better is some scenarios. * lz4hc (http://code.google.com/p/lz4/): This is a variation of LZ4 that achieves much better compression ratio at the cost of being much slower for compressing. Decompression speed is unaffected (and sometimes better than when using LZ4 itself!), so this is very good for read-only datasets. * snappy (http://code.google.com/p/snappy/): A very fast compressor/decompressor. Could be thought as a replacement of the original BloscLZ, but it can behave better is some scenarios. * zlib (http://www.zlib.net/): This is a classic. It achieves very good compression ratios, at the cost of speed. However, decompression speed is still pretty good, so it is a good candidate for read-only datasets. Selecting the compressor is just a matter of specifying the new `cname` parameter in compression functions. For example:: in = numpy.arange(N, dtype=numpy.int64) out = blosc.pack_array(in, cname=lz4) Just to have an overview of the differences between the different compressors in new Blosc, here it is the output of the included compress_ptr.py benchmark: https://github.com/ContinuumIO/python-blosc/blob/master/bench/compress_ptr.py that compresses/decompresses NumPy arrays with different data distributions:: Creating different NumPy arrays with 10**7 int64/float64 elements: *** np.copy() Time for memcpy(): 0.030 s *** the arange linear distribution *** *** blosclz *** Time for comp/decomp: 0.013/0.022 s. Compr ratio: 136.83 *** lz4 *** Time for comp/decomp: 0.009/0.031 s. Compr ratio: 137.19 *** lz4hc*** Time for comp/decomp: 0.103/0.021 s. Compr ratio: 165.12 *** snappy *** Time for comp/decomp: 0.012/0.045 s. Compr ratio: 20.38 *** zlib *** Time for comp/decomp: 0.243/0.056 s. Compr ratio: 407.60 *** the linspace linear distribution *** *** blosclz *** Time for comp/decomp: 0.031/0.036 s. Compr ratio: 14.27 *** lz4 *** Time for comp/decomp: 0.016/0.033 s. Compr ratio: 19.68 *** lz4hc*** Time for comp/decomp: 0.188/0.020 s. Compr ratio: 78.21 *** snappy *** Time for comp/decomp: 0.020/0.032 s. Compr ratio: 11.72 *** zlib *** Time for comp/decomp: 0.290/0.048 s. Compr ratio: 90.90 *** the random distribution *** *** blosclz *** Time for comp/decomp: 0.083/0.025 s. Compr ratio: 4.35 *** lz4 *** Time for comp/decomp: 0.022/0.034 s. Compr ratio: 4.65 *** lz4hc*** Time for comp/decomp: 1.803/0.039 s. Compr ratio: 5.61 *** snappy *** Time for comp/decomp: 0.028/0.023 s. Compr ratio: 4.48 *** zlib *** Time for comp/decomp: 3.146/0.073 s. Compr ratio: 6.17 That means that Blosc in combination with LZ4 can compress at speeds that can be up to 3x faster than a pure memcpy operation. Decompression is a bit slower (but still in the same order than memcpy()) probably because writing to memory is slower than reading. This was using an Intel Core i5-3380M CPU @ 2.90GHz, runnng Python 3.3 and Linux 3.7.10, but YMMV (and will vary!). For more info, you can have a look at the release notes in: https://github.com/ContinuumIO/python-blosc/wiki/Release-notes More docs and examples are available in the documentation site: http://blosc.pydata.org What is it? === python-blosc (http://blosc.pydata.org/) is a Python wrapper for the Blosc compression library. Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Whether this is achieved or not depends of the data compressibility, the number of cores in the system, and other factors. See a series of benchmarks conducted for many different systems: http://blosc.org/trac/wiki/SyntheticBenchmarks. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. There is also a handy command line for Blosc called Bloscpack (https://github.com/esc/bloscpack) that allows you to compress large binary datafiles on-disk. Although the format for Bloscpack has not stabilized yet, it allows you to effectively use Blosc from your favorite shell. Installing == python-blosc is in PyPI repository, so installing it is easy: $ pip install -U blosc # yes, you should omit the python- prefix Download sources The sources are managed through github services at:
[ANN] Python Dinner at FOSDEM 2014
Dear Pythonista friends, write down in your agenda the date of the next AFPyro! The next Aperos Python Belgium will take place on saturday, February 1st in Brussels, during the FOSDEM conference that takes place at ULB from 1st to 2nd February. In addition to the regulars of AFPyro, we are also pleased to meet the attendees of FOSDEM using Python. As usual, you can come just for drinking and sharing one (or many) drinks or if you wish, you can also register for the meal that follows, in a near place, at “Chez Léon” for 20:30, rue des Bouchers 18, B-1000 Brussels. Meeting from 18:45, on first floor of Delirium cafe, Impasse De La Fidélité 4, 1000 Brussels. If you are a FOSDEM attendee, you can meet us at the Python stand, 1st floor of K Building, from 18h to go to Delirium cafe together. Please sign up in order to book the place: http://doodle.com/mn3yck6n3xxidsim Further information about the next AFPyros in Belgium: Aperos Python Belgium Here is the official Announce: http://afpyro.afpy.org/dates/2014/2014_02_01.html Regards, Stéphane Wirtel -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: BLZ 0.6.1 has been released
Announcing BLZ 0.6 series
=
What it is
--
BLZ is a chunked container for numerical data. Chunking allows for
efficient enlarging/shrinking of data container. In addition, it can
also be compressed for reducing memory/disk needs. The compression
process is carried out internally by Blosc, a high-performance
compressor that is optimized for binary data.
The main objects in BLZ are `barray` and `btable`. `barray` is meant
for storing multidimensional homogeneous datasets efficiently.
`barray` objects provide the foundations for building `btable`
objects, where each column is made of a single `barray`. Facilities
are provided for iterating, filtering and querying `btables` in an
efficient way. You can find more info about `barray` and `btable` in
the tutorial:
http://blz.pydata.org/blz-manual/tutorial.html
BLZ can use numexpr internally so as to accelerate many vector and
query operations (although it can use pure NumPy for doing so too)
either from memory or from disk. In the future, it is planned to use
Numba as the computational kernel and to provide better Blaze
(http://blaze.pydata.org) integration.
What's new
--
BLZ has been branched off from the Blaze project
(http://blaze.pydata.org). BLZ was meant as a persistent format and
library for I/O in Blaze. BLZ in Blaze is based on previous carray
0.5 and this is why this new version is labeled 0.6.
BLZ supports completely transparent storage on-disk in addition to
memory. That means that *everything* that can be done with the
in-memory container can be done using the disk as well.
The advantages of a disk-based container is that the addressable space
is much larger than just your available memory. Also, as BLZ is based
on a chunked and compressed data layout based on the super-fast Blosc
compression library, the data access speed is very good.
The format chosen for the persistence layer is based on the
'bloscpack' library and described in the Persistent format for BLZ
chapter of the user manual ('docs/source/persistence-format.rst').
More about Bloscpack here: https://github.com/esc/bloscpack
You may want to know more about BLZ in this blog entry:
http://continuum.io/blog/blz-format
In this version, support for Blosc 1.3 has been added, that meaning
that a new `cname` parameter has been added to the `bparams` class, so
that you can select you preferred compressor from 'blosclz', 'lz4',
'lz4hc', 'snappy' and 'zlib'.
Also, many bugs have been fixed, providing a much smoother experience.
CAVEAT: The BLZ/bloscpack format is still evolving, so don't trust on
forward compatibility of the format, at least until 1.0, where the
internal format will be declared frozen.
Resources
-
Visit the main BLZ site repository at:
http://github.com/ContinuumIO/blz
Read the online docs at:
http://blz.pydata.org/blz-manual/index.html
Home of Blosc compressor:
http://www.blosc.org
User's mail list:
blaze-...@continuum.io
Enjoy!
Francesc Alted
Continuum Analytics, Inc.
--
https://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
[RELEASED] Python 3.4.0b3
On behalf of the Python development team, I'm quite pleased to announce the third beta release of Python 3.4. This is a preview release, and its use is not recommended for production settings. Python 3.4 includes a range of improvements of the 3.x series, including hundreds of small improvements and bug fixes. Major new features and changes in the 3.4 release series include: * PEP 428, a pathlib module providing object-oriented filesystem paths * PEP 435, a standardized enum module * PEP 436, a build enhancement that will help generate introspection information for builtins * PEP 442, improved semantics for object finalization * PEP 443, adding single-dispatch generic functions to the standard library * PEP 445, a new C API for implementing custom memory allocators * PEP 446, changing file descriptors to not be inherited by default in subprocesses * PEP 450, a new statistics module * PEP 451, standardizing module metadata for Python's module import system * PEP 453, a bundled installer for the *pip* package manager * PEP 454, a new tracemalloc module for tracing Python memory allocations * PEP 456, a new hash algorithm for Python strings and binary data * PEP 3154, a new and improved protocol for pickled objects * PEP 3156, a new asyncio module, a new framework for asynchronous I/O Python 3.4 is now in feature freeze, meaning that no new features will be added. The final release is projected for mid-March 2014. To download Python 3.4.0b3 visit: http://www.python.org/download/releases/3.4.0/ Please consider trying Python 3.4.0b3 with your code and reporting any new issues you notice to: http://bugs.python.org/ Enjoy! -- Larry Hastings, Release Manager larry at hastings.org (on behalf of the entire python-dev team and 3.4's contributors) -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: EmPy 3.3.2 released (Python 3.x compatibility)
Summary
A powerful and robust templating system for Python.
Overview
EmPy is a system for embedding Python expressions and statements
in template text; it takes an EmPy source file, processes it, and
produces output. This is accomplished via expansions, which are
special signals to the EmPy system and are set off by a special
prefix (by default the at sign, '@'). EmPy can expand arbitrary
Python expressions and statements in this way, as well as a
variety of special forms. Textual data not explicitly delimited
in this way is sent unaffected to the output, allowing Python to
be used in effect as a markup language. Also supported are
callbacks via hooks, recording and playback via diversions, and
dynamic, chainable filters. The system is highly configurable via
command line options and embedded commands.
Expressions are embedded in text with the '@(...)' notation;
variations include conditional expressions with '@(...?...!...)'
and the ability to handle thrown exceptions with '@(...$...)'. As
a shortcut, simple variables and expressions can be abbreviated as
'@variable', '@object.attribute', '@function(arguments)',
'@sequence' [index], and combinations. Full-fledged statements
are embedded with '@{...}'. Control flow in terms of conditional
or repeated expansion is available with '@[...]'. A '@' followed
by a whitespace character (including a newline) expands to
nothing, allowing string concatenations and line continuations.
Comments are indicated with '@#' and consume the rest of the line,
up to and including the trailing newline. '@%' indicate
significators, which are special forms of variable assignment
intended to specify per-file identification information in a
format which is easy to parse externally. Context name and line
number changes can be done with '@?' and '@!' respectively.
'@...' markups are customizeable by the user and can be used for
any desired purpose. Escape sequences analogous to those in C can
be specified with '@\...', and finally a '@@' sequence expands to
a single literal at sign.
Getting the software
The current version of empy is 3.3.2.
The latest version of the software is available in a tarball here:
http://www.alcyone.com/software/empy/empy-latest.tar.gz.
The official URL for this Web site is
http://www.alcyone.com/software/empy/.
Requirements
EmPy should work with any version of Python from 2.4 onward,
including 3.x.
License
This code is released under the LGPL.
Release history [since 3.3]
- 3.3.2; 2014 Jan 24. Additional fix for source compatibility
between 2.x and 3.0.
- 3.3.1; 2014 Jan 22. Source compatibility for 2.x and 3.0;
1.x and Jython compatibility dropped.
--
Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/
San Jose, CA, USA 37 18 N 121 57 W AIM/Y!M/Jabber erikmaxfrancis
Sitting in the den and / Looking at the phone as if it owed / Owed me
a favor -- Blu Cantrell
--
https://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
Enable unicode
Hi, ALL, In here: http://stackoverflow.com/questions/21397035/set-utf8-on-mysql, I got a suggestion to enable use_unicode. Problem is I'm developing on Windows and it's not that I can recompile my python. I'm using Python2.7 on Windows XP. Any pointer on how do I enable use_unicode? Thank you. -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On Tue, Jan 28, 2014 at 7:26 PM, Igor Korot ikoro...@gmail.com wrote: Hi, ALL, In here: http://stackoverflow.com/questions/21397035/set-utf8-on-mysql, I got a suggestion to enable use_unicode. Problem is I'm developing on Windows and it's not that I can recompile my python. I'm using Python2.7 on Windows XP. Any pointer on how do I enable use_unicode? Before you go any further: MySQL has a broken interpretation of utf8 that allows only a subset of the full Unicode range. Instead, use utf8mb4, which is what the rest of the world calls UTF-8. As far as I know, you can just switch in utf8mb4 everywhere that you're currently using utf8 and it'll work. According to [1] the use_unicode flag is a keyword parameter to connect(). As much as possible, I'd recommend using those parameters rather than explicitly executing SQL statements to reconfigure the connection - it's clearer, and the local client might want to reconfigure itself in response to the change too. Be aware that MySQL has a number of issues with Unicode and sorting (or at least, it did the last time I checked, which was a while ago now), not to mention other problems with its default MyISAM format. You may want to consider PostgreSQL instead. ChrisA [1] http://mysql-python.sourceforge.net/MySQLdb.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Highlighting program variables instead of keywords?
On 28/01/2014 07:19, wxjmfa...@gmail.com wrote: Different, but a little bit related. The work which is done actually on the possibility (not implemented but alreay realized) to colorize (style) the different graphemes of a glyph is very interesting. Python with its absurd Flexible String Representation just become a no go for the kind of task. (Should not be too complicate to understand.) jmf This guy has surely exceeded his three strikes and you're out limit? Please, please somebody do something about it, he's driving me insane with this continuous drivel. No thread appears to be safe from him jumping in with this nonsense. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On Tue, Jan 28, 2014 at 7:56 PM, Igor Korot ikoro...@gmail.com wrote: Hi, Chris, Hi! I'm hoping it was oversight that led to this email coming to me personally instead of to the list, and hoping that you won't mind me responding on-list. On Tue, Jan 28, 2014 at 12:35 AM, Chris Angelico ros...@gmail.com wrote: On Tue, Jan 28, 2014 at 7:26 PM, Igor Korot ikoro...@gmail.com wrote: Hi, ALL, In here: http://stackoverflow.com/questions/21397035/set-utf8-on-mysql, I got a suggestion to enable use_unicode. Problem is I'm developing on Windows and it's not that I can recompile my python. I'm using Python2.7 on Windows XP. Any pointer on how do I enable use_unicode? Before you go any further: MySQL has a broken interpretation of utf8 that allows only a subset of the full Unicode range. Instead, use utf8mb4, which is what the rest of the world calls UTF-8. As far as I know, you can just switch in utf8mb4 everywhere that you're currently using utf8 and it'll work. So instead of using 'utf8' just use 'utf8mb4'? Yes, that's right. Unless utf8mb4 isn't supported, in which case try utf8 and see if you can use the full range (something might be translating it for you, which would probably be a good thing). According to [1] the use_unicode flag is a keyword parameter to connect(). As much as possible, I'd recommend using those parameters rather than explicitly executing SQL statements to reconfigure the connection - it's clearer, and the local client might want to reconfigure itself in response to the change too. Is it supported on all versions of MySQLDB? No idea! I don't use MySQLDB, so just give it a shot and see if it works. Be aware that MySQL has a number of issues with Unicode and sorting (or at least, it did the last time I checked, which was a while ago now), not to mention other problems with its default MyISAM format. You may want to consider PostgreSQL instead. I'm not using MyISAM, only InnoDB. ;-) That's good, but it doesn't cover everything. You may find that non-ASCII strings get mis-sorted. I strongly prefer PostgreSQL for anything where I actually care about the data I'm storing. And yes, that's everything that I store. So I don't use MySQL anywhere any more :) So, how do I properly write the connection lines? I've no idea - I don't actually use MySQLDB, I just looked at the docs :) But try adding use_unicode=True to your connect() call. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
Hi, Chris, On Tue, Jan 28, 2014 at 1:00 AM, Chris Angelico ros...@gmail.com wrote: On Tue, Jan 28, 2014 at 7:56 PM, Igor Korot ikoro...@gmail.com wrote: Hi, Chris, Hi! I'm hoping it was oversight that led to this email coming to me personally instead of to the list, and hoping that you won't mind me responding on-list. Sorry about that. I keep forgetting that gmail web mail is stupid and does not use the list address on Reply. Does not happen to other list, only this one. On Tue, Jan 28, 2014 at 12:35 AM, Chris Angelico ros...@gmail.com wrote: On Tue, Jan 28, 2014 at 7:26 PM, Igor Korot ikoro...@gmail.com wrote: Hi, ALL, In here: http://stackoverflow.com/questions/21397035/set-utf8-on-mysql, I got a suggestion to enable use_unicode. Problem is I'm developing on Windows and it's not that I can recompile my python. I'm using Python2.7 on Windows XP. Any pointer on how do I enable use_unicode? Before you go any further: MySQL has a broken interpretation of utf8 that allows only a subset of the full Unicode range. Instead, use utf8mb4, which is what the rest of the world calls UTF-8. As far as I know, you can just switch in utf8mb4 everywhere that you're currently using utf8 and it'll work. So instead of using 'utf8' just use 'utf8mb4'? Yes, that's right. Unless utf8mb4 isn't supported, in which case try utf8 and see if you can use the full range (something might be translating it for you, which would probably be a good thing). So you mean just try it in the python console and see if it works? According to [1] the use_unicode flag is a keyword parameter to connect(). As much as possible, I'd recommend using those parameters rather than explicitly executing SQL statements to reconfigure the connection - it's clearer, and the local client might want to reconfigure itself in response to the change too. Is it supported on all versions of MySQLDB? No idea! I don't use MySQLDB, so just give it a shot and see if it works. Problem is it might work on my machine, but the person I'm working with/for might have earlier version of the driver. That's actually why I asked on SO - is it supported on all versions of MySQLDB. Be aware that MySQL has a number of issues with Unicode and sorting (or at least, it did the last time I checked, which was a while ago now), not to mention other problems with its default MyISAM format. You may want to consider PostgreSQL instead. I'm not using MyISAM, only InnoDB. ;-) That's good, but it doesn't cover everything. You may find that non-ASCII strings get mis-sorted. I strongly prefer PostgreSQL for anything where I actually care about the data I'm storing. And yes, that's everything that I store. So I don't use MySQL anywhere any more :) Hehe. Originally the software was written for SQLite. Then mySQL support was added and now the new script should be made based on the existing one and this new script will be used in a web based app - Django + jQuery/jQWidgets. The person is not a developer, rather have a management background so he is going with the market trend, which is currently mySQL for web app. ;-) I'd very much prefer to use SQLite as the data we use are text of the different length which is more suitable for SQLite. But So, how do I properly write the connection lines? I've no idea - I don't actually use MySQLDB, I just looked at the docs :) But try adding use_unicode=True to your connect() call. OK, will do, thx. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On Tue, Jan 28, 2014 at 8:20 PM, Igor Korot ikoro...@gmail.com wrote: So instead of using 'utf8' just use 'utf8mb4'? Yes, that's right. Unless utf8mb4 isn't supported, in which case try utf8 and see if you can use the full range (something might be translating it for you, which would probably be a good thing). So you mean just try it in the python console and see if it works? Yep. Play around with four categories of character: ASCII, Latin-1, Basic Multilingual Plane, and Supplementary Multilingual Plane. They correspond to codepoints 128, 256, 65536, and 1114112. Is it supported on all versions of MySQLDB? No idea! I don't use MySQLDB, so just give it a shot and see if it works. Problem is it might work on my machine, but the person I'm working with/for might have earlier version of the driver. That's actually why I asked on SO - is it supported on all versions of MySQLDB. Try it on yours, try it on theirs; or dig through the docs to see when it was added. Hard to say without knowing exactly what version your client is using. I strongly prefer PostgreSQL for anything where I actually care about the data I'm storing. And yes, that's everything that I store. So I don't use MySQL anywhere any more :) Hehe. Originally the software was written for SQLite. Then mySQL support was added and now the new script should be made based on the existing one and this new script will be used in a web based app - Django + jQuery/jQWidgets. The person is not a developer, rather have a management background so he is going with the market trend, which is currently mySQL for web app. ;-) I'd very much prefer to use SQLite as the data we use are text of the different length which is more suitable for SQLite. But Market trend is deceptive AND meaningless. Yes, there's a general notion that low-grade web servers will give you Apache, MySQL, and PHP, while not offering anything else; but that's like saying that a dirt-cheap car comes with a radio and no CD player, ergo you should be listening to the radio even in a car that has (or could have) a CD player. The reason cheap web hosts offer MySQL is because lots of people ask for it, and the reason people ask for it is because lots of hosts offer it. There's no other reason. If you're the expert and he's of management background, then it's your job to sell the technically superior option despite his initial gut feeling... be an advocate, not an automaton. :) And hopefully he'll respect your skills enough to listen to your advice. I don't have much experience with SQLite, but I would suspect that text of different lengths should be able to be handled equally cleanly by any database engine - at least the major ones. You may want to look into whether you use VARCHAR, TEXT, or some other data type, but it should work just fine. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Coordinates TK and Turtle
Hello I have some weird results when I run my code which is meant to display a canvas and a turtle and some text with the turtles coordinates. Basically the turtle coordinates do not seem to correspond with the TK create_text coordinates. t1.goto(100,100) canvas_id = cv1.create_text(t1.xcor(), t1.ycor(), font=(Purisa,12),anchor=nw) cv1.insert(canvas_id, 0, t1.pos()) I end up with this output: http://i1025.photobucket.com/albums/y319/duxbuz/coord-issues_zps1fca6d2b.jpg Could anyone help please? Thanks -- https://mail.python.org/mailman/listinfo/python-list
ANN: EmPy 3.3.2 released (Python 3.x compatibility)
Summary
A powerful and robust templating system for Python.
Overview
EmPy is a system for embedding Python expressions and statements
in template text; it takes an EmPy source file, processes it, and
produces output. This is accomplished via expansions, which are
special signals to the EmPy system and are set off by a special
prefix (by default the at sign, '@'). EmPy can expand arbitrary
Python expressions and statements in this way, as well as a
variety of special forms. Textual data not explicitly delimited
in this way is sent unaffected to the output, allowing Python to
be used in effect as a markup language. Also supported are
callbacks via hooks, recording and playback via diversions, and
dynamic, chainable filters. The system is highly configurable via
command line options and embedded commands.
Expressions are embedded in text with the '@(...)' notation;
variations include conditional expressions with '@(...?...!...)'
and the ability to handle thrown exceptions with '@(...$...)'. As
a shortcut, simple variables and expressions can be abbreviated as
'@variable', '@object.attribute', '@function(arguments)',
'@sequence' [index], and combinations. Full-fledged statements
are embedded with '@{...}'. Control flow in terms of conditional
or repeated expansion is available with '@[...]'. A '@' followed
by a whitespace character (including a newline) expands to
nothing, allowing string concatenations and line continuations.
Comments are indicated with '@#' and consume the rest of the line,
up to and including the trailing newline. '@%' indicate
significators, which are special forms of variable assignment
intended to specify per-file identification information in a
format which is easy to parse externally. Context name and line
number changes can be done with '@?' and '@!' respectively.
'@...' markups are customizeable by the user and can be used for
any desired purpose. Escape sequences analogous to those in C can
be specified with '@\...', and finally a '@@' sequence expands to
a single literal at sign.
Getting the software
The current version of empy is 3.3.2.
The latest version of the software is available in a tarball here:
http://www.alcyone.com/software/empy/empy-latest.tar.gz.
The official URL for this Web site is
http://www.alcyone.com/software/empy/.
Requirements
EmPy should work with any version of Python from 2.4 onward,
including 3.x.
License
This code is released under the LGPL.
Release history [since 3.3]
- 3.3.2; 2014 Jan 24. Additional fix for source compatibility
between 2.x and 3.0.
- 3.3.1; 2014 Jan 22. Source compatibility for 2.x and 3.0;
1.x and Jython compatibility dropped.
--
Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/
San Jose, CA, USA 37 18 N 121 57 W AIM/Y!M/Jabber erikmaxfrancis
Sitting in the den and / Looking at the phone as if it owed / Owed me
a favor -- Blu Cantrell
--
https://mail.python.org/mailman/listinfo/python-list
Re: Highlighting program variables instead of keywords?
On Mon, 27 Jan 2014 23:19:03 -0800, wxjmfauth wrote: Different, but a little bit related. The work which is done actually on the possibility (not implemented but alreay realized) to colorize (style) the different graphemes of a glyph is very interesting. Python with its absurd Flexible String Representation just become a no go for the kind of task. (Should not be too complicate to understand.) No, not complicated at all. Water is wet, therefore the FSR is rubbish. Athens is the capital of Greece, therefor the FSR is rubbish. 1+1 = 2, therefore the FSR is rubbish. The South American Potoo is a member of the Nyctibiidae family, therefore the FSR is rubbish. We get the point, thank you. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
uninstalling python27 killed vim (actual issue was more complicated and involves Mercurial)
I'm posting this information to help others who are transitioning from Python 2.x to Python 3.x and are using Vim and Mercurial on Windows. BACKGROUND Old workstation configuration: 32-bit Windows XP/SP3, Python 2.7.6 and 3.3.3, Mercurial 2.8.2, PATH contained Python before Mercurial, dual boot with Slackware 14.0 New workstation configuration: 64-bit Windows 7/SP1, 64-bit Python 3.3.3, 64-bit Mercurial 2.8.2, PATH contained Python before Mercurial, dual boot with Slackware64 14.1 Early last year, I ported dozens of Python 2.7 tools that I wrote to automate various development tasks to Python 3 (via 2to3 script and testing). After the ports were completed, I had my colleagues replace their Python2 installations with Python3. We quickly uncovered/fixed a few issues (mostly related to Unicode) that I missed in my initial testing. We've been happily running the Python3 versions for months now, but I've been reluctant to remove Python27 from my old (soon to be retired) workstation. PROBLEM Yesterday, I finally decided to say goodbye to Python27 on Windows and uninstalled it from my old workstation. A little while later, I noticed that vim no longer worked. Gvim.exe silently failed and vim.exe failed with the following error: ImportError: No module named site After reading the relevant vim documentation (http://vimdoc.sourceforge.net/htmldoc/if_pyth.html#python-dynamic) and doing a few Google searches, I decided to run vim under windbg to see what was actually going on. Windbg showed that vim loaded the python27.dll from my Mercurial installation just before exiting. A quick look inside Mercurial's library.zip file showed that the site.py module was not included. OK, so now things were starting to make sense. WORKAROUND I took Mercurial out of the PATH, which returned vim to working order. I then wrote the following two line hg.bat script and copied it to a directory include in the PATH: @echo off C:\Program Files\Mercurial\hg.exe %* With this workaround, both vim and hg appear to be once again working on the old workstation. 64-BIT WINDOWS As the vim for Windows package I installed is a 32-bit application, I didn't have this issue on 64-bit Windows using 64-bit Mercurial. However, I believe the same issue would occur if 32-bit Mercurial is used on 64-bit Windows. Hopefully this information is useful to other Python, Vim, and Mercurial users. Peter Santoro -- https://mail.python.org/mailman/listinfo/python-list
Wikipedia XML Dump
Hi I have downloaded and unzipped the xml dump of Wikipedia (40+GB). I want to use Python and the SAX module (running under Windows 7) to carry out off-line phrase-searches of Wikipedia and to return a count of the number of hits for each search. Typical phrase-searches might be of the dog and dog's. I have some limited prior programming experience (from many years ago) and I am currently learning Python from a course of YouTube tutorials. Before I get much further, I wanted to ask: Is what I am trying to do actually feasible? Are there any example programs or code snippets that would help me? Any advice or guidance would be gratefully received. Best regards, Kevin Glover -- https://mail.python.org/mailman/listinfo/python-list
Re: Coordinates TK and Turtle
dux...@gmail.com wrote:
Hello
I have some weird results when I run my code which is meant to display a
canvas and a turtle and some text with the turtles coordinates.
Basically the turtle coordinates do not seem to correspond with the TK
create_text coordinates.
t1.goto(100,100)
canvas_id = cv1.create_text(t1.xcor(), t1.ycor(),
font=(Purisa,12),anchor=nw)
cv1.insert(canvas_id, 0, t1.pos())
I end up with this output:
http://i1025.photobucket.com/albums/y319/duxbuz/coord-
issues_zps1fca6d2b.jpg
Could anyone help please?
The image you provide suggests (x, y) -- (x, -y), but a quick look into the
source reveals that it is a little more complex:
print inspect.getsource(turtle._Screen._write)
def _write(self, pos, txt, align, font, pencolor):
Write txt at pos in canvas with specified font
and color.
Return text item and x-coord of right bottom corner
of text's bounding box.
x, y = pos
x = x * self.xscale
y = y * self.yscale
anchor = {left:sw, center:s, right:se }
item = self.cv.create_text(x-1, -y, text = txt, anchor =
anchor[align],
fill = pencolor, font = font)
x0, y0, x1, y1 = self.cv.bbox(item)
self.cv.update()
return item, x1-1
Rather than repeating the above calculation in your own code I suggest that
you use turtle.write()
import turtle
turtle = turtle.getturtle()
turtle.goto(100, 100)
turtle.write(Hello, world!)
--
https://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.8.0 Released!
I'm pleased to announce a new release of Mailinglogger. Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested This release adds flood limiting to the SummarisingLogger, ensuring that emails it sends aren't bigger than MTAs are prepared to handle. Full docs can be found here: http://packages.python.org/mailinglogger/ For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
Igor Korot ikoro...@gmail.com writes: On Tue, Jan 28, 2014 at 1:00 AM, Chris Angelico ros...@gmail.com wrote: Hi! I'm hoping it was oversight that led to this email coming to me personally instead of to the list, and hoping that you won't mind me responding on-list. Sorry about that. I keep forgetting that gmail web mail is stupid and does not use the list address on Reply. The “reply” command is for replying only to the article author So that is working correctly. Does not happen to other list, only this one. Many mailing lists are misconfigured, breaking the “reply” command so it doesn't do what it's meant to (and you lose the facility to reply only to the author of the message). Instead, for replying to a list, you need the “reply to list” command URL:http://david.woodhou.se/reply-to-list.html. Sadly, it appears GMail does not have this command; you may want to use a better mail client which implements this feature. -- \ “I must say that I find television very educational. The minute | `\ somebody turns it on, I go to the library and read a book.” | _o__)—Groucho Marx | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On Tuesday, January 28, 2014 2:50:20 PM UTC+5:30, Igor Korot wrote: Hi, Chris, On Tue, Jan 28, 2014 at 1:00 AM, Chris Angelico wrote: On Tue, Jan 28, 2014 at 7:56 PM, Igor Korot wrote: Hi, Chris, Hi! I'm hoping it was oversight that led to this email coming to me personally instead of to the list, and hoping that you won't mind me responding on-list. Sorry about that. I keep forgetting that gmail web mail is stupid and does not use the list address on Reply. Does not happen to other list, only this one. In gmail: Gear-icon - Settings - Default-reply behavior select all -- https://mail.python.org/mailman/listinfo/python-list
Re: uninstalling python27 killed vim (actual issue was more complicated and involves Mercurial)
On Tue, 28 Jan 2014 06:13:06 -0500, Peter wrote: I'm posting this information to help others who are transitioning from Python 2.x to Python 3.x and are using Vim and Mercurial on Windows. [...] Thank you Peter for posting your experiences here! We need more of these sorts of informational posts. Regards, -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On Wed, Jan 29, 2014 at 3:06 AM, Rustom Mody rustompm...@gmail.com wrote: Sorry about that. I keep forgetting that gmail web mail is stupid and does not use the list address on Reply. Does not happen to other list, only this one. In gmail: Gear-icon - Settings - Default-reply behavior select all That's an imperfect solution, but it's the one I use. You then have to remember to delete the individual sender's address, or that person will get a duplicate copy. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
ANN: eGenix pyOpenSSL Distribution 0.13.3.1.0.1.6
ANNOUNCING eGenix.com pyOpenSSL Distribution Version 0.13.3.1.0.1.6 An easy-to-install and easy-to-use distribution of the pyOpenSSL Python interface for OpenSSL - available for Windows, Mac OS X and Unix platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.3.1.0.1.6.html INTRODUCTION The eGenix.com pyOpenSSL Distribution includes everything you need to get started with SSL in Python. It comes with an easy-to-use installer that includes the most recent OpenSSL library versions in pre-compiled form, making your application independent of OS provided OpenSSL libraries: http://www.egenix.com/products/python/pyOpenSSL/ pyOpenSSL is an open-source Python add-on that allows writing SSL/TLS- aware network applications as well as certificate management tools: https://launchpad.net/pyopenssl/ OpenSSL is an open-source implementation of the SSL/TLS protocol: http://www.openssl.org/ NEWS This new release of the eGenix.com pyOpenSSL Distribution updates the included pyOpenSSL and OpenSSL versions: New in the eGenix pyOpenSSL Distribution * Updated pyOpenSSL to the upstream trunk revision 171 (pyOpenSSL version 0.13.1+). * Added work-around for compiling pyOpenSSL trunk revision 171 on Windows with OpenSSL 1.0.0 and later. * Included support for TLS 1.1 and 1.2 in pyOpenSSL (rev 171). Please see the TLS support section in the documentation for details. http://www.egenix.com/products/python/pyOpenSSL/doc/#TLS_support * Added SSL.OP_NO_COMPRESSION and SSL.OP_SINGLE_ECDH_USE context options to be able to address the CRIME attack and allow for more secure elliptic curve Diffie-Hellman key exchange setups. * Added HTML Sphinx documentation from the pyOpenSSL trunk version to the package. An online version is available from our website: http://www.egenix.com/products/python/pyOpenSSL/doc/pyopenssl.html * Updated the included CA bundles to the latest Mozilla 2014-01-28 version. * Included ca-bundle*.crt files now have the same modification date as the Mozilla certdata.txt file from which they were generated. * Restored compatibility of the ca_bundle module with Python 2.4. * Enhanced the included https_client.py example to show case OpenSSL best practices: - server name parsing (RFC 2818 support will follow in one of the next releases) - SNI (support for TLS extension to support multiple SSL sites on a single host) - setup secure default SSL options - setup secure default SSL cipher suite - use TLS 1.0 - 1.2 only - disable SSL compression negotiation (prevent CRIME attack) New in OpenSSL -- * Updated included OpenSSL libraries from OpenSSL 1.0.1e to 1.0.1f. See http://www.openssl.org/news/news.html and http://www.openssl.org/news/vulnerabilities.html for a complete list of changes, most important: - CVE-2013-4353: A carefully crafted invalid TLS handshake could crash OpenSSL with a NULL pointer exception. A malicious server could use this flaw to crash a connecting client. - CVE-2013-6450: A flaw in DTLS handling can cause an application using OpenSSL and DTLS to crash. - CVE-2013-6449: A flaw in OpenSSL can cause an application using OpenSSL to crash when using TLS version 1.2. As always, we provide binaries that include both pyOpenSSL and the necessary OpenSSL libraries for all supported platforms: Windows x86 and x64, Linux x86 and x64, Mac OS X PPC, x86 and x64. We've also added egg-file distribution versions of our eGenix.com pyOpenSSL Distribution for Windows, Linux and Mac OS X to the available download options. These make setups using e.g. zc.buildout and other egg-file based installers a lot easier. DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/pyOpenSSL/ UPGRADING Before installing this version of pyOpenSSL, please make sure that you uninstall any previously installed pyOpenSSL version. Otherwise, you could end up not using the included OpenSSL libs. ___ SUPPORT Commercial support for these packages is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. MORE
Re: Wikipedia XML Dump
On Tuesday, January 28, 2014 5:15:32 PM UTC+5:30, Kevin Glover wrote: Hi I have downloaded and unzipped the xml dump of Wikipedia (40+GB). I want to use Python and the SAX module (running under Windows 7) to carry out off-line phrase-searches of Wikipedia and to return a count of the number of hits for each search. Typical phrase-searches might be of the dog and dog's. I have some limited prior programming experience (from many years ago) and I am currently learning Python from a course of YouTube tutorials. Before I get much further, I wanted to ask: Is what I am trying to do actually feasible? Cant really visualize what youve got... When you 'download' wikipedia what do you get? One 40GB file? A zillion files? Some other database format? Another point: sax is painful to use compared to full lxml (dom) But then sax is the only choice when files cross a certain size Thats why the above question Also you may want to explore nltk -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia XML Dump
Another point: sax is painful to use compared to full lxml (dom) But then sax is the only choice when files cross a certain size Thats why the above question No matter what the choice of XML parser, I suspect you'll want to convert it to some other form for processing. Skip -- https://mail.python.org/mailman/listinfo/python-list
boost-python: exposing constructor with an array of other class as argument
Hello there,
I have two different classes that I want to expose using boost-python,
but the constructor of the second class takes and array of the first
one as argument and I can't figure out how to do it.
This is the definition of the classes:
class INT96{
public:
uint64_t value[3];
INT96(){};
INT96(uint64_t x0, uint64_t x1, uint64_t x2);
...
};
template unsigned k
class Xi_CW{
protected:
INT96 A[k];
public:
Xi_CW(INT96 (A)[k]);
...
};
And my attempt to expose them using boost-python:
using namespace boost::python;
typedef Xi_CW4 Xi_CW4;
BOOST_PYTHON_MODULE(xis)
{
class_INT96(INT96, initdouble,double,double())
[...]
;
class_Xi_CW4(Xi_CW4, initINT96[4]())
[...]
;
}
Which results in a no known conversion error. I've tried several
other possibilities but so far no luck...
Any idea how should I do it? Thanks
Ester
--
https://mail.python.org/mailman/listinfo/python-list
Documenting descriptors
I am documenting a few classes with Sphinx that utilize methods decorated with custom descriptors. These properties return data when called and Sphinx is content with a :returns: and :rtype: markup in the properties doc string. They also accept input, but parameter (not really applicable) nor var markup is accepted as valid? Anyone know the trick here? Thanks, jlc -- https://mail.python.org/mailman/listinfo/python-list
Re: Highlighting program variables instead of keywords?
On 1/28/14 2:19 AM, wxjmfa...@gmail.com wrote: Different, but a little bit related. The work which is done actually on the possibility (not implemented but alreay realized) to colorize (style) the different graphemes of a glyph is very interesting. Python with its absurd Flexible String Representation just become a no go for the kind of task. (Should not be too complicate to understand.) jmf JMF, seriously, stop it. You've convinced no one because you have no convincing arguments. It's obnoxious to continue to make this claim. Stop it. Please. If you want to try to convince someone, convince me. Write to me offline: n...@nedbatchelder.com -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: ANN: eGenix pyOpenSSL Distribution 0.13.3.1.0.1.6
On Tue, 28 Jan 2014 17:47:39 +0100, eGenix Team: M.-A. Lemburg wrote: It comes with an easy-to-use installer that includes the most recent OpenSSL library versions in pre-compiled form Hmm, well it all sounds very good, but how would I know that the pre- compiled library doesn't contain a backdoor that sends copies of all the plaintext to the NSA? -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
PyDev 3.3.3 Released
Hi All, PyDev 3.3.3 has been released Details on PyDev: http://pydev.org Details on its development: http://pydev.blogspot.com LiClipse (PyDev standalone with goodies such as support for Django Templates, Kivy Language, Mako Templates, Html, Javascript, etc): http://brainwy.github.io/liclipse/ What is PyDev? --- PyDev is a plugin that enables users to use Eclipse for Python, Jython and IronPython development -- making Eclipse a first class Python IDE -- It comes with many goodies such as code completion, syntax highlighting, syntax analysis, code analysis, refactor, debug, etc. Release Highlights: --- * **Important**: PyDev requires Eclipse 3.8 or 4.3 onwards and Java 7! For older versions, keep using PyDev 2.x (see LiClipse: http://brainwy.github.io/liclipse for a PyDev standalone with all requirements bundled). * **Code Completion**: - Compiled modules are now indexed and shown in the context-insensitive code-completion. - In an empty file, a code-completion request will show options related to creating modules (press Ctrl+Space twice to show only those templates). * **Performance**: - Building (indexing) of Python files is **much** faster. - Code completion does not get slown down by other analysis done in the background due to shell synchronization. * **Interactive Console**: - The interactive console now has tab-completion (so, tab can be used to show completions such as in IPython). * **Debugger**: - **Locals are now properly changed in the debugger** -- along with set next statement and auto-reloading this can make a debug session much more enjoyable! - Added a way to skip functions on a step-in on functions with **#@DontTrace** comments: - **Makes it possible to skip a lot of boilerplate code on a debug session!** - Can be enabled/disabled in the debugger preferences; - Ctrl+1 in a line with a method shows option to add **#@DontTrace** comment (if enabled in the preferences). - Debugging Stackless is much improved, especially for versions of Stackless released from 2014 onwards (special thanks to Anselm Kruis who improved stackless itself for this integration to work properly). - Reload during a debug session is improved and more stable: - Only updates what it can in-place or adds new attributes; - Shows what's being patched in the console output; - New hooks are provided for clients which may want to extend the reload; - See: Auto Reload in Debugger: http://pydev.org/manual_adv_debugger_auto_reload.html for more details. * **General**: - Compiled modules are now indexed, so, **fix import with Ctrl+1 now works with itertools, PyQt and other 'forced builtins'**. - When diffing a Python file, the PyDev comparison (with proper syntax highlighting) is now the default. - When finding a definition in a .pyd file, if there's a related .pyx in the same location, it's opened. - Running unit-tests will not try to import files that are in folders that don't have an __init__.py file. - Alt+Shift+O can be used to toggle mark occurrences. - Ctrl+3 not bound by default anymore on PyDev so that it does not conflict with the Eclipse Ctrl+3 (Ctrl+/ can be used instead). - Fixed recursion issue when finding file in pydev package explorer. - When configuring the interpreter, links are not followed when resolving entries for the PYTHONPATH. - It's possible to launch a directory containing a __main__.py file executable. - Fixed issues when creating django project without any existing project in the workspace. - Fixed deadlock on code-completion. - __pycache__ folders are hidden by default. * **Organize imports**: - When saving a file, if automatically organizing imports, don't remove unused imports even if that option is checked. - When saving a file, if automatically organizing imports, and nothing changes, don't change the buffer (so, no undo command is created). - @NoMove can be used in an import so that the import organizer doesn't mess with it. * **Refactoring**: - Fixed error when moving resource in PYTHONPATH to a dir out of the PYTHONPATH. - On a search make sure we search only python files, not dlls (which could give OutOfMemory errors and make the search considerably slower). - Multiple fixes on the rename module refactoring. Cheers, -- Fabio Zadrozny -- Software Developer LiClipse http://brainwy.github.io/liclipse PyDev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com -- https://mail.python.org/mailman/listinfo/python-list
Pyro4 - reading files
Hello there. I am currently working on a project involving the use of Pyro4. I have a scenario. We have the pc named A, and a pc named B. On pc B lies a python script, that includes pyro, and a method for reading files. On pc A, we create an instance to the pyro object on pc B. And we call the method for reading files. I want to read a file that lies on pc B, from pc A using the method of the pyro object i just created. Is it possible? Thank you. -- https://mail.python.org/mailman/listinfo/python-list
Re: Highlighting program variables instead of keywords?
On 01/28/2014 12:38 PM, Ned Batchelder wrote: JMF, seriously, stop it. You've convinced no one because you have no convincing arguments. It's obnoxious to continue to make this claim. Stop it. Please. If you want to try to convince someone, convince me. Write to me offline: n...@nedbatchelder.com JMF, maybe if you'd actually try to write a program in Python that does what you are talking about with colorizing graphemes, (whatever that means) then you can talk. Sounds to me like you don't even use Python at all, for unicode or anything else. -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia XML Dump
Thanks for the comments, guys. The Wikipedia download is a single XML document, 43.1GB. Any further thoughts? Kevin -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia XML Dump
hi, On 01/29/14 00:31, Kevin Glover wrote: Thanks for the comments, guys. The Wikipedia download is a single XML document, 43.1GB. Any further thoughts? in that case, http://lxml.de/tutorial.html#event-driven-parsing seems to be your only option. hth, burak -- https://mail.python.org/mailman/listinfo/python-list
Development infrastructure - need python packaging gurus help, please
Hi! I'm trying to create a development infrastructure that would allow for simple and unified ways of sharing, *deploying* and *reusing* the code within private entity. I can see that pip with virtual environments and requirements.txt is very similar to dependency management provided by maven or apache ivy. But there seems to be a disconnect between the egg carrying the possibility to be importable and executable, but in the same time considered to be deprecated format which is not fully supported by pip and virtualenv and wheel not having those basic questions answered... Here is what i'm trying to achieve: 1. I want to be able to specify the set of dependencies for the project i'm currently developing and make them available for the import. Think java jar - having it in class path allows for the code reuse (import packages provided) as well as for the execution. 2. I want to be able to put the newly created project in a form of artifact in a some central location from which it can be taken to be imported or executed or included into class path. Again think java jar. If i have it in some location i can use ivy or maven to manage the dependencies and provide it to me upon request 3. I want to be able to deploy the program and execute it. So the problem with all this as i see it: The importable and executable format is egg, but it is deprecated! Sane dependency management is pip and virtualenv, but they don't support eggs fully (pip install --egg is having the disclaimer that it s not working ok sometimes) wheel is pretty much unusable for anything other then installing into the virtualenv. Am i missing something? Thanks in advance, Eugene -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia XML Dump
On 28/01/2014 9:45 PM, kevinglove...@gmail.com wrote: I have downloaded and unzipped the xml dump of Wikipedia (40+GB). I want to use Python and the SAX module (running under Windows 7) to carry out off-line phrase-searches of Wikipedia and to return a count of the number of hits for each search. Typical phrase-searches might be of the dog and dog's. I have some limited prior programming experience (from many years ago) and I am currently learning Python from a course of YouTube tutorials. Before I get much further, I wanted to ask: Is what I am trying to do actually feasible? Rather than parsing through 40GB+ every time you need to do a search, you should get better performance using an XML database which will allow you to do queries directly on the xml data. http://basex.org/ is one such db, and comes with a Python API: http://docs.basex.org/wiki/Clients -- https://mail.python.org/mailman/listinfo/python-list
Re: Wikipedia XML Dump
On Wednesday, January 29, 2014 4:17:47 AM UTC+5:30, Burak Arslan wrote: hi, On 01/29/14 00:31, Kevin Glover wrote: Thanks for the comments, guys. The Wikipedia download is a single XML document, 43.1GB. Any further thoughts? in that case, http://lxml.de/tutorial.html#event-driven-parsing seems to be your only option. Further thoughts?? Just a combo of what Burak and Skip said: I'd explore a thin veneer of even-driven lxml to get from 40 GB monolithic xml to something (more) digestible to nltk -- https://mail.python.org/mailman/listinfo/python-list
Re: Experiences/guidance on teaching Python as a first programming language
In article 20131216213225.2006b30246e3a08ee241a...@gmx.net, Wolfgang Keller felip...@gmx.net wrote: And ever after that experience, I avoided all languages that were even remotely similar to C, such as C++, Java, C#, Javascript, PHP etc. I think that's disappointing, for two reasons. Firstly, C syntax isn't that terrible. It's not just the abysmally appalling, hideously horrifying syntax. At about everything about C is just *not* made for human beings imho. It's just an un-language that gets at about everything wrong. Sort of like Microsoft's products. Sincerely, Wolfgang I don't see how you could create a better high-level LOW-LEVEL language. And that pointer * syntax is really ingenious. (After all, the guys who created it and those who first used it (at Bell Labs) WERE all geniuses!) David -- https://mail.python.org/mailman/listinfo/python-list
Re: Experiences/guidance on teaching Python as a first programming language
In article mailman.4286.1387291924.18130.python-l...@python.org, Neil Cerutti ne...@norwich.edu wrote: On 2013-12-17, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: I would really like to see good quality statistics about bugs per program written in different languages. I expect that, for all we like to make fun of COBOL, it probably has few bugs per unit-of-useful-work-done than the equivalent written in C. I can't think of a reference, but I to recall that bugs-per-line-of-code is nearly constant; it is not language dependent. So, unscientifically, the more work you can get done in a line of code, then the fewer bugs you'll have per amount of work done. -- Neil Cerutti Makes no sense to me. I can't imagine that errors per 100 lines is anywhere near as high with a language that has garbage collection and type checking as with one that has neither. David -- https://mail.python.org/mailman/listinfo/python-list
Re: Enable unicode
On 29Jan2014 02:47, Ben Finney ben+pyt...@benfinney.id.au wrote: Igor Korot ikoro...@gmail.com writes: On Tue, Jan 28, 2014 at 1:00 AM, Chris Angelico ros...@gmail.com wrote: Hi! I'm hoping it was oversight that led to this email coming to me personally instead of to the list, and hoping that you won't mind me responding on-list. Sorry about that. I keep forgetting that gmail web mail is stupid and does not use the list address on Reply. [...] Instead, for replying to a list, you need the “reply to list” command URL:http://david.woodhou.se/reply-to-list.html. Sadly, it appears GMail does not have this command; you may want to use a better mail client which implements this feature. For example, choose and use a real desktop mail client and connect to GMail's IMAP interface. At least then you can pick something with the feature set you want. -- Cameron Simpson c...@zip.com.au -- https://mail.python.org/mailman/listinfo/python-list
Large Two Dimensional Array
Hello, I am trying to implement IBM Model 1. In that I need to create a matrix of 5*5 with double values. Currently I am using dict of dict but it is unable to support such high dimensions and hence gives memory error. Any help in this regard will be useful. I understand that I cannot store the matrix in the RAM but what is the most efficient way to do this? -- https://mail.python.org/mailman/listinfo/python-list
Re: Large Two Dimensional Array
On Wednesday, January 29, 2014 10:55:54 AM UTC+5:30, Ayushi Dalmia wrote: Hello, I am trying to implement IBM Model 1. In that I need to create a matrix of 5*5 with double values. Currently I am using dict of dict but it is unable to support such high dimensions and hence gives memory error. Any help in this regard will be useful. I understand that I cannot store the matrix in the RAM but what is the most efficient way to do this? Also, the matrix indices are words and not integers. I donot want to map. -- https://mail.python.org/mailman/listinfo/python-list
Eclipse IDE printing values automatically
I am using Pydev 2.8 on Eclipse IDE. It is printing some values that haven't been printed with print command. How to deal with this problem? -- https://mail.python.org/mailman/listinfo/python-list
Re: Large Two Dimensional Array
Quoting Ayushi Dalmia (2014-01-29 06:25:54) Hello, I am trying to implement IBM Model 1. In that I need to create a matrix of 5*5 with double values. Currently I am using dict of dict but it is unable to support such high dimensions and hence gives memory error. Any help in this regard will be useful. I understand that I cannot store the matrix in the RAM but what is the most efficient way to do this? -- https://mail.python.org/mailman/listinfo/python-list Hello, I would suggest using h5py [1] or PyTables [2] to store data on disk (both are based on HDF5 [3]), and manipulate data in RAM as NumPy [4] arrays. [1] www.h5py.org [2] www.pytables.org [3] www.hdfgroup.org/HDF5 [4] www.numpy.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Development infrastructure - need python packaging gurus help, please
Eugene Sajine eugu...@gmail.com writes: ... Here is what i'm trying to achieve: 1. I want to be able to specify the set of dependencies for the project i'm currently developing and make them available for the import. Think java jar - having it in class path allows for the code reuse (import packages provided) as well as for the execution. 2. I want to be able to put the newly created project in a form of artifact in a some central location from which it can be taken to be imported or executed or included into class path. Again think java jar. If i have it in some location i can use ivy or maven to manage the dependencies and provide it to me upon request 3. I want to be able to deploy the program and execute it. You may have a look at zc.buildout - it may support your development needs. Once the development is complete, a so called meta-egg may be used to describe the finished program. A meta-egg does not have (significant amounts of) code but essentially states dependencies of other eggs. -- https://mail.python.org/mailman/listinfo/python-list
Re: Large Two Dimensional Array
On Wednesday, January 29, 2014 10:55:54 AM UTC+5:30, Ayushi Dalmia wrote: Hello, I am trying to implement IBM Model 1. In that I need to create a matrix of 5*5 with double values. Currently I am using dict of dict but it is unable to support such high dimensions and hence gives memory error. Any help in this regard will be useful. I understand that I cannot store the matrix in the RAM but what is the most efficient way to do this? Thanks David! -- https://mail.python.org/mailman/listinfo/python-list
[issue17162] Py_LIMITED_API needs a PyType_GenericDealloc
Martin v. Löwis added the comment: I propose a more general solution: add a function PyType_GetSlot. -- keywords: +patch Added file: http://bugs.python.org/file33762/getslot.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17162 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19081] zipimport behaves badly when the zip file changes while the process is running
Gregory P. Smith added the comment: Fixed in the 2.7 branch. The existing patch in the 3.3 and 3.4 branches is incomplete and does not actually fix the problem. Despite what the Misc/NEWS entry claims. The patch I am attaching now (applies to 3.3, I will forward port it to 3.4) fixes the remaining issue with zipimport failing subimports when the zip file has changed while the process is running. Marking as a release blocker as one of the following needs to happen for both 3.3 and 3.4 prior to their final release: A) The -gps05 patch needs to be applied (and forward ported to 3.4); I can do that. B) The Misc/NEWS entry claiming that this issue is fixed should simply be removed from the Misc/NEWS file and the releases should happen without this patch. After 3.4.0 and 3.3.4 I will commit this patch and re-add the Misc/NEWS entry under the 3.4.1 and 3.3.5 sections. Release managers for 3.3 and 3.4, please chime in. (+nosy'd) -- nosy: +georg.brandl, larry priority: normal - release blocker versions: -Python 2.7 Added file: http://bugs.python.org/file33763/issue19081-subimport-fixes-py33-gps05.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19081 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Larry Hastings added the comment: Attached is a second patch. * Now includes input and output checksums. Checksums are now truncated to 16 characters each, otherwise the line is 80 columns. * Fixes the doubled-up signature lines for type object slot default signatures. I ran gcc -E and wrote a quick script to print out all lines with doubled-up signatures. There were only two: divmod and rdivmod. * Pretty sure this was in the first patch, but just thought I'd mention it: for functions using optional groups, we can't generate a legal signature. So Clinic kicks out the name of the function instead of sig=, meaning that it puts back the docstring first line for human consumption! I am so clever, tee hee. -- keywords: +patch Added file: http://bugs.python.org/file33764/larry.sig=.marker.for.signatures.diff.2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19081] zipimport behaves badly when the zip file changes while the process is running
Roundup Robot added the comment: New changeset ca5431a434d6 by Gregory P. Smith in branch '2.7': Remove unneeded use of globals() and locals() in test on imports http://hg.python.org/cpython/rev/ca5431a434d6 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19081 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20414] Python 3.4 has two Overlapped types
STINNER Victor added the comment:
One sub issue then is naming: _overlapped renamed
Overlapped.GetOverlappedResult to Overlapped.getresult. I think the _winapi
name is better, since all other methods also use Windows API function names.
I also prefer Overlapped.GetOverlappedResult name.
There are other differences. Using _overlapped, the overlapped object is
created before calling an operation like WriteFile() or AcceptEx() (ex:
ov.WriteFile(...)). Using _winapi, the operation may create an overlapped
object if asked (ex: WriteFile(..., overlapped=True)).
The structure is different.
_winapi.c:
---
typedef struct {
PyObject_HEAD
OVERLAPPED overlapped;
/* For convenience, we store the file handle too */
HANDLE handle;
/* Whether there's I/O in flight */
int pending;
/* Whether I/O completed successfully */
int completed;
/* Buffer used for reading (optional) */
PyObject *read_buffer;
/* Buffer used for writing (optional) */
Py_buffer write_buffer;
} OverlappedObject;
---
overlapped.c:
---
enum {TYPE_NONE, TYPE_NOT_STARTED, TYPE_READ, TYPE_WRITE, TYPE_ACCEPT,
TYPE_CONNECT, TYPE_DISCONNECT, TYPE_CONNECT_NAMED_PIPE,
TYPE_WAIT_NAMED_PIPE_AND_CONNECT};
typedef struct {
PyObject_HEAD
OVERLAPPED overlapped;
/* For convenience, we store the file handle too */
HANDLE handle;
/* Error returned by last method call */
DWORD error;
/* Type of operation */
DWORD type;
union {
/* Buffer used for reading (optional) */
PyObject *read_buffer;
/* Buffer used for writing (optional) */
Py_buffer write_buffer;
};
} OverlappedObject;
---
And the object in overlapped.c has much more methods.
_winapi.c:
---
static PyMethodDef overlapped_methods[] = {
{GetOverlappedResult, (PyCFunction) overlapped_GetOverlappedResult,
METH_O, NULL},
{getbuffer, (PyCFunction) overlapped_getbuffer, METH_NOARGS, NULL},
{cancel, (PyCFunction) overlapped_cancel, METH_NOARGS, NULL},
{NULL}
};
---
overlapped.c:
---
static PyMethodDef Overlapped_methods[] = {
{getresult, (PyCFunction) Overlapped_getresult,
METH_VARARGS, Overlapped_getresult_doc},
{cancel, (PyCFunction) Overlapped_cancel,
METH_NOARGS, Overlapped_cancel_doc},
{ReadFile, (PyCFunction) Overlapped_ReadFile,
METH_VARARGS, Overlapped_ReadFile_doc},
{WSARecv, (PyCFunction) Overlapped_WSARecv,
METH_VARARGS, Overlapped_WSARecv_doc},
{WriteFile, (PyCFunction) Overlapped_WriteFile,
METH_VARARGS, Overlapped_WriteFile_doc},
{WSASend, (PyCFunction) Overlapped_WSASend,
METH_VARARGS, Overlapped_WSASend_doc},
{AcceptEx, (PyCFunction) Overlapped_AcceptEx,
METH_VARARGS, Overlapped_AcceptEx_doc},
{ConnectEx, (PyCFunction) Overlapped_ConnectEx,
METH_VARARGS, Overlapped_ConnectEx_doc},
{DisconnectEx, (PyCFunction) Overlapped_DisconnectEx,
METH_VARARGS, Overlapped_DisconnectEx_doc},
{ConnectNamedPipe, (PyCFunction) Overlapped_ConnectNamedPipe,
METH_VARARGS, Overlapped_ConnectNamedPipe_doc},
{WaitNamedPipeAndConnect,
(PyCFunction) Overlapped_WaitNamedPipeAndConnect,
METH_VARARGS, Overlapped_WaitNamedPipeAndConnect_doc},
{NULL}
};
---
_winapi.c doesn't provide AcceptEx() nor Accept().
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20414
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20264] Update patchcheck to looks for files with clinic comments
Larry Hastings added the comment: I've attached a script here that uses the new tweaked format of Clinic blocks. The new tweaked format isn't checked in yet--that change is being tracked with #20326. Once that's checked in, though, the attached script will check that both the input and output blocks are unchanged and up-to-date, respectively. -- dependencies: +Argument Clinic should use a non-error-prone syntax to mark text signatures Added file: http://bugs.python.org/file33765/detect.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20264 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17162] Py_LIMITED_API needs a PyType_GenericDealloc
STINNER Victor added the comment: +return *(void**)(((char*)type) + slotoffsets[slot]); New Python versions may add new slots. What do you think of returning NULL if the slot number is higher than the maximum slot? It looks like #define Py_tp_free 74 is the highest slot number since Python 3.2. For example, Python 3.4 has a new tp_finalize slot, but I don't see it in typeslots.h. -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17162 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20414] Python 3.4 has two Overlapped types
Richard Oudkerk added the comment: _overlapped is linked against the socket library whereas _winapi is not so it can be bundled in with python3.dll. I did intend to switch multiprocessing over to using _overlapped but I did not get round to it. Since this is a private module the names of methods do not matter to much. Note that getresult and GetOverlappedResult return values in different forms. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20414 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20414] Python 3.4 has two Overlapped types
STINNER Victor added the comment: I did intend to switch multiprocessing over to using _overlapped but I did not get round to it. Do you mean that _overlapped module is newer and should be used instead of _winapi? If multiprocssing is patched to use _overlapped, we can drop overlapped code from _winapi, should we keep functions like WriteFile() without overlapped support? (I think that we should keep these functions, it was discussed to support the native Windows API for files.) IMO such change can be done in Python 3.5, it is risky and can wait. But until that, I'm concerned by overlapped deallocator which is different in the two modules. Attached fixes _overlapped module to use the same logic than _winapi: give up on Windows XP during Python finalization if the overlapped is still pending, don't deallocate memory, exit immediatly. See issue #19565 for the rationale of this change. -- keywords: +patch Added file: http://bugs.python.org/file33766/overlapped_dealloc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20414 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files
Vajrasky Kok added the comment: Here is the updated patch for list module based on Zachary and Serhiy's reviews. -- Added file: http://bugs.python.org/file33767/clinic_listobject_v4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20185 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files
Vajrasky Kok added the comment: Forgot to say that, in list module, anything is convertable except __getitem__. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20185 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Larry Hastings added the comment: I'm surprised it made a review link. It didn't apply cleanly for me here. While merging I noticed that the imperative declension fix had snuck out of the diff somehow. So I redid that. Attached is an updated patch. Also I should mention: clinic.py currently accepts both the old and new comment format. I'll leave support for the old one in until just before the last release candidate. -- Added file: http://bugs.python.org/file33768/larry.sig=.marker.for.signatures.diff.3.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20415] Could method isinstance take a list as parameter?
New submission from Chen ZHANG: Since the usage of isinstance could be isinstance(1, (int, float)), I'm wondering why it doesn't accept [int, float], if I do so, it would raise TypeError saying arg 2 must be a type or tuple of types. What's the difference in effect between a tuple and a list here? (I know a tuple is immutable, but I think it doesn't matter when passing some other types of non-string iterables(yielding strings), am I right? -- components: Library (Lib) messages: 209519 nosy: Chen.ZHANG priority: normal severity: normal status: open title: Could method isinstance take a list as parameter? type: enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20415 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files
Vajrasky Kok added the comment: Here is the updated patch for marshal module based on Zachary's review. -- Added file: http://bugs.python.org/file33769/clinic_marshal_v4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20185 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20308] inspect.Signature doesn't support user classes without __init__ or __new__
Stefan Krah added the comment: One test fails --without-doc-strings: http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%203.x/builds/6266/steps/test/logs/stdio -- nosy: +skrah ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20308 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20415] Could method isinstance take a list as parameter?
Mark Dickinson added the comment: See related discussion on python-ideas here: https://mail.python.org/pipermail/python-ideas/2011-July/010610.html -- nosy: +mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20415 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20338] Idle: increase max calltip width
Stefan Krah added the comment: I think test_idle is failing on many build slaves following this commit. -- nosy: +skrah ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20338 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
New submission from STINNER Victor: Attached patched disables references for int and float types. -- files: marshal3_numbers.patch keywords: patch messages: 209524 nosy: haypo priority: normal severity: normal status: open title: Marshal: special case int and float, don't use references type: performance versions: Python 3.4 Added file: http://bugs.python.org/file33770/marshal3_numbers.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
STINNER Victor added the comment: Use attached bench.py to compare performances. Without the patch: --- dumps v0: 389.6 ms data size v0: 45582.9 kB loads v0: 573.3 ms dumps v1: 391.4 ms data size v1: 45582.9 kB loads v1: 558.0 ms dumps v2: 166.9 ms data size v2: 41395.4 kB loads v2: 482.2 ms dumps v3: 431.2 ms data size v3: 41395.4 kB loads v3: 543.8 ms dumps v4: 361.8 ms data size v4: 37000.9 kB loads v4: 560.4 ms --- With the patch: --- dumps v0: 391.4 ms data size v0: 45582.9 kB loads v0: 578.2 ms dumps v1: 392.3 ms data size v1: 45582.9 kB loads v1: 556.8 ms dumps v2: 167.7 ms data size v2: 41395.4 kB loads v2: 484.6 ms dumps v3: 170.3 ms data size v3: 41395.4 kB loads v3: 467.0 ms dumps v4: 122.8 ms data size v4: 37000.9 kB loads v4: 468.9 ms --- dumps v3 is 60% faster, loads v3 is also 14% *faster*. dumps v4 is 66% faster, loads v4 is 16% faster. -- Added file: http://bugs.python.org/file33771/bench.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
STINNER Victor added the comment: Performance of Python 3.3.3+: --- dumps v0: 374.8 ms data size v0: 45582.9 kB loads v0: 625.3 ms dumps v1: 374.6 ms data size v1: 45582.9 kB loads v1: 605.1 ms dumps v2: 152.9 ms data size v2: 41395.4 kB loads v2: 556.5 ms --- So with the patch, the Python 3.4 default version (4) is *faster* (dump 20% faster, load 16% faster) and produces *smaller files* (10% smaller). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
STINNER Victor added the comment: Oh by the way, on Python 3.4, the file size (on version 3 and 4) is unchanged with my patch. Writing a reference produces takes exactly the same size than an integer. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20166] window x64 c-extensions not works on python3.4.0b2
Stefan Krah added the comment: Ping. The blocker seems to have passed beta3. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20166 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19081] zipimport behaves badly when the zip file changes while the process is running
Georg Brandl added the comment: Since this is a pretty big code churn, I'd prefer B) for 3.3.4. (3.3.5 will be soon anyway.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19081 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Vajrasky Kok added the comment: I am doing clinic conversion for marshal module so I am adding myself to nosy list to make sure both tickets are synchronized. http://bugs.python.org/issue20185 -- nosy: +vajrasky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
STINNER Victor added the comment: I am doing clinic conversion for marshal module so I am adding myself to nosy list to make sure both tickets are synchronized. The Derby is suspended until the release of Python 3.4 final. I consider this issue as an important performance regression that should be fixed before Python 3.4 final. -- nosy: +larry priority: normal - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +kristjan.jonsson, pitrou, serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Serhiy Storchaka added the comment: Did you tested for numerous shared int and floats? [1000] * 100 and [1000.0] * 100? AFAIK this was important use cases for adding 3 or 4 versions. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
STINNER Victor added the comment: Did you tested for numerous shared int and floats? [1000] * 100 and [1000.0] * 100? AFAIK this was important use cases for adding 3 or 4 versions. Here are new benchmarks on Python 3.4 with: Integers: [1000] * 100 Floats: [1000.0] * 100 Integers, without the patch: dumps v3: 62.8 ms data size v3: 4882.8 kB loads v3: 10.7 ms Integers, with the patch: dumps v3: 18.6 ms (-70%) data size v3: 4882.8 kB (same size) loads v3: 27.7 ms (+158%) Floats, without the patch: dumps v3: 62.5 ms data size v3: 4882.8 kB loads v3: 11.0 ms Floats, with the patch: dumps v3: 29.3 ms (-53%) data size v3: 8789.1 kB (+80%) loads v3: 25.5 ms (+132%) The version 3 was added by: --- changeset: 82816:01372117a5b4 user:Kristján Valur Jónsson swesk...@gmail.com date:Tue Mar 19 18:02:10 2013 -0700 files: Doc/library/marshal.rst Include/marshal.h Lib/test/test_marshal.py Misc/NEWS Python/marshal.c description: Issue #16475: Support object instancing, recursion and interned strings in marshal --- This issue tells about sharing string constants, common tuples, even common code objects, not sharing numbers. For real data, here are interesting numbers: http://bugs.python.org/issue16475#msg176013 Integers only represent 4.8% of serialized data, and only 8.2% of these integers can be shared. (Floats represent 0.29%.) Whereas strings repsent 58% and 57% can be shared. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Martin v. Löwis added the comment: For the record, format 3 was added through issue16475, format 4 was added through issue19219. In msg175962, Kristjan argued that there is no reason _not_ to share int objects, e.g. across multiple code objects. Now it seems that this argument is flawed: there is a reason, namely the performance impact. OTOH, I consider both use case (marshaling a large number of integers, and desiring to share ints across code objects) equally obscure: you shouldn't worry about marshal performance too much if you have loads of tiny int objects, and you shouldn't worry whether these ints get shared or not. As a compromise, we could suppress the sharing for small int objects, since they are singletons, anyway. This would allow marshal to preserve/copy the object graph, while not impacting the use case that the original poster on python-dev presented. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20416] Marshal: special case int and float, don't use references
Antoine Pitrou added the comment: Integers, without the patch: dumps v3: 62.8 ms data size v3: 4882.8 kB loads v3: 10.7 ms Integers, with the patch: dumps v3: 18.6 ms (-70%) data size v3: 4882.8 kB (same size) loads v3: 27.7 ms (+158%) As I wrote on python-dev, dumps performance isn't important for the pyc use case, but loads performance is. Therefore it appears this patch goes into the wrong direction. You are also ignoring the *runtime* benefit of sharing objects: smaller memory footprint of the actual Python process. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20417] ensurepip should not be installed with --without-ensurepip
New submission from Arfrever Frehtes Taifersar Arahesis: ensurepip should not be installed when --without-ensurepip was passed to configure. I attach the patch. -- assignee: dstufft components: Installation files: ensurepip_installation.patch keywords: patch messages: 209536 nosy: Arfrever, dstufft priority: normal severity: normal stage: patch review status: open title: ensurepip should not be installed with --without-ensurepip versions: Python 3.4 Added file: http://bugs.python.org/file33772/ensurepip_installation.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20417 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20417] ensurepip should not be installed with --without-ensurepip
Donald Stufft added the comment: I don't see any reason not to install ensurepip in this situation. That flag controls whether or not ``python -m ensurepip`` will be executed during the install, but ensurepip itself will still be installed. It is not an optional module -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20417 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20166] window x64 c-extensions not works on python3.4.0b2
Martin v. Löwis added the comment: I think the change should be reverted, and the original issue closed as won't fix. Alternatively, to fix the original issue, the specific linker warning could be suppressed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20166 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20418] socket.getaddrinfo fails for hostname that is all digits 0-9
New submission from ariel-wikimedia:
With python 2.7.5 (running on fedora 20 with all updates), socket.getaddrinfo
for a hostname such as 836937931829 will fail. Docker produces these sorts of
hostnames (really random hex strings, but some hex strings only contain digits
0-9).
How to reproduce:
Add the lines
172.17.10.53blobber
172.17.10.54836937931829
to /etc/hosts
run the following:
import socket
print socket.getaddrinfo('172.17.10.53',80,socket.AF_INET,0,socket.SOL_TCP)
print socket.getaddrinfo('blobber',80,socket.AF_INET,0,socket.SOL_TCP)
print socket.getaddrinfo('172.17.10.54',80,socket.AF_INET,0,socket.SOL_TCP)
print socket.getaddrinfo('836937931829',80,socket.AF_INET,0,socket.SOL_TCP)
Expected output:
[(2, 1, 6, '', ('172.17.10.53', 80))]
[(2, 1, 6, '', ('172.17.10.53', 80))]
[(2, 1, 6, '', ('172.17.10.54', 80))]
[(2, 1, 6, '', ('172.17.10.54', 80))]
Actual output:
[(2, 1, 6, '', ('172.17.10.53', 80))]
[(2, 1, 6, '', ('172.17.10.53', 80))]
[(2, 1, 6, '', ('172.17.10.54', 80))]
Traceback (most recent call last):
File ./test-getaddrinfo.py, line 6, in module
print socket.getaddrinfo('836937931829',80,socket.AF_INET,0,socket.SOL_TCP)
socket.gaierror: [Errno -2] Name or service not known
--
components: Library (Lib)
messages: 209539
nosy: ariel-wikimedia
priority: normal
severity: normal
status: open
title: socket.getaddrinfo fails for hostname that is all digits 0-9
type: behavior
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20418
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20414] Python 3.4 has two Overlapped types
Martin v. Löwis added the comment: It seems fine to me to link pythonXY.dll with wsock32.dll. This is a standard library in Windows NT+, so there is no need to avoid linking with it. I also now agree that any change that we may make is too big for 3.4, so I propose to defer any action on this issue to 3.5. -- versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20414 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Nick Coghlan added the comment: Looks good to me :) I also like the fact it simplifies the internal APIs by making it really trivial to detect the presence of a clinic signature from C. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Roundup Robot added the comment: New changeset d6311829da15 by Larry Hastings in branch 'default': Issue #20326: Argument Clinic now uses a simple, unique signature to http://hg.python.org/cpython/rev/d6311829da15 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Changes by Larry Hastings la...@hastings.org: -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20326] Argument Clinic should use a non-error-prone syntax to mark text signatures
Larry Hastings added the comment: Yeah. I did a pretty terrible job of articulating why the fn_name( signature was a bad idea in the first place ;-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20338] Idle: increase max calltip width
Terry J. Reedy added the comment: On 1/28/2014 4:48 AM, Stefan Krah wrote: Stefan Krah added the comment: I think test_idle is failing on many build slaves following this commit. The two failures I saw on the first 4 3.x bots with Idle failures are the result of changes in the list docstrings. I presume these are related to use of Argument Clinic. I knew and documented that such failure was a possibility. On my Win 7 machine Jan 20, 32-bit repository build just before I added the tests that now fail. list.__doc__ list() - new empty list\nlist(iterable) - new list initialized from iterable's items list.__init__.__doc__ 'x.__init__(...) initializes x; see help(type(x)) for signature' Jan 26, 64-bit installed 3.4.0b3: list.__doc__ list(iterable) - new list initialized from iterable's items list.__init__.__doc__ 'Initializes self. See help(type(self)) for accurate signature.' At some point, I will switch to using inspect.signature for Idle calltips and simplify the Idle code. I will also simplify the tests and make them more robust. I will re-compile and adjust the tests by the end of the day. Terry -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20338 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18795] pstats - allow stats sorting by cumulative time per call and total time per call
Alexandre Dias added the comment: Could I get an update on this please? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18795 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9709] test_distutils warning: initfunc exported twice on Windows
Roundup Robot added the comment: New changeset 69827c2ab9d0 by Stefan Krah in branch 'default': Issue #9709: Revert 97fb852c5c26. Many extensions are not using PyMODINIT_FUNC. http://hg.python.org/cpython/rev/69827c2ab9d0 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9709 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20411] IndexError in sys.__interactivehook__ with pyreadline installed
Changes by Tshepang Lekhonkhobe tshep...@gmail.com: -- nosy: +tshepang ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20411 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20166] window x64 c-extensions not works on python3.4.0b2
Stefan Krah added the comment: Thanks, that seems to be the best course of action. Fixed in 69827c2ab9d0. -- resolution: - fixed stage: - committed/rejected status: open - closed superseder: - test_distutils warning: initfunc exported twice on Windows type: - compile error ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20166 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9709] test_distutils warning: initfunc exported twice on Windows
Stefan Krah added the comment: Too many extensions are not using PyMODINIT_FUNC (See #20166). Closing as WONT_FIX. -- resolution: fixed - wont fix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9709 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20411] IndexError in sys.__interactivehook__ with pyreadline installed
Roundup Robot added the comment: New changeset ca6efeedfc0e by Jason R. Coombs in branch 'default': Issue #20411: Use readline.get_current_history_length to check for the presence of a history, rather than get_history_item, which assumes a history is present. http://hg.python.org/cpython/rev/ca6efeedfc0e -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20411 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files
Vajrasky Kok added the comment: Here is the updated patch for float object based on Zachary and Serhiy's reviews. Some methods that can not be converted are: __getnewargs__, __round__, float_new. So these files are ready for Python 3.4: resource, typeobject, listobject, and floatobject. These files are not ready yet: gc, longobject. -- Added file: http://bugs.python.org/file33773/clinic_floatobject_v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20185 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20411] IndexError in sys.__interactivehook__ with pyreadline installed
Jason R. Coombs added the comment: After further consideration and investigation, I believe the fix is to simply use the API as exposed by pyreadline to check the length of the history to detect the presence of an existing history. I've tested that fix locally and it seems to be working suitably. Please review and suggest corrections as appropriate. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20411 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20172] Derby #3: Convert 67 sites to Argument Clinic across 4 files (Windows)
Changes by Zachary Ware zachary.w...@gmail.com: Added file: http://bugs.python.org/file33774/bda6dc12a123.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20172 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20172] Derby #3: Convert 67 sites to Argument Clinic across 4 files (Windows)
Changes by Zachary Ware zachary.w...@gmail.com: Added file: http://bugs.python.org/file33775/41ad3c4fc03c.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20172 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20172] Derby #3: Convert 67 sites to Argument Clinic across 4 files (Windows)
Changes by Zachary Ware zachary.w...@gmail.com: Removed file: http://bugs.python.org/file33774/conglomerate.v6-post-20326.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20172 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20414] Python 3.4 has two Overlapped types
Changes by Zachary Ware zachary.w...@gmail.com: -- nosy: +zach.ware ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20414 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18846] python.exe stdout stderr issues again
SSmith added the comment: Please pay some attention to this. This ISSUE is still valid in 3.4b4! Issue #18338 resolves only part of the problem. Look at this part of the OP: invoking python.exe prints its default output to stderr: [ in]python 1 null [out]Python 3.4.0a1 (v3.4.0a1:46535f65e7f3, Aug 3 2013, 22:59:31) [MSC v.1600 32 bit (Intel)] on win32 [out]Type help, copyright, credits or license for more information. [out] [ in]python 2 null [out] -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18846 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
