devpi-0.9.1: pushing tested releases to pypi
Hi all, devpi-0.9.1 is out which fixes bugs and introduces support for pushing a tested release candidate from a private index to pypi. See http://doc.devpi.net on the ease of doing devpi upload, test and push commands as well as general information on the devpi-server and devpi tools. Thanks to all issue reporters and to Andi Albrecht for his pull request. best, holger -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] Shed Skin 0.9.4
Hi all, I have just released version 0.9.4 of Shed Skin, a (restricted-)Python-(2.x)-to-C++ compiler. This is the fourth maintenance release since 0.9, so no new major features were added. There have been many minor improvements though, and 3 new examples were added, bringing the total number of examples to 75. Please see my blog for more details about the release: http://shed-skin.blogspot.nl/ Or the release notes for the full list of changes: http://code.google.com/p/shedskin/wiki/releasenotes The homepage can be found here: http://code.google.com/p/shedskin/ Thanks, Mark Dufour. -- http://www.youtube.com/watch?v=E6LsfnBmdnk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: eGenix mxODBC Django Database Engine - Django ODBC Adapter 1.2.0
ANNOUNCING eGenix.com mxODBC Django Database Engine MS SQL Server ORM and ODBC Adapter for the Django Web Framework Version 1.2.0 The mxODBC Django Database Engine is our commercially supported product for connecting Django to ODBC compatible databases on Windows, Mac OS X, Linux and FreeBSD platforms. This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Django-Database-Engine-1.2.0-GA.html INTRODUCTION The mxODBC Django Database Engine product allows you to easily connect your Django website to just about any database backend on the market today, giving you the reliability of the commercially supported eGenix.com product mxODBC and the flexibility of the ODBC standard as middle-tier architecture. Flexible Database Access With the mxODBC Django Database Engine 1.2, you get two ways to connect to your databases: 1. access all your Microsoft SQL Server databases using the seamless Django ORM integration we provide for MS SQL Server 2005, 2008 and 2012, or 2. tap into the powerful SQL-based mxODBC Database Interface directly from within Django and access databases such as MS SQL Server, Oracle, IBM DB2 and Informix , Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB, and many others. Across Platforms mxODBC Django Database Engine works on Windows, Linux, Mac OS X, FreeBSD, Solaris and AIX, providing you with the same consistent interface, flexibility and robustness across all of these platforms. For more information, please see the product page: http://www.egenix.com/products/django/mxODBCDjango/ NEWS The 1.2.0 release of mxODBC Django Database Engine is the first public release of our Django database engine. Features * Fully integrated with the Django ORM: No need to learn a new API - simply continue to use the known Django ORM interface. * Compatible with all current Django versions: The mxODBC Django Database Engine supports Django 1.4 and 1.5. * Compatible with all recommended Python versions: The database engine supports Python 2.6 and 2.7; both as UCS2 (narrow) and UCS4 (wide) Unicode variant on Unix platforms. * Full Unicode Support: The database engine can communicate with the database using native Unicode and 8-bit encodings such as UTF-8 or CP1252. * Full 64-bit Support: The underlying mxODBC 3.2 library fully supports 64-bit platforms such as Mac OS X 10.6 (Snow Leopard) or later, and 64-bit Linux systems that use unixODBC, iODBC or DataDirect ODBC managers. * Cross-platform Connection Objects: The database engine will automatically choose the right platform specific ODBC manager for you. * Per Connection Adjustable ODBC Manager Interface: mxODBC supports many different ODBC managers. The mxODBC Django Database Engine allows you to select the ODBC manager on a per-connection basis. * Per Connection Customization of Interface Parameters: The database engines allows adjusting many different parameters to adapt the engine to your specific database needs, should you have special requirements. MS SQL Server ORM Features -- * MS SQL Server fully integrated into the Django ORM: Access MS SQL Server through the Django ORM, just like any other Django ORM database. * MS SQL Server Regular Expression Emulation: Even though MS SQL Server itself does not support regular expressions, the mxODBC Django Database Engine provides an emulation for simple regular expressions to simplify porting existing applications to a SQL Server backend. * MS SQL Server Aggregate Function Support: We provide a special aggregate function implementation to have the Django ORM support SQL Server aggregate functions. * MS SQL Server Timestamp Support: SQL Server support millisecond accuracy on timestamps. The database engine will take care of applying the necessary rounding for the microsecond precision Python timestamps in a seamless way. * Support for all popular SQL Server ODBC drivers: The mxODBC Django Database Engine supports the MS SQL Server Native Client on Windows, the MS SQL Server Native Client for Linux, as well as the FreeTDS ODBC driver. Commercial drivers from well-known driver vendors are also supported. * Support for accessing SQL Server from Windows and Unix platforms: On Windows and Linux you can use the SQL Server Native Client, on other Unix platforms and Mac OS X, the FreeTDS driver or other commercial drivers can be used. Direct mxODBC Interface to other Databases
Tonight's meeting + s Database link
First off what a fun meeting it was tonight with a great conversation. Let's do more of them. Next this is the link to that JSON Database I mentioned but could not recall the name on. http://www.rethinkdb.com/ RethinkDB overview RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn. See the highlights of RethinkDB Simple programming model: • JSON data model and immediate consistency. • Distributed joins, subqueries, aggregation, atomic updates. • Secondary, compound, and arbitrarily computed indexes. • Hadoop-style map/reduce. Easy administration: • Friendly web and command-line administration tools. • Takes care of machine failures and network interrupts. • Multi-datacenter replication and failover. Horizontal scalability: • Sharding and replication to multiple nodes. • Queries are automatically parallelized and distributed. • Lock-free operation via MVCC concurrency. RethinkDB compared to other databases: • Read the FAQ for information on architectural tradeoffs. • Find out how RethinkDB compares to MongoDB. • See our take on what makes RethinkDB different. -- http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
On Tue, 18 Jun 2013 00:12:34 -0400, Dave Angel wrote:
On 06/17/2013 10:42 PM, Steven D'Aprano wrote:
On Mon, 17 Jun 2013 21:06:57 -0400, Dave Angel wrote:
On 06/17/2013 08:41 PM, Steven D'Aprano wrote:
SNIP
In Python 3.2 and older, the data will be either UTF-4 or UTF-8,
selected when the Python compiler itself is compiled.
I think that was a typo. Do you perhaps UCS-2 or UCS-4
Yes, that would be better.
UCS-2 is identical to UTF-16, except it doesn't support non-BMP
characters and therefore doesn't have surrogate pairs.
UCS-4 is functionally equivalent to UTF-16,
Perhaps you mean UTF-32 ?
Yes, sorry for the repeated confusion.
as far as I can tell. (I'm
not really sure what the difference is.)
Now you've got me curious, by bringing up surrogate pairs. Do you know
whether a narrow build (say 3.2) really works as UTF16, so when you
encode a surrogate pair (4 bytes) to UTF-8, it encodes a single Unicode
character into a single UTF-8 sequence (prob. 4 bytes long) ?
In a Python narrow build, the internal storage of strings is equivalent
to UTF-16: all characters in the Basic Multilingual Plane require two
bytes:
py sys.maxunicode
65535
py sys.getsizeof('π') - sys.getsizeof('')
2
Outside of the BMP, characters are treated as a pair of surrogates:
py c = chr(0x10F000) # one character...
py len(c) # ...stored as a pair of surrogates
2
Encoding and decoding works fine:
py c.encode('utf-8').decode('utf-8') == c
True
py c.encode('utf-8')
b'\xf4\x8f\x80\x80'
The problem with surrogates is that it is possible to accidentally
separate the pair, which leads to broken, invalid text:
py c[0].encode('utf-8')
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'utf-8' codec can't encode character '\udbfc' in
position 0: surrogates not allowed
(The error message is a little misleading; surrogates are allowed, but
only if they make up a valid pair.)
Python's handling of UTF-16 is, as far as I know, correct. What isn't
correct is that the high-level Python string methods assume that two
bytes == one character, which can lead to surrogates being separated,
which gives you junk text. Wide builds don't have this problem, because
every character == four bytes, and neither does Python 3.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Tonight's meeting + s Database link -- Sorry
Sorry all, I managed to send that last email to wrong Python list. -Kevin On Jun 17, 2013, at 10:55 PM, Kevin LaTona li...@studiosola.com wrote: First off what a fun meeting it was tonight with a great conversation. Let's do more of them. Next this is the link to that JSON Database I mentioned but could not recall the name on. http://www.rethinkdb.com/ RethinkDB overview RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn. See the highlights of RethinkDB Simple programming model: • JSON data model and immediate consistency. • Distributed joins, subqueries, aggregation, atomic updates. • Secondary, compound, and arbitrarily computed indexes. • Hadoop-style map/reduce. Easy administration: • Friendly web and command-line administration tools. • Takes care of machine failures and network interrupts. • Multi-datacenter replication and failover. Horizontal scalability: • Sharding and replication to multiple nodes. • Queries are automatically parallelized and distributed. • Lock-free operation via MVCC concurrency. RethinkDB compared to other databases: • Read the FAQ for information on architectural tradeoffs. • Find out how RethinkDB compares to MongoDB. • See our take on what makes RethinkDB different. -- http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
On 06/17/2013 08:50 AM, Simpleton wrote: On 17/6/2013 2:58 μμ, Michael Torrie wrote: a = 5 b = a a --- memory address b --- memory address I like to think a and b as references to the same memory address Not quite: a and b _are_ memory addresses, At the same time, a and b are references to the data (the objects) stored in those memory locations. The distinction is probably more important in languages like C/C++, where the _language_ gives you direct access to, and can manipulate, these memory addresses (through pointers). Python handles it differently and does not give you this sort of ability, it all occurs under the hood. Yes, the id() function will tell you the addresses, but you can't do anything with them other than perhaps compare them. It's really pretty much useless information. -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On Tue, Jun 18, 2013 at 11:03 AM, ru...@yahoo.com wrote: if Python had perfect documentation, he still wouldn't read it. If your crystal ball is that good, could you try using it to solve some of Nikos' problems? I have done so, many times. Sometimes it helps, often it doesn't. Once, it led me to accept his root password. You doubtless saw how THAT went over. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On Tue, Jun 18, 2013 at 6:15 AM, Guy Scree nob...@nowhere.com wrote: I recommend that all participants in this thread, especially Alex and Anton, research the term Pathological Altruism I don't intend to buy a book about it, but based on flipping through a few Google results and snippets, I'm thinking that this is the Paladin fault that I know from Dungeons Dragons. :) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On Tue, Jun 18, 2013 at 2:39 PM, alex23 wuwe...@gmail.com wrote: tl;dr Stop acting like a troll and we'll stop perceiving you as such. This being Python-list, we duck-type. You don't have to declare that you're a troll, like you would in C; you just react like a troll and we'll treat you as one. We never ask are you a troll, we just ask do you quack like a troll. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
On Mon, 17 Jun 2013 23:39:10 -0700, Larry Hudson wrote:
On 06/17/2013 08:50 AM, Simpleton wrote:
On 17/6/2013 2:58 μμ, Michael Torrie wrote:
a = 5
b = a
a --- memory address
b --- memory address
I like to think a and b as references to the same memory address
Not quite: a and b _are_ memory addresses,
Not in Python they aren't. a and b are names in a namespace.
At the same time, a and b
are references to the data (the objects) stored in those memory
locations.
Not in Python they aren't. In Python, objects are free to move around
memory. Not all implementations take advantage of this freedom, but some
like Jython, IronPython and PyPy do.
The distinction is probably more important in languages like C/C++,
where the _language_ gives you direct access to, and can manipulate,
these memory addresses (through pointers). Python handles it
differently and does not give you this sort of ability, it all occurs
under the hood. Yes, the id() function will tell you the addresses,
but you can't do anything with them other than perhaps compare them.
It's really pretty much useless information.
The id() function does not tell you the address of the object, except by
accident. The id() function gives you an arbitrary ID number for the
object:
steve@runes:~$ ipy
IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433
Type help, copyright, credits or license for more information.
id([])
43
id('*')
44
steve@runes:~$ jython
Jython 2.5.1+ (Release_2_5_1, Aug 4 2010, 07:18:19)
[OpenJDK Client VM (Sun Microsystems Inc.)] on java1.6.0_18
Type help, copyright, credits or license for more information.
id([])
1
id('*')
2
That some implementations happen to use a fixed memory address as the ID
number is, well, a mere accident of implementation. That's not what id()
*is*, any more than id() returns the next value in an integer sequence
starting from 43 just because that's what IronPython happens to do.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Help me with the script? How to find items in csv file A and not in file B and vice versa
Hello,
Let's say I want to compare two csv files: file A and file B. They are both
similarly built - the first column has product IDs (one product per row) and
the columns provide some stats about the products such as sales in # and $.
I want to compare these files - see which product IDs appear in the first
column of file A and not in B, and which in B and not A.
Finally, it would be very great if the result could be written into two new CSV
files - one product ID per row in the first column. (no other data in the other
columns needed)
This is the script I tried:
==
import csv
#open CSV's and read first column with product IDs into variables pointing to
lists
A = [line.split(',')[0] for line in open('Afile.csv')]
B = [line.split(',')[0] for line in open('Bfile.csv')]
#create variables pointing to lists with unique product IDs in A and B
respectively
inAnotB = list(set(A)-set(B))
inBnotA = list(set(B)-set(A))
print inAnotB
print inBnotA
c = csv.writer(open(inAnotB.csv, wb))
c.writerow([inAnotB])
d = csv.writer(open(inBnotA.csv, wb))
d.writerow([inBnotA])
print done!
=
But it doesn't produce the required results.
It prints IDs in this format:
247158132\n
and nothing to the csv files.
You could probably tell I'm a newbie.
Could you help me out?
here's some dummy data:
https://docs.google.com/file/d/0BwziqsHUZOWRYU15aEFuWm9fajA/edit?usp=sharing
https://docs.google.com/file/d/0BwziqsHUZOWRQVlTelVveEhsMm8/edit?usp=sharing
Thanks a bunch in advance! :)
--
http://mail.python.org/mailman/listinfo/python-list
ANN: eGenix mxODBC Django Database Engine - Django ODBC Adapter 1.2.0
ANNOUNCING eGenix.com mxODBC Django Database Engine MS SQL Server ORM and ODBC Adapter for the Django Web Framework Version 1.2.0 The mxODBC Django Database Engine is our commercially supported product for connecting Django to ODBC compatible databases on Windows, Mac OS X, Linux and FreeBSD platforms. This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Django-Database-Engine-1.2.0-GA.html INTRODUCTION The mxODBC Django Database Engine product allows you to easily connect your Django website to just about any database backend on the market today, giving you the reliability of the commercially supported eGenix.com product mxODBC and the flexibility of the ODBC standard as middle-tier architecture. Flexible Database Access With the mxODBC Django Database Engine 1.2, you get two ways to connect to your databases: 1. access all your Microsoft SQL Server databases using the seamless Django ORM integration we provide for MS SQL Server 2005, 2008 and 2012, or 2. tap into the powerful SQL-based mxODBC Database Interface directly from within Django and access databases such as MS SQL Server, Oracle, IBM DB2 and Informix , Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB, and many others. Across Platforms mxODBC Django Database Engine works on Windows, Linux, Mac OS X, FreeBSD, Solaris and AIX, providing you with the same consistent interface, flexibility and robustness across all of these platforms. For more information, please see the product page: http://www.egenix.com/products/django/mxODBCDjango/ NEWS The 1.2.0 release of mxODBC Django Database Engine is the first public release of our Django database engine. Features * Fully integrated with the Django ORM: No need to learn a new API - simply continue to use the known Django ORM interface. * Compatible with all current Django versions: The mxODBC Django Database Engine supports Django 1.4 and 1.5. * Compatible with all recommended Python versions: The database engine supports Python 2.6 and 2.7; both as UCS2 (narrow) and UCS4 (wide) Unicode variant on Unix platforms. * Full Unicode Support: The database engine can communicate with the database using native Unicode and 8-bit encodings such as UTF-8 or CP1252. * Full 64-bit Support: The underlying mxODBC 3.2 library fully supports 64-bit platforms such as Mac OS X 10.6 (Snow Leopard) or later, and 64-bit Linux systems that use unixODBC, iODBC or DataDirect ODBC managers. * Cross-platform Connection Objects: The database engine will automatically choose the right platform specific ODBC manager for you. * Per Connection Adjustable ODBC Manager Interface: mxODBC supports many different ODBC managers. The mxODBC Django Database Engine allows you to select the ODBC manager on a per-connection basis. * Per Connection Customization of Interface Parameters: The database engines allows adjusting many different parameters to adapt the engine to your specific database needs, should you have special requirements. MS SQL Server ORM Features -- * MS SQL Server fully integrated into the Django ORM: Access MS SQL Server through the Django ORM, just like any other Django ORM database. * MS SQL Server Regular Expression Emulation: Even though MS SQL Server itself does not support regular expressions, the mxODBC Django Database Engine provides an emulation for simple regular expressions to simplify porting existing applications to a SQL Server backend. * MS SQL Server Aggregate Function Support: We provide a special aggregate function implementation to have the Django ORM support SQL Server aggregate functions. * MS SQL Server Timestamp Support: SQL Server support millisecond accuracy on timestamps. The database engine will take care of applying the necessary rounding for the microsecond precision Python timestamps in a seamless way. * Support for all popular SQL Server ODBC drivers: The mxODBC Django Database Engine supports the MS SQL Server Native Client on Windows, the MS SQL Server Native Client for Linux, as well as the FreeTDS ODBC driver. Commercial drivers from well-known driver vendors are also supported. * Support for accessing SQL Server from Windows and Unix platforms: On Windows and Linux you can use the SQL Server Native Client, on other Unix platforms and Mac OS X, the FreeTDS driver or other commercial drivers can be used. Direct mxODBC Interface to other Databases
Re: Don't feed the troll...
Op 17-06-13 19:56, ru...@yahoo.com schreef: On 06/17/2013 02:15 AM, Antoon Pardon wrote: Op 17-06-13 05:46, ru...@yahoo.com schreef: On 06/16/2013 02:04 PM, Steven D'Aprano wrote: Yes. Trying to start flame wars with Nikos is unacceptable behaviour. It is unproductive, it makes this a hostile, unpleasant place to be, it ruins the environment for the rest of the community, it's off topic, and it simply doesn't work to discourage trolls. The difficulty with trying to suppress such responses is that the flamers get just as much pleasure from having a target to unrestrainedly spew their pent up anger and vile at, as the troll gets from simulating that reaction. The result is a positive feedback loop. Well if asocial behaviour of one provokes asocial behaviour in others, you can't claim the problem is not the social behaviour of the first. Sure I can. If you have a photodetector that activates a bright light when it detects a flash, you can blame the first flash for the fact that the bright light is on all the time. Or you can say that stray flashes are to be expected now and then in the environment of this system and the fault is responding to them with a bright light. But that doesn't make sense. Your photodetector working as it does, is just as expected as the happening of stray flashes. There is no reason to differentiate between these two in terms of being expected or not. As I said (and you disagree with below), I did see some attempts to adapt his behavior but it is not realistic to expect immediate acquiescence to every request made here, especially given that a lot of them were/are bullshit. I don't care whether it is realistic or not. If he can't conform his behaviour in a reasonable way, he doesn't belong here. It is not realistic to expect someone who is just learing to swim to survive a jump in the deep. So we expect those people not to jump in the deep. We don't tolerate them jumping in the deep on the expectation that others will pull them out. That is wat Nikos keeps doing here, jumping in the deep. And a lot of people feel it is time we let him (metaphorically drown). I speculate that half of his bad behavior is simple I want now and don't care about your conventions. The rest is a reaction to we're the alphas, your a beta attitude expressed by many here and later, overt hostility directed at him. He has changed some things -- his posting method, he's made an effort to understand his encoding issues, etc.' I don't see that much change in his style. He just admitted not reading help files (because they are too technical for him). So essentialy he is asking we give him a beginners tutorial in everything he doesn't understand without much effort of him trying to understand things on his own and without much appreciation for the time of others. See my reply to ChrisA. Your reply doesn't address his unwillingness to read the documentation which was IMO rather apparant. My personal feeling is that he tends to ask on the list too quickly, but I suspect he also does more than you're giving him credit for. He seems to be naive (eg the password event), open and honest so when he says he has been trying to fix something for hours I am prone to believe him. I don't care. In the end he is still jumping in the deep expecting others to drag him out. I don't care how much he does. Just as I don't care how much energy someone has put into learning to swim. If your skills are not adequate you don't jump into the deep. I think his approach to fixing is to try making changes more or less at random, in part because he doesn't understand the docs (or doesn't look at them because they haven't made sense to him in the past) and in part because he hasn't developed any skill in debugging (a skill that I think most everyone here takes for granted but which doesn't come naturally to some people) and which also accounts for the poor formulation of his questions. I don't care whether he has trouble developping debuging skills or not. Just as I don't care if someone has trouble learning to swim or not. If it is reasonable to expect those skill in a specific environment, you are just rude if you enter without those skill and expect others to get you out of the troubles you probably will fall victim to. In the mean time you and steve can just killfile those you think are just egging him on. Unfortunately it is not a symmetrical situation. Nikos responds only in his own threads and is more killable that many of the eggers who both more numerous and respond in many other threads that are of interest. Can you explain how these people can egg Nikos on in threads in which he doesn't participate? I also don't find your assymmetry of much relevance. It is just happens how history played out. There is no priciple difference. In both cases we have people being annoyed by the behaviour of others. I you want to advise others should
Re: Help me with the script? How to find items in csv file A and not in file B and vice versa
Alan Newbie wrote:
Hello,
Let's say I want to compare two csv files: file A and file B. They are
both similarly built - the first column has product IDs (one product per
row) and the columns provide some stats about the products such as sales
in # and $.
I want to compare these files - see which product IDs appear in the first
column of file A and not in B, and which in B and not A. Finally, it would
be very great if the result could be written into two new CSV files - one
product ID per row in the first column. (no other data in the other
columns needed)
This is the script I tried:
==
import csv
#open CSV's and read first column with product IDs into variables pointing
#to lists
A = [line.split(',')[0] for line in open('Afile.csv')]
B = [line.split(',')[0] for line in open('Bfile.csv')]
#create variables pointing to lists with unique product IDs in A and B
#respectively
inAnotB = list(set(A)-set(B))
inBnotA = list(set(B)-set(A))
print inAnotB
print inBnotA
c = csv.writer(open(inAnotB.csv, wb))
c.writerow([inAnotB])
d = csv.writer(open(inBnotA.csv, wb))
d.writerow([inBnotA])
print done!
=
But it doesn't produce the required results.
It prints IDs in this format:
247158132\n
Python reads lines from a file with the trailing newline included, and
line.split(,) with only one column (i. e. no comma) keeps the whole line.
As you already know about the csv module you should use it to read your
data, e. g. instead of
A = [line.split(',')[0] for line in open('Afile.csv')]
try
with open(Afile.csv, rb) as f:
a = {row[0] for row in csv.reader(f)}
...
I used {...} instead of [...], so a is already a set and you can proceed:
in_a_not_b = a - b
Finally as a shortcut for
for item in in_a_not_b:
writer.writerow([item])
use the writerows() method to write your data:
with open(inAnotB.csv, wb) as f:
writer = csv.writer(f)
writer.writerows([item] for item in_a_not_b)
Note that I'm wrapping every item in the set rather than the complete set as
a whole. If you wanted to be clever you could spell that even more succinct
as
writer.writerows(zip(in_a_not_b))
and nothing to the csv files.
You could probably tell I'm a newbie.
Could you help me out?
here's some dummy data:
https://docs.google.com/file/d/0BwziqsHUZOWRYU15aEFuWm9fajA/edit?usp=sharing
https://docs.google.com/file/d/0BwziqsHUZOWRQVlTelVveEhsMm8/edit?usp=sharing
Thanks a bunch in advance! :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
Στις 18/6/2013 9:39 πμ, ο/η Larry Hudson έγραψε: Not quite: a and b _are_ memory addresses, At the same time, a and b are references to the data (the objects) stored in those memory locations. The distinction is probably more important in languages like C/C++, where the _language_ gives you direct access to, and can manipulate, these memory addresses (through pointers). Python handles it differently and does not give you this sort of ability, it all occurs under the hood. Yes, the id() function will tell you the addresses, but you can't do anything with them other than perhaps compare them. It's really pretty much useless information. So, a and b are actual memory addresses. Does the term of a pointer exist in Python? I mean if print(a) or print(b) outputs the object that a and b are linked to, then how do we access a's and b's memory locations themselves t create links among variables, one pointing to the other and so on? Can a variable point to another variable or variables never point to other variables but instead are *only* linked to the objects of those var's instead? -- What is now proved was at first only imagined! -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
Op 18-06-13 01:02, Steven D'Aprano schreef: On Mon, 17 Jun 2013 09:31:53 +0200, Antoon Pardon wrote: Op 16-06-13 22:04, Steven D'Aprano schreef: On Sun, 16 Jun 2013 20:16:34 +0200, Antoon Pardon wrote: You are trying to get it both ways. On the one hand you try to argue that there are no boundaries I have never, ever argued that there are no boundaries. I have repeatedly made it clear to Nikos when I thought he was behaving improperly. And I've done the same to others when they've acted improperly. That doesn't mean much. People can and do contradict themselves. So the fact that you made it clear to Nikos that he behaved improperly doesn't contradict you arguing somewhere else in a way that strongly suggest there are no boudaries. Except that I have never, ever argued or suggested or even hinted that there are no boundaries. The most you might legitimately accuse me of is failing to be sufficiently vigilant at enforcing boundaries, according to *your* idea of what is sufficient. But I'll take note that you assert there are boundaries. So I'll take it that there is nothing wrong with playing Internet Police and taking people to task who transgress this boundaries? There is an enormous difference between doing what I, and others, have done, which is to *politely* and *fairly* tell Nikos when he has transgressed, and what the flame-warriors have done, which is just fire off invective and insults. I disagree. You have been polite to the person who is ruining it for a lot of other people. What good is it to politely and fairly tell someone he is transgressing, when he will just continue in the same way. At some point you keeping to be polite and answering his questions, becomes enabling behaviour. Your politely and fairly pointing out his transgressions just becomes a way in cooperating with his annoying behaviour. You keeping it polite and fair doens't mean much. It isn't that difficult to act as an asshole while presenting oneself as being polite and fair. And no I don't want to imply you are an asshole. I just want to make it clear I don't put much weight is being polite and fair. Not long ago I got taken to task, politely, off-list for responding to Ranting Rick with sarcasm. Sometimes the momentary pleasure of a flame outweighs the knowledge that it probably isn't doing any good and may be doing harm. I get that and don't hold it against anyone if they succumb to temptation once in a while. (Those like Peter Otten, who have been regulars here for *years* while still showing the patience of a saint, never fail to astonish me. If I could be even half as good.) But continuing to flame after being asked not to, and defending flamers, that crosses the line from spirit is willing, flesh is weak into *willfully bad* territory. You were asked not to continue encouraging Nikos's assholery behaviour. So it seems you are in that *willfully bad* territory yourself. And no, politely and fairly telling Nikos he is transgressing doesn't cut it. If he keeps acting like an asshole and you keep helping him you are encouraging his assholery behaviour no matter how many times you politely and fairly point out his transgressions. One thing I would like to make clear, is that I find you making it clear he behaviour is improper, to be inadequate for the reason that it ignores the possibility that you are playing a troll game. Oh my, that's funny. But seriously, don't do that. I won't put up with that sort of thing. You rarely contribute in this community, and now here you are trying to take the moral high ground by defending flaming and criticising those who give actual helpful, on-topic advice. I won't be called a troll by you. Do it again, and you're plonked. I didn't call you a troll. I just wanted you to consider you might be participating in what is essentially a troll game. And that what you see as pointing out a transgression, is just a kind of move in the game of the troll. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: Help me with the script? How to find items in csv file A and not in file B and vice versa
Hi Peter,
First off - many (many!) thanks.
There's some error I don't understand.
Here's the amended script I used:
import csv
#open CSV's and read first column with product IDs into variables pointing to
lists
with open(Afile.csv, rb) as f:
a = {row[0] for row in csv.reader(f)}
with open(Bfile.csv, rb) as g:
b = {row[0] for row in csv.reader(g)}
#create variables pointing to lists with unique product IDs in A and B
respectively
in_a_not_b = a-b
in_b_not_a = b-a
print in_a_not_b
print in_b_not_a
with open(inAnotB.csv, wb) as f:
writer = csv.writer(f)
writer.writerows([item] for item in_a_not_b)
with open(inAnotB.csv, wb) as g:
writer = csv.writer(g)
writer.writerows([item] for item in_b_not_a)
print done!
and when I run it I get an invalid syntex error and (as a true newbie I used a
GUI)in_a_not_b is highlighted in the
with open(inAnotB.csv, wb) as f:
writer = csv.writer(f)
writer.writerows([item] for item in_a_not_b)
part.
Could you please point our what I'm doing wrong?
Thanks again :)
On Tuesday, June 18, 2013 11:39:41 AM UTC+3, Peter Otten wrote:
Alan Newbie wrote:
Hello,
Let's say I want to compare two csv files: file A and file B. They are
both similarly built - the first column has product IDs (one product per
row) and the columns provide some stats about the products such as sales
in # and $.
I want to compare these files - see which product IDs appear in the first
column of file A and not in B, and which in B and not A. Finally, it would
be very great if the result could be written into two new CSV files - one
product ID per row in the first column. (no other data in the other
columns needed)
This is the script I tried:
==
import csv
#open CSV's and read first column with product IDs into variables pointing
#to lists
A = [line.split(',')[0] for line in open('Afile.csv')]
B = [line.split(',')[0] for line in open('Bfile.csv')]
#create variables pointing to lists with unique product IDs in A and B
#respectively
inAnotB = list(set(A)-set(B))
inBnotA = list(set(B)-set(A))
print inAnotB
print inBnotA
c = csv.writer(open(inAnotB.csv, wb))
c.writerow([inAnotB])
d = csv.writer(open(inBnotA.csv, wb))
d.writerow([inBnotA])
print done!
=
But it doesn't produce the required results.
It prints IDs in this format:
247158132\n
Python reads lines from a file with the trailing newline included, and
line.split(,) with only one column (i. e. no comma) keeps the whole line.
As you already know about the csv module you should use it to read your
data, e. g. instead of
A = [line.split(',')[0] for line in open('Afile.csv')]
try
with open(Afile.csv, rb) as f:
a = {row[0] for row in csv.reader(f)}
...
I used {...} instead of [...], so a is already a set and you can proceed:
in_a_not_b = a - b
Finally as a shortcut for
for item in in_a_not_b:
writer.writerow([item])
use the writerows() method to write your data:
with open(inAnotB.csv, wb) as f:
writer = csv.writer(f)
writer.writerows([item] for item in_a_not_b)
Note that I'm wrapping every item in the set rather than the complete set as
a whole. If you wanted to be clever you could spell that even more succinct
as
writer.writerows(zip(in_a_not_b))
and nothing to the csv files.
You could probably tell I'm a newbie.
Could you help me out?
here's some dummy data:
https://docs.google.com/file/d/0BwziqsHUZOWRYU15aEFuWm9fajA/edit?usp=sharing
https://docs.google.com/file/d/0BwziqsHUZOWRQVlTelVveEhsMm8/edit?usp=sharing
Thanks a bunch in advance! :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
On Tue, 18 Jun 2013 11:49:36 +0300, Νίκος wrote:
Στις 18/6/2013 9:39 πμ, ο/η Larry Hudson έγραψε:
Not quite: a and b _are_ memory addresses, At the same time, a and b
are references to the data (the objects) stored in those memory
locations.
The distinction is probably more important in languages like C/C++,
where the _language_ gives you direct access to, and can manipulate,
these memory addresses (through pointers). Python handles it
differently and does not give you this sort of ability, it all occurs
under the hood. Yes, the id() function will tell you the addresses,
but you can't do anything with them other than perhaps compare them.
It's really pretty much useless information.
So, a and b are actual memory addresses.
No, no, no, a thousand times no.
Does the term of a pointer exist in Python?
No.
I mean if print(a) or
print(b) outputs the object that a and b are linked to, then how do we
access a's and b's memory locations themselves t create links among
variables, one pointing to the other and so on?
You cannot. You can only have links between OBJECTS, not between
VARIABLES. There is no way to have a name a set to point to another
name b. All you can do is have a name a set to refer to the same
object as b has *right now*. If b changes to another object, a will
not follow.
Can a variable point to another variable or variables never point to
other variables but instead are *only* linked to the objects of those
var's instead?
Names are *always* linked to objects, not to other names.
a = []
b = a # Now a and b refer to the same list
a = {} # Now a refers to a dict, and b refers to the same list as before
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
RE: Python Liscensing
From: Steven Hern Sent: 06 June 2013 08:49 To: 'webmas...@python.org' Cc: Dave Jordan Subject: Python Liscensing Dear Sir/Madam, We are an educational establishment which wishes to use Python 3.3.2 - Does the license cover multi-users in a classroom environment? Thanks Steven Hern Wigan and Leigh College Desktop Services -- http://mail.python.org/mailman/listinfo/python-list
decorator to fetch arguments from global objects
Using a CouchDB server we have a different database object potentially for
every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it once
in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Now it kinda of works already with the decorator below.
The problem is that the argument is positional so I end up maybe passing it
twice.
So I have to enforce that 'db' if there is passed as first argument..
It would be a lot easier removing the db from the arguments but then it
would look too magic and I didn't want to change the signature.. any other
advice?
def with_optional_db(func):
Decorator that sets the database to the global current one if
not passed in or if passed in and None
@wraps(func)
def _with_optional_db(*args, **kwargs):
func_args = func.func_code.co_varnames
db = None
# if it's defined in the first elements it needs to be
# assigned to *args, otherwise to kwargs
if 'db' in func_args:
assert 'db' == func_args[0], Needs to be the first defined
else:
db = kwargs.get('db', None)
if db is None:
kwargs['db'] = get_current_db()
assert kwargs['db'] is not None, Can't have a not defined database
ret = func(*args, **kwargs)
return ret
return _with_optional_db
--
http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
Στις 18/6/2013 12:05 μμ, ο/η Steven D'Aprano έγραψε:
Names are *always* linked to objects, not to other names.
a = []
b = a # Now a and b refer to the same list
a = {} # Now a refers to a dict, and b refers to the same list as before
I see, thank you Steven.
But since this is a fact how do you create complicated data structures
that rely on various variables pointing one to another liek we did in
C++(cannot recall their names) ?
--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help me with the script? How to find items in csv file A and not in file B and vice versa
alonn...@gmail.com wrote: and when I run it I get an invalid syntex error and (as a true newbie I used a GUI)in_a_not_b is highlighted in the with open(inAnotB.csv, wb) as f: writer = csv.writer(f) writer.writerows([item] for item in_a_not_b) The syntax for the for-clause in a comprehension is for x in something thus you are missing the in keyword: writer.writerows([item] for item in in_a_not_b) Bye, Andreas -- http://mail.python.org/mailman/listinfo/python-list
Re: Variables versus name bindings [Re: A certainl part of an if() structure never gets executed.]
On Tue, Jun 18, 2013 at 7:51 PM, Νίκος supp...@superhost.gr wrote:
Στις 18/6/2013 12:05 μμ, ο/η Steven D'Aprano έγραψε:
Names are *always* linked to objects, not to other names.
a = []
b = a # Now a and b refer to the same list
a = {} # Now a refers to a dict, and b refers to the same list as before
I see, thank you Steven.
But since this is a fact how do you create complicated data structures that
rely on various variables pointing one to another liek we did in C++(cannot
recall their names) ?
Why do you need to? Linked lists, trees, and so on are just tools.
They're usually used to implement data structures like mappings,
growable arrays, lists that can have elements inserted into them, etc,
etc. Python does these sorts of things in better ways. You should not
need to worry about memory locations, pointers, etc. Now, if you want
to have one object reference another, that can be done in plenty of
ways. Check the Python tutorial.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
python-django for dynamic web survey?
Hi guys! Please help me with your advices and ideas. I need to create a web survey that will dynamically (randomly) select questions and descriptions from a dataset, present them to users, collect their answers and store them back in the dataset. (Every user gets different set of questions randomly selested from the pool.) Since I am very inexperienced with this stuff I want to ask you if you thing that python (which I use) and django (which I have never used before) would be a good way to do this easily? I have seen there are some django survey and crowdsourcing librarires... What do you think? Is there an easier way? I would like some fast, clean, modifyAndDoNotCodeFromScratch solution. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Liscensing
On 18 June 2013 09:56, Steven Hern s.h...@wigan-leigh.ac.uk wrote: We are an educational establishment which wishes to use Python 3.3.2 – Does the license cover multi-users in a classroom environment? Yes, absolutely. Many educational institutions universities, schools, etc. use Python in classroom environments (the fact that it is a classroom really makes no difference). Here is the full license: http://docs.python.org/3.3/license.html And here is the relevant text (from clause 2): ''' Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python 3.3.2 alone or in any derivative version, provided, however, that PSF’s License Agreement and PSF’s notice of copyright, i.e., “Copyright © 2001-2013 Python Software Foundation; All Rights Reserved” are retained in Python 3.3.2 alone or in any derivative version prepared by Licensee. ''' Half of that text refers to making a derivative version of Python (which I assume you're not intending to do). Otherwise it essentially just says that you can use it anywhere you like for anything you want without paying any money. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Help me with the script? How to find items in csv file A and not in file B and vice versa
thanks a lot :) -- http://mail.python.org/mailman/listinfo/python-list
Re: python-django for dynamic web survey?
Django makes your life a lot easier in many ways, but you still need some time to learn it. The task you're trying it's not trivial though, depending on your experience it might take a while with any library/framework.. -- http://mail.python.org/mailman/listinfo/python-list
Re: python-django for dynamic web survey?
On 18-Jun-13 12:59 PM, andrea crotti wrote: Django makes your life a lot easier in many ways, but you still need some time to learn it. The task you're trying it's not trivial though, depending on your experience it might take a while with any library/framework.. I have an overall experience with languages, frameworks etc. I was hoping to get some simple example and than modify it to do what I need it to do. -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
Oscar Benjamin oscar.j.benja...@gmail.com writes: There is a very simple solution used by many mailing lists Yes, that solution is described in RFC 2369: the “List-Post” field in the header of every message sent through the mailing list. which is to set the Reply-To header to point back to the mailing list. That is not a solution, since the ‘Reply-To’ field already has a different purpose contrary to your intent. It is to be set by the person sending the message, if they choose. It is not for some intermediary to interfere with. It is a field for the sender to direct *individual* responses back to themselves – and, if they don't set that field, no intermediary should abuse it. That way any old email client on any OS/computer/phone/website etc. has the required button to reply to the list without CCing anyone. By breaking the standard “reply to author” behaviour. This is not a solution. The “List-Post” field has been standard for more than a decade. If anyone is using an MUA that doesn't use it, please imrpove that situation: pressure your vendor to fix that deficiency, and/or switch to a better mail client until then. It also reduces the chance of accidentally replying off-list. What damage is done by accidentally replying off-list? At worst, you merely need to send the message again to the list. The damage is minimal, and easily rectified. Your proposed interference with the “Reply-To” field, though, invites much more serious errors: it sets up a person to send a message to people they did *not* intend, when using a function (“reply to author”, often simply called “reply”) specifically for reaching the sender *only*. If your message contains information only intended to be seen by the author to whom they are replying, the standard behaviour for “Reply-To” gives the reasonable expectation it will go only to the author. But if a broken mailing list that munges “Reply-To” to direct your reply to the whole mailing list, that is damage which can't be un-done. Please don't propose breaking standard behaviour by interfering with the meaning of standard fields. We have exactly the fields we need already: the RFC 2369 fields are in the header of every message from the mailing list. The “List-Post” field, saying where mail should be directed to reach the mailing list, is exactly what is needed. -- \ “Ours is a world where people don't know what they want and are | `\ willing to go through hell to get it.” —Donald Robert Perry | _o__) Marquis | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
On Tue, Jun 18, 2013 at 10:47 AM, andrea crotti andrea.crott...@gmail.com
wrote:
def with_optional_db(func):
Decorator that sets the database to the global current one if
not passed in or if passed in and None
@wraps(func)
def _with_optional_db(*args, **kwargs):
func_args = func.func_code.co_varnames
db = None
# if it's defined in the first elements it needs to be
# assigned to *args, otherwise to kwargs
if 'db' in func_args:
assert 'db' == func_args[0], Needs to be the first defined
else:
db = kwargs.get('db', None)
if db is None:
kwargs['db'] = get_current_db()
assert kwargs['db'] is not None, Can't have a not defined database
ret = func(*args, **kwargs)
return ret
return _with_optional_db
If db is always the first argument, you could also use type (or hasattr)
checking of the first argument, and push a db if it's not one.
Or be specific about it and take a db or some kind of placeholder (such as
an object, DEFAULT_DB, or the default string)
Cheers
--
Fábio Santos
--
http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
andrea crotti andrea.crotti.0 at gmail.com writes:
Using a CouchDB server we have a different database object potentially for
every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it once
in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Now it kinda of works already with the decorator below.
The problem is that the argument is positional so I end up maybe passing
it twice.
So I have to enforce that 'db' if there is passed as first argument..
It would be a lot easier removing the db from the arguments but then it
would look too magic and I didn't want to change the signature.. any other
advice?
def with_optional_db(func):
Decorator that sets the database to the global current one if
not passed in or if passed in and None
at wraps(func)
def _with_optional_db(*args, **kwargs):
func_args = func.func_code.co_varnames
db = None
# if it's defined in the first elements it needs to be
# assigned to *args, otherwise to kwargs
if 'db' in func_args:
assert 'db' == func_args[0], Needs to be the first defined
else:
db = kwargs.get('db', None)
if db is None:
kwargs['db'] = get_current_db()
assert kwargs['db'] is not None, Can't have a not defined database
ret = func(*args, **kwargs)
return ret
return _with_optional_db
I'm not sure, whether your code would work. I get the logic for the db in
kwargs case, but why are you checking whether db is in func_args? Isn't the
real question whether it's in args ?? In general, I don't understand why you
want to use .func_code.co_varnames here. You know how you defined your
function (or rather method):
class Entity:
def save_doc(db):
...
Maybe I misunderstood the problem?
Wolfgang
--
http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On 2013-06-18, Chris Angelico ros...@gmail.com wrote: On Tue, Jun 18, 2013 at 2:39 PM, alex23 wuwe...@gmail.com wrote: tl;dr Stop acting like a troll and we'll stop perceiving you as such. This being Python-list, we duck-type. You don't have to declare that you're a troll, like you would in C; you just react like a troll and we'll treat you as one. We never ask are you a troll, we just ask do you quack like a troll. Indeed. The is he a troll question is a discussion about internals. And like many Python users, some of us do like to discuss questions about internals (though we hopefully know enough not to depend on the answers being the same tomorrow). -- Grant Edwards grant.b.edwardsYow! What I want to find at out is -- do parrots know gmail.commuch about Astro-Turf? -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
2013/6/18 Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de
andrea crotti andrea.crotti.0 at gmail.com writes:
Using a CouchDB server we have a different database object potentially
for
every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it once
in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Now it kinda of works already with the decorator below.
The problem is that the argument is positional so I end up maybe passing
it twice.
So I have to enforce that 'db' if there is passed as first argument..
It would be a lot easier removing the db from the arguments but then it
would look too magic and I didn't want to change the signature.. any other
advice?
def with_optional_db(func):
Decorator that sets the database to the global current one if
not passed in or if passed in and None
at wraps(func)
def _with_optional_db(*args, **kwargs):
func_args = func.func_code.co_varnames
db = None
# if it's defined in the first elements it needs to be
# assigned to *args, otherwise to kwargs
if 'db' in func_args:
assert 'db' == func_args[0], Needs to be the first defined
else:
db = kwargs.get('db', None)
if db is None:
kwargs['db'] = get_current_db()
assert kwargs['db'] is not None, Can't have a not defined
database
ret = func(*args, **kwargs)
return ret
return _with_optional_db
I'm not sure, whether your code would work. I get the logic for the db in
kwargs case, but why are you checking whether db is in func_args? Isn't the
real question whether it's in args ?? In general, I don't understand why
you
want to use .func_code.co_varnames here. You know how you defined your
function (or rather method):
class Entity:
def save_doc(db):
...
Maybe I misunderstood the problem?
Wolfgang
--
http://mail.python.org/mailman/listinfo/python-list
Well the point is that I could allow someone to not use db as argument of
the function if he only wants to use the global db object..
Or at least I want to check that it's the first argument and not in another
position, just as a sanity check.
I might drop some magic and make it a bit simpler though, even the default
argument DEFAULT_DB could be actually good, and I would not even need the
decorator at that point..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On 2013-06-18, Antoon Pardon antoon.par...@rece.vub.ac.be wrote: Op 17-06-13 19:56, ru...@yahoo.com schreef: I don't see that much change in his style. He just admitted not reading help files (because they are too technical for him). So essentialy he is asking we give him a beginners tutorial in everything he doesn't understand without much effort of him trying to understand things on his own and without much appreciation for the time of others. See my reply to ChrisA. Your reply doesn't address his unwillingness to read the documentation which was IMO rather apparant. It's not only apparent, he explicitly stated that he refused to go read the references he has been provided because he prefers to have his questions answered by a live persion. IMO, anybody who behaves like doesn't deserve any more responses. -- Grant Edwards grant.b.edwardsYow! I guess you guys got at BIG MUSCLES from doing too gmail.commuch STUDYING! -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
andrea crotti andrea.crotti.0 at gmail.com writes:
2013/6/18 Wolfgang Maier wolfgang.maier at biologie.uni-freiburg.de
andrea crotti andrea.crotti.0 at gmail.com writes:
Using a CouchDB server we have a different database object potentially for
every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it once
in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Now it kinda of works already with the decorator below.
The problem is that the argument is positional so I end up maybe passing
it twice.
So I have to enforce that 'db' if there is passed as first argument..
It would be a lot easier removing the db from the arguments but then it
would look too magic and I didn't want to change the signature.. any other
advice?
def with_optional_db(func):
Decorator that sets the database to the global current one if
not passed in or if passed in and None
at wraps(func) def _with_optional_db(*args, **kwargs):
func_args = func.func_code.co_varnames
db = None
# if it's defined in the first elements it needs to be
# assigned to *args, otherwise to kwargs
if 'db' in func_args:
assert 'db' == func_args[0], Needs to be the first defined
else:
db = kwargs.get('db', None)
if db is None:
kwargs['db'] = get_current_db()
assert kwargs['db'] is not None, Can't have a not defined database
ret = func(*args, **kwargs)
return ret
return _with_optional_db
I'm not sure, whether your code would work. I get the logic for the db in
kwargs case, but why are you checking whether db is in func_args? Isn't the
real question whether it's in args ?? In general, I don't understand why you
want to use .func_code.co_varnames here. You know how you defined your
function (or rather method):class Entity:
def save_doc(db):
...
Maybe I misunderstood the problem?Wolfgang
--http://mail.python.org/mailman/listinfo/python-list
Well the point is that I could allow someone to not use db as argument
of the function if he only wants to use the global db object..
Or at least I want to check that it's the first argument and not in
another position, just as a sanity check.
I might drop some magic and make it a bit simpler though, even the default
argument DEFAULT_DB could be actually good, and I would not even need the
decorator at that point..
Ok, if you really just that you want to have an optional argument then
providing a default argument for db would certainly be the standard and most
readable solution. Of course, if you don't want to change your method and
you would like to keep db as first argument that's not an option (except if
you make all other arguments optional too). Maybe it would help if you could
provide the full parameter list for your method.
Still, from your description I think your current code is wrong. Have you
tried it without passing in db as a keyword argument? I guess in that case
your code just checks that the first parameter in your save_doc method is
called db, which it always is, then executes your kwargs['db'] =
get_current_db() assignment, then it will try to pass db twice (once as
positional argument and once in kwargs) and still fail.
The problem is that with .func_code.co_varnames you aren't checking whether
db is passed as first argument, but whether your method defines it as the
first parameter. To see what's getting passed you have to check args!
Best,
Wolfgang
--
http://mail.python.org/mailman/listinfo/python-list
dynamic if statement
I am new to python and struggling with creating a dynamic if statement. I have a set of queries that are run against various databases/tables. The result is all the same in that I always get back the same field names. What I want to do is total the results differently based on the table. so for instance I query fld1, fld2, fld3, qty, qty2 from table1 then I loop thru the results if fld1 = 'a' add qty to some_total1 I query fld1, fld2, fld3, qty, qty2 from table2 then I loop thru the results if fld2 = 'b' add qty to some_total1 I query fld1, fld2, fld3, qty, qty2 from table3 then I loop thru the results if fld3 = 'c' add qty2 to some_total1 I created a database pair that contains (table1,fld1 = 'a',add qty to some_total1) (table2,fld2 = 'b',qty to some_total1) (table3,fld3 = 'c',qty2 to some_total1) So I know which table I am using, I query my list pair but I cant see how to build the if statement using the results from the query. something like this would be the result var1 = fld1 = 'a' result = add qty to some_total1 if var1: result -- http://mail.python.org/mailman/listinfo/python-list
weird behavior. bug perhaps?
Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
On Jun 18, 7:23 pm, zoom z...@yahoo.com wrote: Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? [Never used scipy so pls excuse if I am off...] Given list m, in the class you are doing m - matrix(m) which you dont seem to be doing in the interpreter. -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
Note that print [shape(m)[1],1] just prints a list with two elements where the first element is shape(m)[1] and the second is the number 1 (regardless of the value of m). I'm pretty sure that's not what you want. 2013/6/18 zoom z...@yahoo.com Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(**unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)**[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . --**--**-- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)**[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)**[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? -- http://mail.python.org/**mailman/listinfo/python-listhttp://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: dynamic if statement
On 2013-06-18 07:10, upperdec...@gmail.com wrote: I have a set of queries that are run against various databases/tables. The result is all the same in that I always get back the same field names. I query fld1, fld2, fld3, qty, qty2 from table1 then I loop thru the results if fld1 = 'a' add qty to some_total1 ... I created a database pair that contains (table1,fld1 = 'a',add qty to some_total1) (table2,fld2 = 'b',qty to some_total1) (table3,fld3 = 'c',qty2 to some_total1) Given the data-structure you have, and that it's hard-coded (rather than able to take dynamic/dangerous user input) for the table-name, I'd do something like this (untested) for tablename, compare_field, compare_value, source_field in ( (table1, fld1, a, qty), (table2, fld2, b, qty), (table3, fld3, c, qty2), ): # using string-building rather than escaping because # 1) we know the tablenames are hard-coded/safe from above, and # 2) this sort of escaping often chokes query parsers query = SELECT fld1, fld2, fld3, qty, qty2 from %s % tablename cursor.execute(query) name_index_map = dict( (info[0], i) for info, i in enumerate(cursor.description) ) for row in cursor.fetchall(): db_value = row[name_index_map[compare_field]] if db_value == compare_value: addend = row[name_index_map[source_field]] some_total_1 += addend -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
On 2013-06-18 15:23, zoom wrote: Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? As rusi noted, the difference between your two snippets is that in one, you converted the list of lists to a matrix object in your test suite but not in your interactive session. Most numpy functions like mean() will convert their arguments to regular numpy.ndarray objects rather than matrix objects. matrix is a subclass of ndarray that adds special behavior: in particular, operations on matrix objects retain their 2D-ness even when an ndarray would flatten down to a 1D array. [~] |1 import numpy as np [~] |2 m = [[1,2],[3,4],[3,4],[3,4]] [~] |3 a = np.array(m) [~] |4 b = np.matrix(m) [~] |5 np.mean(a, axis=1) array([ 1.5, 3.5, 3.5, 3.5]) [~] |6 np.mean(b, axis=1) matrix([[ 1.5], [ 3.5], [ 3.5], [ 3.5]]) [~] |7 np.mean(a, axis=1).shape (4,) [~] |8 np.mean(b, axis=1).shape (4, 1) This will propagate through the rest of your computation. Personally, I recommend avoiding the matrix type. It causes too many problems. Stick to plain ndarrays. You will probably want to ask further numpy questions on the numpy-discussion mailing list: http://www.scipy.org/scipylib/mailing-lists.html -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: dynamic if statement
On 18/06/2013 15:56, Tim Chase wrote: On 2013-06-18 07:10, upperdec...@gmail.com wrote: I have a set of queries that are run against various databases/tables. The result is all the same in that I always get back the same field names. I query fld1, fld2, fld3, qty, qty2 from table1 then I loop thru the results if fld1 = 'a' add qty to some_total1 ... I created a database pair that contains (table1,fld1 = 'a',add qty to some_total1) (table2,fld2 = 'b',qty to some_total1) (table3,fld3 = 'c',qty2 to some_total1) Given the data-structure you have, and that it's hard-coded (rather than able to take dynamic/dangerous user input) for the table-name, I'd do something like this (untested) for tablename, compare_field, compare_value, source_field in ( (table1, fld1, a, qty), (table2, fld2, b, qty), (table3, fld3, c, qty2), ): # using string-building rather than escaping because # 1) we know the tablenames are hard-coded/safe from above, and # 2) this sort of escaping often chokes query parsers query = SELECT fld1, fld2, fld3, qty, qty2 from %s % tablename cursor.execute(query) name_index_map = dict( (info[0], i) for info, i in enumerate(cursor.description) Looks like this should be :- for i, info in enumerate(cursor.description) ) for row in cursor.fetchall(): db_value = row[name_index_map[compare_field]] if db_value == compare_value: addend = row[name_index_map[source_field]] some_total_1 += addend -tkc -- Steve is going for the pink ball - and for those of you who are watching in black and white, the pink is next to the green. Snooker commentator 'Whispering' Ted Lowe. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
On 06/18/2013 04:27 PM, rusi wrote: On Jun 18, 7:23 pm, zoomz...@yahoo.com wrote: Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? [Never used scipy so pls excuse if I am off...] Given list m, in the class you are doing m - matrix(m) which you dont seem to be doing in the interpreter. yes, that's the thing. thanks a lot FYI this happens because shape(mean(m,1)) (4, 1) shape(mean(array(m),1)) (4,) thanks again -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
On 06/18/2013 05:25 PM, Robert Kern wrote: On 2013-06-18 15:23, zoom wrote: Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via python test.py command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: m = [[1,2],[3,4],[3,4],[3,4]] from scipy import matrix, tile, mean, shape print shape(m) (4, 2) print [shape(m)[1],1] [2, 1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? As rusi noted, the difference between your two snippets is that in one, you converted the list of lists to a matrix object in your test suite but not in your interactive session. Most numpy functions like mean() will convert their arguments to regular numpy.ndarray objects rather than matrix objects. matrix is a subclass of ndarray that adds special behavior: in particular, operations on matrix objects retain their 2D-ness even when an ndarray would flatten down to a 1D array. [~] |1 import numpy as np [~] |2 m = [[1,2],[3,4],[3,4],[3,4]] [~] |3 a = np.array(m) [~] |4 b = np.matrix(m) [~] |5 np.mean(a, axis=1) array([ 1.5, 3.5, 3.5, 3.5]) [~] |6 np.mean(b, axis=1) matrix([[ 1.5], [ 3.5], [ 3.5], [ 3.5]]) [~] |7 np.mean(a, axis=1).shape (4,) [~] |8 np.mean(b, axis=1).shape (4, 1) This will propagate through the rest of your computation. Personally, I recommend avoiding the matrix type. It causes too many problems. Stick to plain ndarrays. You will probably want to ask further numpy questions on the numpy-discussion mailing list: http://www.scipy.org/scipylib/mailing-lists.html didn't see it earlier. thanks. so ndarrays, you say... -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
On 6/18/2013 5:47 AM, andrea crotti wrote:
Using a CouchDB server we have a different database object potentially
for every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it
once in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
If save_doc does not use an instance of Entity (self) or Entity itself
(cls), it need not be put in the class.
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Decorators are only worthwhile if used repeatedly. What you specified
can easily be written, for instance, as
def save_doc(db=None):
if db is None:
db = fetch_from_global()
if isinstance(db, dbclass):
save_it()
else:
raise ValueError('need dbobject')
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: dynamic if statement
On 2013-06-18 16:27, Mark Lawrence wrote: On 18/06/2013 15:56, Tim Chase wrote: name_index_map = dict( (info[0], i) for info, i in enumerate(cursor.description) Looks like this should be :- for i, info in enumerate(cursor.description) Doh, indeed, you're correct. As forewarned though, it *was* completely untested, so anything remotely approaching working code is merely a tribute to how easy it is to write Python that actually runs. :-) -tkc -- http://mail.python.org/mailman/listinfo/python-list
Why is regex so slow?
I've got a 170 MB file I want to search for lines that look like:
[2010-10-20 16:47:50.339229 -04:00] INFO (6): songza.amie.history - ENQUEUEING:
/listen/the-station-one
This code runs in 1.3 seconds:
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
m = pattern.search(line)
if m:
count += 1
print count
--
If I add a pre-filter before the regex, it runs in 0.78 seconds (about
twice the speed!)
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
if 'ENQ' not in line:
continue
m = pattern.search(line)
if m:
count += 1
print count
--
Every line which contains 'ENQ' also matches the full regex (61425
lines match, out of 2.1 million total). I don't understand why the
first way is so much slower.
Once the regex is compiled, you should have a state machine pattern
matcher. It should be O(n) in the length of the input to figure out
that it doesn't match as far as ENQ. And that's exactly how long it
should take for if 'ENQ' not in line to run as well. Why is doing
twice the work also twice the speed?
I'm running Python 2.7.3 on Ubuntu Precise, x86_64.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
I don't understand why the first way is so much slower. I have no obvious answers, but a couple suggestions: 1. Can you anchor the pattern at the beginning of the line? (use match() instead of search()) 2. Does it get faster it you eliminate the (.*) part of the pattern? It seems that if you find a line matching the first part of the pattern, you could just as easily split the line yourself instead of creating a group. Skip -- http://mail.python.org/mailman/listinfo/python-list
os.putenv() has no effect
Hi group, I've tracked down a bug in my application to a rather strange phaenomenon: os.putenv() doesn't seem to have any effect on my platform (x86-64 Gentoo Linux, Python 3.2.3): os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.putenv(PATH, /) os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.getenv(FOO) os.putenv(FOO, BAR) os.getenv(FOO) Does anybody know why this would happen or what I could be doing wrong? Help is greatly appreciated. Thanks in advance, Johannes -- Wo hattest Du das Beben nochmal GENAU vorhergesagt? Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Jun 18, 2013, at 1:01 PM, Skip Montanaro wrote: I don't understand why the first way is so much slower. I have no obvious answers, but a couple suggestions: 1. Can you anchor the pattern at the beginning of the line? (use match() instead of search()) That's one of the things we tried. Didn't make any difference. 2. Does it get faster it you eliminate the (.*) part of the pattern? Just tried that, it also didn't make any difference. It seems that if you find a line matching the first part of the pattern, you could just as easily split the line yourself instead of creating a group. At this point, I'm not so much interested in making this faster as understanding why it's so slow. I'm tempted to open this up as a performance bug against the regex module (which I assume will be rejected, at least for the 2.x series). --- Roy Smith r...@panix.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Wed, Jun 19, 2013 at 3:08 AM, Roy Smith r...@panix.com wrote: I'm tempted to open this up as a performance bug against the regex module (which I assume will be rejected, at least for the 2.x series). Yeah, I'd try that against 3.3 before opening a performance bug. Also, it's entirely possible that performance is majorly different in 3.x anyway, on account of strings being Unicode. Definitely merits another look imho. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
2013/6/18 Terry Reedy tjre...@udel.edu
On 6/18/2013 5:47 AM, andrea crotti wrote:
Using a CouchDB server we have a different database object potentially
for every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it
once in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
If save_doc does not use an instance of Entity (self) or Entity itself
(cls), it need not be put in the class.
I missed a self it's a method actually..
...
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
Decorators are only worthwhile if used repeatedly. What you specified can
easily be written, for instance, as
def save_doc(db=None):
if db is None:
db = fetch_from_global()
if isinstance(db, dbclass):
save_it()
else:
raise ValueError('need dbobject')
Yes that's exactly why I want a decorator, to avoid all this boilerplate
for every function method that uses a db object..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 18/06/2013 17:45, Roy Smith wrote:
I've got a 170 MB file I want to search for lines that look like:
[2010-10-20 16:47:50.339229 -04:00] INFO (6): songza.amie.history - ENQUEUEING:
/listen/the-station-one
This code runs in 1.3 seconds:
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
m = pattern.search(line)
if m:
count += 1
print count
--
If I add a pre-filter before the regex, it runs in 0.78 seconds (about
twice the speed!)
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
if 'ENQ' not in line:
continue
m = pattern.search(line)
if m:
count += 1
print count
--
Every line which contains 'ENQ' also matches the full regex (61425
lines match, out of 2.1 million total). I don't understand why the
first way is so much slower.
Once the regex is compiled, you should have a state machine pattern
matcher. It should be O(n) in the length of the input to figure out
that it doesn't match as far as ENQ. And that's exactly how long it
should take for if 'ENQ' not in line to run as well. Why is doing
twice the work also twice the speed?
I'm running Python 2.7.3 on Ubuntu Precise, x86_64.
I'd be interested in how the 'regex' module
(http://pypi.python.org/pypi/regex) compares. :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 18/06/2013 18:08, Roy Smith wrote: On Jun 18, 2013, at 1:01 PM, Skip Montanaro wrote: I don't understand why the first way is so much slower. I have no obvious answers, but a couple suggestions: 1. Can you anchor the pattern at the beginning of the line? (use match() instead of search()) That's one of the things we tried. Didn't make any difference. 2. Does it get faster it you eliminate the (.*) part of the pattern? Just tried that, it also didn't make any difference. It seems that if you find a line matching the first part of the pattern, you could just as easily split the line yourself instead of creating a group. At this point, I'm not so much interested in making this faster as understanding why it's so slow. I'm tempted to open this up as a performance bug against the regex module (which I assume will be rejected, at least for the 2.x series). --- Roy Smith r...@panix.com Out of curiousity have the tried the new regex module from pypi rather than the stdlib version? A heck of a lot of work has gone into it see http://bugs.python.org/issue2636 -- Steve is going for the pink ball - and for those of you who are watching in black and white, the pink is next to the green. Snooker commentator 'Whispering' Ted Lowe. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On 18.06.2013 19:24, inq1ltd wrote: if you are trying to add a dir to a linux path you need to understand how to add or change environment variables. Yeah, about this; I actually am fully aware of what I'm doing. research this; $ export PATH= $PATH: ???/???/??? You really couldn't have missed the point more if you tried. I'm not working with Bash, but with Python. os.putenv() has no effect in my case (not just with the PATH environment variable). Regards, Johannes -- http://mail.python.org/mailman/listinfo/python-list
multiprocessing pipes with custom pickler
Hi, I need inter-process communication in Python, and was looking at the documentation here: http://docs.python.org/2/library/multiprocessing.html I am using a custom pickler, though, in order to deal with some objects that are not serialize-able through the built-in pickler. Is there any way to tell the pipe's send method to use my pickler? I could also just send my already-pickled binary data using the existing send method, but pickling/unpickling twice seems like a hack. Maybe the send_bytes method would be the best option, if it doesn't pickle the data? thanks for the help, imran -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On Tuesday, June 18, 2013 06:49:31 PM Johannes Bauer wrote: Hi group, I've tracked down a bug in my application to a rather strange phaenomenon: os.putenv() doesn't seem to have any effect on my platform (x86-64 Gentoo Linux, Python 3.2.3): os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/b in' os.putenv(PATH, /) os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/b in' os.getenv(FOO) os.putenv(FOO, BAR) os.getenv(FOO) Does anybody know why this would happen or what I could be doing wrong? Help is greatly appreciated. Thanks in advance, Johannes if you are trying to add a dir to a linux path you need to understand how to add or change environment variables. research this; $ export PATH= $PATH: ???/???/??? jd inqvista.com -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On 06/18/2013 12:49 PM, Johannes Bauer wrote: Hi group, I've tracked down a bug in my application to a rather strange phaenomenon: os.putenv() doesn't seem to have any effect on my platform (x86-64 Gentoo Linux, Python 3.2.3): os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.putenv(PATH, /) os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.getenv(FOO) os.putenv(FOO, BAR) os.getenv(FOO) Does anybody know why this would happen or what I could be doing wrong? Help is greatly appreciated. Quoting (retyping) from the getenv docs, ...however, calls to putenv() don't update os.environ, so it is actually preferable to assign to items of os.environ. As to why, I'm not at all sure. Only that many environments don't support putenv(). But why that should stop it working in the obvious way ? No idea. os.environ is not an ordinary dict, it's a mapping object. And among other things, when you modify os.environ, Python will call putenv. Quoting from the os.environ docs, If the platform supports the putenv() function, this mapping may be used to modify the environment. putenv() will be called automatically wehn the mapping is modified. In other words, you shouldn't use putenv(), but instead modify os.environ. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On 18.06.2013 20:09, Dave Angel wrote: In other words, you shouldn't use putenv(), but instead modify os.environ. Huh... this is surprising to me. Because I actually looked it up in the manual and vaguely remember that there stood that os.environ is just a copy of the environment variables at interpreter start and not authorative and (more importantly) that assigning to that dict would/could somehow lead to memory leaks. I am extremely certain that I found that passage, but can't find it right now anymore (probably staring right at it and can't find it still) :-/ Anyways, I'll give it a shot, thank you! Best regards, Joe -- Wo hattest Du das Beben nochmal GENAU vorhergesagt? Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On 18.06.2013 20:12, Johannes Bauer wrote: I am extremely certain that I found that passage, but can't find it right now anymore (probably staring right at it and can't find it still) :-/ Obviously, yes: Note On some platforms, including FreeBSD and Mac OS X, setting environ may cause memory leaks. Refer to the system documentation for putenv. But still, setting os.environ works, so I'll go with that. Weird that I read the memory leak part but missed the obvious so it is actually preferable to assign to items of os.environ. Thanks again, Best regards, Johannes -- Wo hattest Du das Beben nochmal GENAU vorhergesagt? Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org -- http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
Does anybody know why this would happen or what I could be doing wrong? os.putenv will only affect the environment in subprocesses. Consider this session fragment: % python Python 2.7.2 (default, Oct 17 2012, 03:11:33) [GCC 4.4.6 [TWW]] on sunos5 Type help, copyright, credits or license for more information. import os os.putenv(PATH, /tmp) os.system(/usr/bin/env) ... PATH=/tmp ... Skip -- http://mail.python.org/mailman/listinfo/python-list
collecting variable assignments through settrace
Hi, I'm writing a custom profiler that uses sys.settrace. I was wondering if there was any way of tracing the assignments of variables inside a function as its executed, without looking at locals() at every single line and comparing them to see if anything has changed. Sort of like xdebug's collect_assignments parameter in PHP. thanks, imran -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 18.06.2013 19:20, Chris Angelico wrote:
Yeah, I'd try that against 3.3 before opening a performance bug. Also,
it's entirely possible that performance is majorly different in 3.x
anyway, on account of strings being Unicode. Definitely merits another
look imho.
Hmmm, at least Python 3.2 seems to have the same issue. I generated test
data with:
#!/usr/bin/python3
import random
random.seed(0)
f = open(error.log, w)
for i in range(150):
q = random.randint(0, 99)
if q == 0:
print(ENQUEUEING: /listen/ fhsduifhsd uifhuisd hfuisd hfuihds
iufhsd, file = f)
else:
print(fiosdjfoi sdmfio sdmfio msdiof msdoif msdoimf oisd
mfoisdm f,
file = f)
Resulting file has a size of 91530018 and md5 of
2d20c3447a0b51a37d28126b8348f6c5 (just to make sure we're on the same
page because I'm not sure the PRNG is stable across Python versions).
Testing with:
#!/usr/bin/python3
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
# if 'ENQ' not in line:
# continue
m = pattern.search(line)
if m:
count += 1
print(count)
The pre-check version is about 42% faster in my case (0.75 sec vs. 1.3
sec). Curious. This is Python 3.2.3 on Linux x86_64.
Regards,
Johannes
--
Wo hattest Du das Beben nochmal GENAU vorhergesagt?
Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
- Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
In article mailman.3549.1371576854.3114.python-l...@python.org, Mark Lawrence breamore...@yahoo.co.uk wrote: Out of curiousity have the tried the new regex module from pypi rather than the stdlib version? A heck of a lot of work has gone into it see http://bugs.python.org/issue2636 I just installed that and gave it a shot. It's *slower* (and, much higher variation from run to run). I'm too exhausted fighting with OpenOffice to get this into some sane spreadsheet format, so here's the raw timings: Built-in re module: 0:01.32 0:01.33 0:01.32 0:01.33 0:01.35 0:01.32 0:01.35 0:01.36 0:01.33 0:01.32 regex with flags=V0: 0:01.66 0:01.53 0:01.51 0:01.47 0:01.81 0:01.58 0:01.78 0:01.57 0:01.64 0:01.60 regex with flags=V1: 0:01.53 0:01.57 0:01.65 0:01.61 0:01.83 0:01.82 0:01.59 0:01.60 0:01.55 0:01.82 -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Tuesday, June 18, 2013 11:45:29 AM UTC-5, Roy Smith wrote:
I've got a 170 MB file I want to search for lines that look like:
[2010-10-20 16:47:50.339229 -04:00] INFO (6): songza.amie.history -
ENQUEUEING: /listen/the-station-one
This code runs in 1.3 seconds:
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
m = pattern.search(line)
if m:
count += 1
print count
Is the power of regexps required to solve such a simplistic problem? I believe
string methods should suffice.
py line = [2010-10-20 16:47:50.339229 -04:00] INFO (6): songza.amie.history -
ENQUEUEING: /listen/the-station-one
py idx = line.find('ENQ')
py if idx 0:
match = line[idx:]
py match
'ENQUEUEING: /listen/the-station-one'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Tuesday, June 18, 2013 2:10:16 PM UTC-4, Johannes Bauer wrote: Resulting file has a size of 91530018 and md5 of 2d20c3447a0b51a37d28126b8348f6c5 (just to make sure we're on the same page because I'm not sure the PRNG is stable across Python versions). If people want to test against my original data, grab https://s3.amazonaws.com/songza/recruiting/logs.tar.gz and do cat app[12].error.log error.log. -- http://mail.python.org/mailman/listinfo/python-list
Oportunidade: Desenvolvedor Python
Fabrica de software localizada na Barra da Tijuca contrata: Desenvolvedor Python Objetivo geral da Posição: Desenvolvimento de sistemas Web com Python/Django, HTML5, Javascript e CSS. Prérequisitos: Experiência com Python/Django ou outro framework MVC. Familiarizado com desenvolvimento frontend: padrões Web, (X)HTML, HTML5, CSS, javas (jQuery). Conhecimentos de linguagem SQL e Stored Procedures Conhecimentos de bancos de dados MySQL / Postgresql Conhecimentos de padrões de projeto (design patterns) Interessante como diferenciais: Computer Vision (OpenCV) Outras linguagens: PHP, C/C++ Desenvolvimento para smartphones: iOS e Android TDD Agile development (Scrum, XP, etc.) SO Linux/Unix like Desenvolvimento para Android e iOS Bancos de dados NoSQL (ex: MongoDB) API do Facebook / Twitter Formação: Nenhuma formação específica é obrigatória, entretanto podem ser considerado um diferencial. Deveres: Desenvolver seguindo o padrão de qualidade esperado, manter sincronia com a equipe, estima cumprir prazos de entregas. A empresa oferece salário compatível com o mercado + Benefícios (Assistência Médica + Assistência Odontológica + Bônus por Resultado + Vale Transporte). Os interessados deverão enviar o CV para kgar...@arpexcapital.com.br , mencionando no assunto Desenvolvedor Python. -- http://mail.python.org/mailman/listinfo/python-list
Oportunidade: Desenvolvedor Python
Fabrica de software localizada na Barra da Tijuca contrata: Desenvolvedor Python Objetivo geral da Posição: Desenvolvimento de sistemas Web com Python/Django, HTML5, Javascript e CSS. Prérequisitos: Experiência com Python/Django ou outro framework MVC. Familiarizado com desenvolvimento frontend: padrões Web, (X)HTML, HTML5, CSS, javas (jQuery). Conhecimentos de linguagem SQL e Stored Procedures Conhecimentos de bancos de dados MySQL / Postgresql Conhecimentos de padrões de projeto (design patterns) Interessante como diferenciais: Computer Vision (OpenCV) Outras linguagens: PHP, C/C++ Desenvolvimento para smartphones: iOS e Android TDD Agile development (Scrum, XP, etc.) SO Linux/Unix like Desenvolvimento para Android e iOS Bancos de dados NoSQL (ex: MongoDB) API do Facebook / Twitter Formação: Nenhuma formação específica é obrigatória, entretanto podem ser considerado um diferencial. Deveres: Desenvolver seguindo o padrão de qualidade esperado, manter sincronia com a equipe, estima cumprir prazos de entregas. A empresa oferece salário compatível com o mercado + Benefícios (Assistência Médica + Assistência Odontológica + Bônus por Resultado + Vale Transporte). Os interessados deverão enviar o CV para kgar...@arpexcapital.com.br , mencionando no assunto Desenvolvedor Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 18/06/2013 20:21, Roy Smith wrote: In article mailman.3549.1371576854.3114.python-l...@python.org, Mark Lawrence breamore...@yahoo.co.uk wrote: Out of curiousity have the tried the new regex module from pypi rather than the stdlib version? A heck of a lot of work has gone into it see http://bugs.python.org/issue2636 I just installed that and gave it a shot. It's *slower* (and, much higher variation from run to run). I'm too exhausted fighting with OpenOffice to get this into some sane spreadsheet format, so here's the raw timings: Built-in re module: 0:01.32 0:01.33 0:01.32 0:01.33 0:01.35 0:01.32 0:01.35 0:01.36 0:01.33 0:01.32 regex with flags=V0: 0:01.66 0:01.53 0:01.51 0:01.47 0:01.81 0:01.58 0:01.78 0:01.57 0:01.64 0:01.60 regex with flags=V1: 0:01.53 0:01.57 0:01.65 0:01.61 0:01.83 0:01.82 0:01.59 0:01.60 0:01.55 0:01.82 I reckon that about 1/3 of that time is spent in PyArg_ParseTupleAndKeywords, just getting the arguments! There's a higher initial overhead in using regex than string methods, so working just a line at time will take longer. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
* Johannes Bauer wrote:
The pre-check version is about 42% faster in my case (0.75 sec vs. 1.3
sec). Curious. This is Python 3.2.3 on Linux x86_64.
A lot of time is spent with dict lookups (timings at my box, Python 3.2.3)
in your inner loop (150 times...)
#!/usr/bin/python3
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
#if 'ENQ' not in line:
#continue
m = pattern.search(line)
if m:
count += 1
print(count)
runs ~ 1.39 s
replacing some dict lookups with index lookups:
#!/usr/bin/python3
def main():
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
#if 'ENQ' not in line:
#continue
m = pattern.search(line)
if m:
count += 1
print(count)
main()
runs ~ 1.15s
and further:
#!/usr/bin/python3
def main():
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
search = pattern.search
count = 0
for line in open('error.log'):
#if 'ENQ' not in line:
#continue
m = search(line)
if m:
count += 1
print(count)
main()
runs ~ 1.08 s
and for reference:
#!/usr/bin/python3
def main():
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
search = pattern.search
count = 0
for line in open('error.log'):
if 'ENQ' not in line:
continue
m = search(line)
if m:
count += 1
print(count)
main()
runs ~ 0.71 s
The final difference is probably just the difference between a hardcoded
string search and a generic NFA.
nd
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
Roy Smith roy at panix.com writes: Every line which contains 'ENQ' also matches the full regex (61425 lines match, out of 2.1 million total). I don't understand why the first way is so much slower. One invokes a fast special-purpose substring searching routine (the str.__contains__ operator), the other a generic matching engine able to process complex patterns. It's hardly a surprise for the specialized routine to be faster. That's like saying I don't understand why my CPU is slower than my GPU at calculating 3D structures. That said, there may be ways to improve the regex engine to deal with such special cases in a speedier way. But there will still be some overhead related to the fact that you are invoking a powerful generic engine rather than a lean and mean specialized routine. (to be fair, on CPython there's also the fact that operators are faster than method calls, so some overhead is added by that too) Once the regex is compiled, you should have a state machine pattern matcher. It should be O(n) in the length of the input to figure out that it doesn't match as far as ENQ. And that's exactly how long it should take for if 'ENQ' not in line to run as well. You should read again on the O(...) notation. It's an asymptotic complexity, it tells you nothing about the exact function values at different data points. So you can have two O(n) routines, one of which always twice faster than the other. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
* André Malo wrote:
* Johannes Bauer wrote:
The pre-check version is about 42% faster in my case (0.75 sec vs. 1.3
sec). Curious. This is Python 3.2.3 on Linux x86_64.
A lot of time is spent with dict lookups (timings at my box, Python 3.2.3)
in your inner loop (150 times...)
[...]
Missed one basic timing BTW:
#!/usr/bin/python3
def main():
for line in open('error.log'):
pass
main()
runs ~ 0.53 s
nd
--
Already I've seen people (really!) write web URLs in the form:
http:\\some.site.somewhere
[...] How soon until greengrocers start writing apples $1\pound
or something? -- Joona I Palaste in clc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Tuesday, June 18, 2013 4:05:25 PM UTC-4, Antoine Pitrou wrote:
One invokes a fast special-purpose substring searching routine (the
str.__contains__ operator), the other a generic matching engine able to
process complex patterns. It's hardly a surprise for the specialized routine
to be faster.
Except that the complexity in regexes is compiling the pattern down to a FSM.
Once you've got the FSM built, the inner loop should be pretty quick. In C, the
inner loop for executing a FSM should be something like:
for(char* p = input; p; ++p) {
next_state = current_state[*p];
if (next_state == MATCH) {
break;
}
}
which should compile down to a couple of machine instructions which run
entirely in the instruction pipeline cache. But I'm probably simplifying it
more than I should :-)
(to be fair, on CPython there's also the fact that operators are faster
than method calls, so some overhead is added by that too)
I've been doing some experimenting, and I'm inclined to believe this is indeed
a significant part of it. I also took some ideas from André Malo and factored
out some name lookups from the inner loop. That bummed me another 10% in speed.
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.putenv() has no effect
On 6/18/2013 12:49 PM, Johannes Bauer wrote: Hi group, I've tracked down a bug in my application to a rather strange phaenomenon: os.putenv() doesn't seem to have any effect on my platform (x86-64 Gentoo Linux, Python 3.2.3): os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.putenv(PATH, /) os.getenv(PATH) '/usr/joebin:/usr/local/bin:/usr/bin:/bin:/usr/games/bin:/usr/sbin:/sbin:~/bin' os.getenv(FOO) os.putenv(FOO, BAR) os.getenv(FOO) Does anybody know why this would happen From the doc: When putenv() is supported, assignments to items in os.environ are automatically translated into corresponding calls to putenv(); however, calls to putenv() don’t update os.environ, so it is actually preferable to assign to items of os.environ. Also Such changes to the environment affect subprocesses started with os.system(), popen() or fork() and execv() Not obvious fact: getenv gets values from the os.environ copy of the environment, which is not affected by putenv. See http://bugs.python.org/issue1159 or what I could be doing wrong? Using putenv(key, value) instead of os.environ[key] = value, which suggests that you did not read the full doc entry, which says to use the latter ;-). -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 2013-06-18, Antoine Pitrou solip...@pitrou.net wrote: Roy Smith roy at panix.com writes: You should read again on the O(...) notation. It's an asymptotic complexity, it tells you nothing about the exact function values at different data points. So you can have two O(n) routines, one of which always twice faster than the other. And you can have two O(n) routines, one of which is twice as fast for one value of n and the other is twice as fast for a different value of n (and that's true for any value of 'twice': 2X 10X 100X). All the O() tells you is the general shape of the line. It doesn't tell you where the line is or how steep the slope is (except in the case of O(1), where you do know the slope is 0. It's perfectly feasible that for the range of values of n that you care about in a particular application, there's an O(n^2) algorithm that's way faster than another O(log(n)) algorithm. [Though that becomes a lot less likely as n gets large.] -- Grant Edwards grant.b.edwardsYow! Where's SANDY DUNCAN? at gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 6/18/2013 4:30 PM, Grant Edwards wrote: On 2013-06-18, Antoine Pitrou solip...@pitrou.net wrote: Roy Smith roy at panix.com writes: You should read again on the O(...) notation. It's an asymptotic complexity, it tells you nothing about the exact function values at different data points. So you can have two O(n) routines, one of which always twice faster than the other. Or one that is a million times as fast. And you can have two O(n) routines, one of which is twice as fast for one value of n and the other is twice as fast for a different value of n (and that's true for any value of 'twice': 2X 10X 100X). All the O() tells you is the general shape of the line. It doesn't tell you where the line is or how steep the slope is (except in the case of O(1), where you do know the slope is 0. It's perfectly feasible that for the range of values of n that you care about in a particular application, there's an O(n^2) algorithm that's way faster than another O(log(n)) algorithm. In fact, Tim Peters put together two facts to create the current list.sort. 1. O(n*n) binary insert sort is faster than O(n*logn) merge sort, with both competently coded in C, for n up to about 64. Part of the reason is that binary insert sort is actually O(n*logn) (n binary searches) + O(n*n) (n insertions with a shift averaging n/2 items). The multiplier for the O(n*n) part is much smaller because on modern CPUs, the shift needed for the insertion is a single machine instruction. 2. O(n*logn) sorts have a lower assymtotic complexity because they divide the sequence roughly in half about logn times. In other words, they are 'fast' because they split a list into lots of little pieces. So Tim's aha moment was to think 'Lets stop splitting when pieces are less than or equal to 64, rather than splitting all the way down to 1 or 2. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: collecting variable assignments through settrace
On 6/18/2013 2:38 PM, skunkwerk wrote: Hi, I'm writing a custom profiler that uses sys.settrace. I was wondering if there was any way of tracing the assignments of variables inside a function as its executed, without looking at locals() at every single line and comparing them to see if anything has changed. The stdlib has an obscure module bdb (basic debugger) that is used in both pdb (python debugger) and idlelib.Debugger. The latter can display global and local variable names. I do not know if it does anything other than rereading globals() and locals(). It only works with a file loaded in the editor, so it potentially could read source lines to looks for name binding statements (=, def, class, import) and determine the names just bound. On the other hand, re-reading is probably fast enough for human interaction. My impression from another issue is that traceing causes the locals dict to be updated with each line, so you do not actually have to have to call locals() with each line. However, that would mean you have to make copies to compare. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: A certainl part of an if() structure never gets executed.
On 13.06.2013 20:00, Νικόλαος Κούρας wrote: if '-' not in name + month + year: cur.execute( '''SELECT * FROM works WHERE clientsID = (SELECT id FROM clients WHERE name = %s) and MONTH(lastvisit) = %s and YEAR(lastvisit) = %s ORDER BY lastvisit ASC''', (name, month, year) ) elif '-' not in name + year: cur.execute( '''SELECT * FROM works WHERE clientsID = (SELECT id FROM clients WHERE name = %s) and YEAR(lastvisit) = %s ORDER BY lastvisit ASC''', (name, year) ) elif '-' not in month + year: cur.execute( '''SELECT * FROM works WHERE MONTH(lastvisit) = %s and YEAR(lastvisit) = %s ORDER BY lastvisit ASC''', (month, year) ) elif '-' not in year: cur.execute( '''SELECT * FROM works WHERE YEAR(lastvisit) = %s ORDER BY lastvisit ASC''', year ) This finally worked! I spared myself to read the whole thread cause its super long and I find it admirable that some people took time to troubleshoot the issue and (even) added optimizations of what can be done differently/better.. even so it seems there was only a wrong char used in the ifs?.. And as a short note - you should add code which handles the worst-case scenario (everthing fails, nothing valid, avoid into code which might require variables/data which is not present and so forth..) But generally speaking: As a best practice in any language not just Python. If you have the option to print out evaluations and values, use it! As rule of thumb for debugging code: 1. Print out what values you get in (from user, in(to) a function) 2a. Test statements and logic either by setting a print (in your case inside each if) with something like if1, if2, ... else fired or what you prefer OR do a direct comparison if the evaluation statements are not super long (someone mentioned that..) 2b. In case of longer statements/calculations, break them into chunks, simplify the problem to smaller ones and try to verify the results as possible (being far apart and getting closer after each time is already a good hint in calculations..) 3. If you catch data from a database and you see now results - print out if your search/query even can return anything at all or if your query itself has a flaw or similar. Optional but also useful: 4. On the end of a function, print if the output of the function is as expected This small technique costs minimal time, but brings a brilliant ability: 1. Understand the code flow 2. Understand mistakes 3. Save yourself a big time by searching targeted for code parts which are executed.. ..and ignore not directly connected code - for example when looking at other peoples code without documentation, comments or introduction. This can spare loads of headaches. And all this can be done in a couple of minutes.. Jan -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Tue, 18 Jun 2013 12:45:29 -0400, Roy Smith wrote:
I've got a 170 MB file I want to search for lines that look like:
[2010-10-20 16:47:50.339229 -04:00] INFO (6): songza.amie.history -
ENQUEUEING: /listen/the-station-one
This code runs in 1.3 seconds:
--
import re
pattern = re.compile(r'ENQUEUEING: /listen/(.*)')
count = 0
for line in open('error.log'):
m = pattern.search(line)
if m:
count += 1
print count
--
If I add a pre-filter before the regex, it runs in 0.78 seconds (about
twice the speed!)
You say that as if it were surprising. It's not. Regex searches have
higher overhead than bog standard dumb `substring in string` tests, so
reducing the number of high-overhead regex searches calls with a low-
overhead `in` test is often a nett win.
Not always, it depends on how many hits/misses you have, and the relative
overhead of each, but as a general rule, I would always try pre-filtering
as the obvious way to optimize a regex.
Even if the regex engine is just as efficient at doing simple character
matching as `in`, and it probably isn't, your regex tries to match all
eleven characters of ENQUEUEING while the `in` test only has to match
three, ENQ.
[...]
Once the regex is compiled, you should have a state machine pattern
matcher. It should be O(n) in the length of the input to figure out
that it doesn't match as far as ENQ.
Yes, but O(N) doesn't tell you how long it takes to run, only how it
*scales*.
class MyDumbString(str):
def __contains__(self, substr):
time.sleep(1000)
return super(MyDumbString, self).__contains__(substring)
MyDumbString `in` is also O(N), but it is a wee less efficient than the
built-in version...
Regexes do a lot of work, because they are far more complex than dumb
string __contains__. It should not surprise you that the overhead is
greater, even when matching a plain-ol' substring with no metacharacters.
Especially since Python does not traditionally use regexes for
everything, like Perl does, and so has not spent the effort to complicate
the implementation in order to squeeze every last microsecond out of it.
And that's exactly how long it
should take for if 'ENQ' not in line to run as well. Why is doing
twice the work also twice the speed?
It's not doing twice the work. It's doing less work, overall, by spending
a moment to trivially filter out the lines which cannot possibly match,
before spending a few moments to do a more careful match. If the number
of non-matching lines is high, as it is in your data, then the cheaper
pre-filter pays for itself and then some.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On 06/18/2013 09:51 PM, Steven D'Aprano wrote: SNIP Even if the regex engine is just as efficient at doing simple character matching as `in`, and it probably isn't, your regex tries to match all eleven characters of ENQUEUEING while the `in` test only has to match three, ENQ. The rest of your post was valid, and useful, but there's a misconception in this paragraph; I hope you don't mind me pointing it out. In general, for simple substring searches, you can search for a large string faster than you can search for a smaller one. I'd expect if ENQUEUING in bigbuffer to be faster than if ENQ in bigbuffer assuming that all occurrences of ENQ will actually match the whole thing. If CPython's implementation doesn't show the speed difference, maybe there's some room for optimization. See Boyer-Moore if you want a peek at the algorithm. When I was writiing a simple search program, I could typically search for a 4-character string faster than REP SCASB could match a one character string. And that's a single instruction (with prefix). -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Beginner Question: 3D Models
Hi there, Sorry for the complete beginner question but I thought the readers here might be able to provide me with some guidance. I've done some programming with Visual Basic and VBA plus a little PHP, CSS and HTML. I'm looking at developing a program for work that can be distributed to others (i.e. and exe file). The application would open various dialogue boxes and ask the user for input and eventually perform mathematical calculations on the input. From what I've read Python would have no trouble with this. However, for one part of the program I'd like to be able to create a 3D model based on the user input. The model would be very basic consisting of a number of lines and objects. We have 3D models of each component within our CAD system so it would be great if we could utilize those models. Most of the 3D graphic capabilities I've seen seem to center around either gaming or mathematical plotting. Are there any capabilities to import existing CAD geometry, arrange the components in particular 3D coordinates in space and then view the results in some sort of 3D viewer? Ideally the user would then be able to zoom in and orbit around looking at the model. Is this possible? Is Python the right language? Thanks AB -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is regex so slow?
On Tue, 18 Jun 2013 22:11:01 -0400, Dave Angel wrote: On 06/18/2013 09:51 PM, Steven D'Aprano wrote: SNIP Even if the regex engine is just as efficient at doing simple character matching as `in`, and it probably isn't, your regex tries to match all eleven characters of ENQUEUEING while the `in` test only has to match three, ENQ. The rest of your post was valid, and useful, but there's a misconception in this paragraph; I hope you don't mind me pointing it out. Of course not, I'm always happy to learn if I'm mistaken. In general, for simple substring searches, you can search for a large string faster than you can search for a smaller one. I'd expect if ENQUEUING in bigbuffer to be faster than if ENQ in bigbuffer And so it is: steve@runes:~$ python2.7 -m timeit -s sub = 'ENQ' \ -s s = 'blah '*1 + 'ENQUIRING blah blah blah' \ sub in s 1 loops, best of 3: 38.3 usec per loop steve@runes:~$ python2.7 -m timeit -s sub = 'ENQUIRING' \ -s s = 'blah '*1 + 'ENQUIRING blah blah blah' \ sub in s 10 loops, best of 3: 15.4 usec per loop Thank you. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Question: 3D Models
On Tue, 18 Jun 2013 19:47:34 -0700, andrewblundon wrote: However, for one part of the program I'd like to be able to create a 3D model based on the user input. The model would be very basic consisting of a number of lines and objects. We have 3D models of each component within our CAD system so it would be great if we could utilize those models. [...] Is this possible? Is Python the right language? Is Blender the sort of thing you are looking for? https://duckduckgo.com/html/?q=blender%20python -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On 06/18/2013 02:22 AM, Antoon Pardon wrote: Op 17-06-13 19:56, ru...@yahoo.com schreef: On 06/17/2013 02:15 AM, Antoon Pardon wrote: Op 17-06-13 05:46, ru...@yahoo.com schreef: On 06/16/2013 02:04 PM, Steven D'Aprano wrote: Yes. Trying to start flame wars with Nikos is unacceptable behaviour. It is unproductive, it makes this a hostile, unpleasant place to be, it ruins the environment for the rest of the community, it's off topic, and it simply doesn't work to discourage trolls. The difficulty with trying to suppress such responses is that the flamers get just as much pleasure from having a target to unrestrainedly spew their pent up anger and vile at, as the troll gets from simulating that reaction. The result is a positive feedback loop. Well if asocial behaviour of one provokes asocial behaviour in others, you can't claim the problem is not the social behaviour of the first. Sure I can. If you have a photodetector that activates a bright light when it detects a flash, you can blame the first flash for the fact that the bright light is on all the time. Or you can say that stray flashes are to be expected now and then in the environment of this system and the fault is responding to them with a bright light. But that doesn't make sense. Your photodetector working as it does, is just as expected as the happening of stray flashes. There is no reason to differentiate between these two in terms of being expected or not. I was using the photodetector/light system as a emotion-free analog of the troll/troll-feeders positive feedback system for which you claimed it was clearly the troll's fault for initiating the feedback condition. My intent was to point out that cause and effect are intertwined in feedback systems and it is equally valid to blame those responding to the troll for the end result as to blame the troll. And, since occasional trolls are to be expected, one is even justified in putting the preponderance of blame on the responders. As I said (and you disagree with below), I did see some attempts to adapt his behavior but it is not realistic to expect immediate acquiescence to every request made here, especially given that a lot of them were/are bullshit. I don't care whether it is realistic or not. If he can't conform his behaviour in a reasonable way, he doesn't belong here. It is not realistic to expect someone who is just learing to swim to survive a jump in the deep. So we expect those people not to jump in the deep. We don't tolerate them jumping in the deep on the expectation that others will pull them out. That is wat Nikos keeps doing here, jumping in the deep. And a lot of people feel it is time we let him (metaphorically drown). see Drowning below I speculate that half of his bad behavior is simple I want now and don't care about your conventions. The rest is a reaction to we're the alphas, your a beta attitude expressed by many here and later, overt hostility directed at him. He has changed some things -- his posting method, he's made an effort to understand his encoding issues, etc.' I don't see that much change in his style. He just admitted not reading help files (because they are too technical for him). So essentialy he is asking we give him a beginners tutorial in everything he doesn't understand without much effort of him trying to understand things on his own and without much appreciation for the time of others. See my reply to ChrisA. Your reply doesn't address his unwillingness to read the documentation which was IMO rather apparant. My reply certainly did address that and did so explicitly. Now if you mean that you don't care *why* he doesn't want to read them, the only thing that matters is that he doesn't/won't, them we have different standard for evaluating people and I don't accept yours. To me the reason does matter as it affects my evaluation of how they may adapt in the future. My personal feeling is that he tends to ask on the list too quickly, but I suspect he also does more than you're giving him credit for. He seems to be naive (eg the password event), open and honest so when he says he has been trying to fix something for hours I am prone to believe him. I don't care. In the end he is still jumping in the deep expecting others to drag him out. I don't care how much he does. Just as I don't care how much energy someone has put into learning to swim. If your skills are not adequate you don't jump into the deep. see Drowning below. I think his approach to fixing is to try making changes more or less at random, in part because he doesn't understand the docs (or doesn't look at them because they haven't made sense to him in the past) and in part because he hasn't developed any skill in debugging (a skill that I think most everyone here takes for granted but which doesn't come naturally to some people) and which also accounts for the poor formulation of
Re: Don't feed the troll...
On 06/18/2013 01:21 AM, Chris Angelico wrote: On Tue, Jun 18, 2013 at 2:39 PM, alex23 wuwe...@gmail.com wrote: tl;dr Stop acting like a troll and we'll stop perceiving you as such. This being Python-list, we duck-type. You don't have to declare that you're a troll, like you would in C; you just react like a troll and we'll treat you as one. We never ask are you a troll, we just ask do you quack like a troll. People are much more complex than Python objects. While duck-typing is a useful heuristic it does not guarantee accurate results. And keep in mind that stereotyping and racial profiling are forms of duck typing. You need to be careful when duck-typing people. -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On 2013-06-18, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: On Tue, 18 Jun 2013 13:38:40 + (UTC), Grant Edwards On 2013-06-18, Chris Angelico ros...@gmail.com wrote: On Tue, Jun 18, 2013 at 2:39 PM, alex23 wuwe...@gmail.com wrote: tl;dr Stop acting like a troll and we'll stop perceiving you as such. This being Python-list, we duck-type. You don't have to declare that you're a troll, like you would in C; you just react like a troll and we'll treat you as one. We never ask are you a troll, we just ask do you quack like a troll. Indeed. The is he a troll question is a discussion about internals. And like many Python users, some of us do like to discuss questions about internals (though we hopefully know enough not to depend on the answers being the same tomorrow). And suddenly I have visions of a Druidical reading of the entrails... Well, that might explain how some of the code I've seen recently in this newsgroup was written. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: decorator to fetch arguments from global objects
On Tue, 18 Jun 2013 10:47:57 +0100, andrea crotti wrote:
Using a CouchDB server we have a different database object potentially
for every request.
We already set that db in the request object to make it easy to pass it
around form our django app, however it would be nice if I could set it
once in the API and automatically fetch it from there.
Basically I have something like
class Entity:
def save_doc(db)
...
You missed a self, and a colon :-)
I would like basically to decorate this function in such a way that:
- if I pass a db object use it
- if I don't pass it in try to fetch it from a global object
- if both don't exist raise an exception
def decorate(method):
@functools.wraps(method)
def inner(self, db=None, *args, **kwargs):
if db is None:
db = get_current_db()
return method(self, db, *args, **kwargs)
return inner
class Entity:
@decorate
def save_doc(self, db):
...
ought to do it, assuming get_current_db() raises an appropriate exception.
The usual Python argument passing rules apply. If you call the wrapped
function with positional arguments, db must be given explicitly. If you
call it with keyword arguments, it doesn't.
If you prefer, you can make db a keyword only argument:
# Python 3 version:
def decorate(method):
@functools.wraps(method)
def inner(self, *args, **kwargs, db=None):
if db is None:
db = get_current_db()
return method(self, db, *args, **kwargs)
return inner
# Python 2 version:
def decorate(method):
@functools.wraps(method)
def inner(self, *args, **kwargs):
db = kwargs.get('db', None)
try:
del kwargs['db']
except KeyError:
pass
if db is None:
db = get_current_db()
return method(self, db, *args, **kwargs)
return inner
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: python-django for dynamic web survey?
Hi guys! Please help me with your advices and ideas. I need to create a web survey that will dynamically (randomly) select questions and descriptions from a dataset, present them to users, collect their answers and store them back in the dataset. (Every user gets different set of questions randomly selested from the pool.) How random do the questions need to be? Is a handful or two different combinations sufficient? Can you manually create these different surveys using, for example, surveymonkey, and distribute them? -- Sent from Gmail Mobile -- http://mail.python.org/mailman/listinfo/python-list
Re: A certainl part of an if() structure never gets executed.
Nick the Gr33k supp...@superhost.gr wrote:
On 16/6/2013 4:55 ??, Tim Roberts wrote:
Nick the Gr33k supp...@superhost.gr wrote:
Because Python lets you use arbitrary values in a Boolean context, the net
result is exactly the same.
What is an arbitrary value? don even knwo what arbitrary means literally
in English.
Basically, it means any. In Python, you can use ANY value where a
Boolean is expected. All types have a Boolean meaning. For integers, 0 is
false, anything else is true. For strings, an empty string is false,
anything else is true. For lists, an empty list [] is false, anything else
is true. For tuples, an empty tuple () is false, anything else is true.
For dicts, an empty dict {} is false, anything else is true.
The argument being returned in an and or or expression is the one
that *determined' the evaluation of the expression.
That's not exactly how I'd put it, but the statement is correct. The last
thing it had to evaulate is the result of the expression.
And actually what's being returned is not the argument itself but the
argument's value.
But this is no different than any other programming language. Expressions
always use the value of their operands, and they always return a value.
The name vs value thing is critical to understanding Python, in my opinion,
and it can be a stumbling block when you're coming from another language.
Here's how I think about it.
Python had two distinct spaces: there is a space for names, and there is a
space for objects (which are values). Objects live in a nameless, faceless
object cloud.
A name is always bound to some object (which might be the None object). A
name always knows its object, but an object never knows what names it is
bound to.
The only things that can be used in expressions and function arguments are
objects. Names are merely the way we specify which objects to be used.
a = [3]
That creates a nameless list containing a single integer object with the
value 3. It then binds the name a to that list. Note that the list has
no clue that it is bound to any names.
b = a
That binds b to the same list. b and a are not related in any way,
except that they happen to be bound to the same object. Note that there is
still only one list.
a.append(4)
That modifies the list so that it now contains [3,4]. b is bound to the
same list, so if you
print(b)
you'll see [3,4]
Now, let's say I do this:
a = [5]
Here's where people get tripped up. This does not change our original
list. Instead, it creates a new nameless list containing 5, and binds the
name a to that list. a and b are no longer bound to the same object.
--
Tim Roberts, t...@probo.com
Providenza Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list
Re: A certainl part of an if() structure never gets executed.
I think this is an excellent description of name binding with mutable
objects. I just have one clarification to insert below.
On 06/19/2013 01:08 AM, Tim Roberts wrote:
Nick the Gr33k supp...@superhost.gr wrote:
On 16/6/2013 4:55 ??, Tim Roberts wrote:
Nick the Gr33k supp...@superhost.gr wrote:
Because Python lets you use arbitrary values in a Boolean context, the net
result is exactly the same.
What is an arbitrary value? don even knwo what arbitrary means literally
in English.
Basically, it means any. In Python, you can use ANY value where a
Boolean is expected. All types have a Boolean meaning. For integers, 0 is
false, anything else is true. For strings, an empty string is false,
anything else is true. For lists, an empty list [] is false, anything else
is true. For tuples, an empty tuple () is false, anything else is true.
For dicts, an empty dict {} is false, anything else is true.
The argument being returned in an and or or expression is the one
that *determined' the evaluation of the expression.
That's not exactly how I'd put it, but the statement is correct. The last
thing it had to evaulate is the result of the expression.
And actually what's being returned is not the argument itself but the
argument's value.
But this is no different than any other programming language. Expressions
always use the value of their operands, and they always return a value.
The name vs value thing is critical to understanding Python, in my opinion,
and it can be a stumbling block when you're coming from another language.
Here's how I think about it.
Python had two distinct spaces: there is a space for names, and there is a
space for objects (which are values). Objects live in a nameless, faceless
object cloud.
A name is always bound to some object (which might be the None object). A
name always knows its object, but an object never knows what names it is
bound to.
The only things that can be used in expressions and function arguments are
objects. Names are merely the way we specify which objects to be used.
Names are *one of* the ways we specify which objects are to be used.
(We can also specify objects via an container and a subscript or slice,
or via an attribute of another object. And probably another way or two.)
a = [3]
That creates a nameless list containing a single integer object with the
value 3. It then binds the name a to that list. Note that the list has
no clue that it is bound to any names.
b = a
That binds b to the same list. b and a are not related in any way,
except that they happen to be bound to the same object. Note that there is
still only one list.
a.append(4)
That modifies the list so that it now contains [3,4]. b is bound to the
same list, so if you
print(b)
you'll see [3,4]
Now, let's say I do this:
a = [5]
Here's where people get tripped up. This does not change our original
list. Instead, it creates a new nameless list containing 5, and binds the
name a to that list. a and b are no longer bound to the same object.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
[issue16901] In http.cookiejar.FileCookieJar() the .load() and .revert() methods don't work
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16901 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9740] Support for HTTP 1.1 persistent connections throughout the standard library
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14000] Subprocess stdin.flush does not flush
Jian Wen added the comment: The following code shows how to use pts. #!/usr/bin/env python import os import pty import shlex import time _args = /usr/bin/ssh example.com args = shlex.split(_args) pid, child_fd = pty.fork() if pid == 0: # Child os.execv(/usr/bin/ssh, args) else: # Parent while True: os.write(child_fd, '# keep alive\n') os.read(child_fd, 1024) time.sleep(2) -- nosy: +Jian.Wen ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14000 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18251] test_subprocess_jy fails when Argument has embedded quote
New submission from Sowmya: test_subprocess_jy fails with below error: test test_subprocess_jy failed -- Traceback (most recent call last): File \'L:\\apps\\ascii\\jython\\70files\\current\\win\\Lib\\test\\test_subprocess_jy.py\', line 13, in testDefaultEnvIsInherited p1 = Popen([sys.executable, \'-c\', File \'L:\\apps\\ascii\\jython\\70files\\current\\win\\Lib\\subprocess.py\', line 755, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File \'L:\\apps\\ascii\\jython\\70files\\current\\win\\Lib\\subprocess.py\', line 1269, in _execute_child raise OSError(e.getMessage() or e) OSError: Argument has embedded quote, use the explicit CMD.EXE call. Solution : This testcase needs to be changed to suit the changes mentioned in Oracle 7u21 - (http://www.oracle.com/technetwork/java/javase/7u21-relnotes-1932873.html#jruntime). Python code needs to be modified to use Cmd.exe while running commands that contains quotes in it -- files: test_subclasses_jy.py messages: 191389 nosy: sowmyalakkappa priority: normal severity: normal status: open title: test_subprocess_jy fails when Argument has embedded quote type: enhancement versions: Python 2.6 Added file: http://bugs.python.org/file30632/test_subclasses_jy.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18251 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18252] timeit makes code run faster?
New submission from icedream91: I used Python 3.3.2 to try this problem: http://projecteuler.net/problem=23 , and I got a correct answer. When I wanted to check how long it took, I found something strange: When I ran 23.py directly, it showed that it took about 13s. But if I use timeit module, it showed that it only took about 9s! I have tried these for some times, in both Ubuntu 12.10 and Windows 8, I don't understand why timeit will make the same code run faster. Thanks. -- components: Benchmarks files: 23.py messages: 191390 nosy: icedream91 priority: normal severity: normal status: open title: timeit makes code run faster? versions: Python 3.3 Added file: http://bugs.python.org/file30633/23.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18252 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18252] timeit makes code run faster?
Changes by icedream91 icedrea...@gmail.com: Removed file: http://bugs.python.org/file30633/23.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18252 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18252] timeit makes code run faster?
icedream91 added the comment: I used Python 3.3.2 to try this problem: http://projecteuler.net/problem=23 , and I got a correct answer. When I wanted to check how long it took, I found something strange: When I ran 23.py directly, it showed that it took about 13s. But if I use timeit module, it showed that it only took about 9s! I have tried these for some times, in both Ubuntu 12.10 and Windows 8, I don't understand why timeit will make the same code run faster. Thanks. -- Added file: http://bugs.python.org/file30634/23.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18252 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18253] make standard library PEP8 compliant
New submission from Friedrich Spee von Langenfeld: The modules in the standard library aren´t PEP( compliant. I´ve written a script to change this. It uses autopep8.py (must be in the path) and is written for Windows users. -- components: Library (Lib) files: autopepframework.py messages: 191392 nosy: Friedrich.Spee.von.Langenfeld priority: normal severity: normal status: open title: make standard library PEP8 compliant type: enhancement Added file: http://bugs.python.org/file30635/autopepframework.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18253 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18252] timeit makes code run faster?
Richard Oudkerk added the comment: I think if you use timeit then the code is wrapped inside a function before it is compiled. This means that your code can mostly use faster local lookups rather than global lookups. -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18252 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
