web2py 1.56 is OUT

[Massimo Di Pierro]

web2py 1.56 is out, including a new web site with better documentation

http://www.web2py.com

What is web2py?
=

- It is the web framework used by PyCon 2009 for registration.

- It a very easy and very powerful Python web framework.

- It is fast and rock solid. It has a very clean design and it is  
strong on security.


- Includes a web based development environment, maintenance  
environment, and database

administrative interface.

- Includes a Database Abstraction Layer that works with SQLite, MySQL,  
PostgreSQL, FireBird, MSSQL, Oracle, AND the Google App Engine. The  
DAL does migrations.


- Includes a pure Python based template language with no indentation  
requirements.


- Includes jQuery, simplejson, markdown, feedparser, PyRSS2, nicEdit,  
EditArea and a lot more.


- Runs on any platform that runs python. The binary versions can even  
run off a USB drive without dependences.


- The same apps can run on a PC with Oracle or on the Google App  
Engine or on Windows Mobile, without changes, including DB API.



New features Include:
=

- Authentication
- Authorization (Role Based Access Controller)
- CRUD
- portable url fetch function
- portable geocoding function
- PEP8 compliant
- Python 2.5 and Python 2.6 compliant
- Runs on Jython (although without db drivers)
- Runs on IronPython (although without CSV, db drivers and internal  
web-server)

- DAL shortcuts
- SQLFORM has default image preview
- new generic.html view
- more examples and documentation
- always backward compatible

Example of code (complete app)
=
## in model db.py
from gluon.tools import *
db=SQLDB()
db.define_table('puppy', db.Field('name'),  
db.Field('image','upload'))

auth=Auth(globals(),db)
crud=Crud(flobals(),db)

## in controller default.py
def user():
 to expose register, login, logout, etc 
return dict(form=auth())

@auth.requires_login()
def data():
  to expose select, create, update, delete, etc 
return dict(form=crud())

@auth.requires_login()
def download():
 for downloading uploaded images 
return response.download(request,db)

## in view default/user.html
{{extend 'layout.html'}}
div{{=form}}/div

## in view default/data.html
{{extend 'layout.html'}}
div{{=form}}/div

This complete code will allow to register, login, change password,  
upload images of puppies (with relative names), select puppies, edit  
records, read and preview images, enforce authorization, and more.


Thanks to all the contributors:
===

* Attila Csipa (cron job)
* Bill Ferrett (modular DAL design)
* CJ Lazell (tester)
* DenesL (validators)
* Douglas Andrade (2.6 compliance, docstrings)
* Francisco Gama (bug fixing)
* Fran Boon (authorization and authentication)
* Fred Yanowski (XHTML compliance)
* Jonathan Benn (is_url validator and tests)
* Jose Jachuf (Firebird support)
* Kyle Smith (javascript)
* Limodou (winservice)
* Marcel Leuthi (Oracle support)
* Mark Larsen (taskbar widget)
* Mark Moore (databases and daemon scripts)
* Markus Gritsch (bug fixing)
* Martin Hufsky (expressions in DAL)
* Mateusz Banach (stickers)
* Michael Willis (shell)
* Nathan Freeze (admin design)
* Niall Sweeny (MSSQL support)
* Niccolo Polo (epydoc)
* Ondrej Such (MSSQL support)
* Pai (internationalization)
* Phyo Arkar Lwin (web hosting and Jython tester)
* Robin Bhattacharyya (Google App Engine support)
* Sharriff Aina (tester and PyAMF integration)
* Sterling Hankins (tester)
* Stuart Rackham (MSSQL support)
* Telman Yusupov (Oracle support)
* Timothy Farrell (python 2.6 compliance, windows support)
* Yarko Tymciurak (design)
* Younghyun Jo (internationalization)

--
http://mail.python.org/mailman/listinfo/python-announce-list

   Support the Python Software Foundation:
   http://www.python.org/psf/donations.html

Re: programming by evolution?

[Xah Lee]

looking at the eXtreme Programing fuckheads's traffic history:
http://groups.google.com/group/comp.software.extreme-programming/about

for those who are not aware, it was one of the snake oil wildly
popular in around 2001.

  Xah
∑ http://xahlee.org/

☄


Jason wrote:
I just started reading OnLisp and hit this quote:

Why wait for hindsight? As Montaigne found, nothing clarifies your
ideas like trying to write them down. Once you’re freed from the worry
that you’ll paint yourself into a corner, you can take full advantage
of this possibility. The ability to plan programs as you write them
has two momentous consequences: programs take less time to write,
because when you plan and write at the same time, you have a real
program to focus your attention; and they turn out better, because the
final design is always a product of evolution. So long as you maintain
a certain discipline while searching for your program’s destiny—so
long as you always rewrite mistaken parts as soon as it becomes clear
that they’re mistaken—the final product will be a program more elegant
than if you had spent weeks planning it beforehand.

I've always heard design is 90% of the job. You have to have a good
design in place before writing anything.

This quote from OnLisp is a radical paradigm for me (being an engineer
who designs things). My question to you is does this apply in
practice? In the real world when the rubber hits the road, can you
program this way and develop good software used by lots of people?


-

On Feb 4, 10:36 pm, Xah Lee xah...@gmail.com wrote:

yes and no. The problem is that such opinion is not a scientific
opinion. It can't be verified or denied.

Any dynamic lang, such as Python, PHP, Ruby, Mathematica can be said
the same thing. And, prob more so for functional langs.

The langs that this cannot be done, is C, Java, where the lang is
inflexible, low level, or requires big structure that's hard to
change.

if you want software engineering books, i suggest try some books that
are based on statistical survey, as opposed to some dignitary's
“opinion” or current fashion  trends that comes with a jargon. These
type of books are a dime a dozen, every year, they come and go. The
stastical survey approach takes major understaking and cost. The
opinion type any motherfucking “guru” can write. e.g. Paul Graham has
you buy into certain concept of “hackers” fucker and claim they are
like “painters”. Certain Gabriel wants you to believe in poetry and C
is the last language. (see Book Review: Patterns of
Softwarehttp://xahlee.org/PageTwo_dir/Personal_dir/bookReviewRichardGabriel.html
) et al. Certain Gosling wants you to believe Java is the last lang on
earth that'll wipe out Microsoft (circa 1998).

Pascal Constanza wrote:
 Yes. There are actually complete software development methodologies
 built around these ideas. Google for extreme programming and agile
 software methodologies.

Pascal Constanza is a Common Lisp fanatic. Note here, that eXtreme
Programing is one of the snake oil, that ran like rampant wild fire in
the industry around 2002, with many books published on it on the
supposed motherfucking hip Software Engineering practices, but today
you don't hear much of it. I haven't looked at “Agile programing”
agile my ass, but it is probably a waste of time.

... what society overwhelmingly asks for is snake oil. Of course, the
snake oil has the most impressive names —otherwise you would be
selling nothing— like “Structured Analysis and Design”, “Software
Engineering”, “Maturity Models”, “Management Information Systems”,
“Integrated Project Support Environments” “Object Orientation” and
“Business Process Re-engineering” (the latter three being known as
IPSE, OO and BPR, respectively).” — Edsger W Dijkstra (1930-2002), in
EWD 1175: The strengths of the academic enterprise.

you want to be a good programer? Study math, the math of programing,
and master the langs and protocols and tools you use. You want to
climb the tech industry ladder? get better at people, learn politics.
You want to be rich? Study business. Chances are, you are a coding
geek, and at heart you dna is not geared to be interested in politics
or business, and you'll never be rich or big ceo just because of that.

See also:
• Why Software Suck
 http://xahlee.org/UnixResource_dir/writ/why_software_suck.html

  Xah
∑ http://xahlee.org/

☄

--
http://mail.python.org/mailman/listinfo/python-list

Re: is python Object oriented??

[Bruno Desthuilliers]

thmpsn@gmail.com a écrit :

On Feb 4, 3:11 am, Bruno Desthuilliers bruno.
42.desthuilli...@websiteburo.invalid wrote:

thmpsn@gmail.com a écrit :




On Feb 3, 1:14 am, David Cournapeau courn...@gmail.com wrote:

(snip)

after all, we have used FILE* for years and I have no idea about the FILE
structure.

Your lack of knowledge about it doesn't mean that it has somehow
magically private members. The only reason that most of us don't
know what a FILE is is that it's definition is implementation-defined
(i.e., every compiler may define it differently).
That doesn't have anything to do with private members. For example, on
my system, stdio.h defines FILE as:
struct _iobuf {
char *_ptr;
int   _cnt;
char *_base;
int   _flag;
int   _file;
int   _charbuf;
int   _bufsiz;
char *_tmpfname;
};

Didn't you notice kind of a pattern here ?


You mean the leading underscores? I don't think they're used for the
same purpose as in Python. 


Not exactly the same purpose, indeed, but there's still something close: 
mark the names as special.



In C/C++, identifiers that begin with an
underscore are reserved (at least at the outer level). Thus, if the
member '_ptr' would instead be named 'ptr', something like this would
break things:

#define ptr // OOPS!!
#include stdio.h

That shouldn't happen with '_ptr', since programs are not supposed to
define those kinds of names (yeah, right).


And here again : would break, shouldn't, supposed. Convention over 
enforcement.



typedef struct _iobuf FILE;
Given this information, nothing prevents me from writing things like:
FILE* fp = fopen(file.txt, r);
if (!fp) { /* do something */ }
printf(fp-_cnt = %d\n, fp-cnt);
printf(fp-_flag = %d\n, fp-_flag);
printf(fp-_file = %d\n, fp-_file);
fp-_flag = 0x20; // OOPS!!



Indeed - and that's exactly the point : nothing prevents you from
accessing the implementation, *and yet, you don't*


I sure don't, but I still *can*.


So what ?


CHAOS MAY ENSUE!!


C is not really a new technology, and has been the basis for most 
systems for years and years. Looks like nothing that bad as CHAOS 
really happened so far...

--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[Mark Dickinson]

On Feb 5, 1:18 am, Chris Rebert c...@rebertia.com wrote:
 For an integer:
 is_even = bin(the_int)[2:].count('1') % 2 == 0

But the OP has to use if and while.  How about:

while 2+2 != 5:
if 'wkw' in 'just being awkward':
is_even = bin(the_int)[2:].count('1') % 2 == 0
break

or (Python 2.5 compatible):

def count_set_bits(n):
# make sure we include an if, to
# satisfy OP's requirements:
if n  0:
raise ValueError
count = 0
while n:
count += 1
n = n-1
return count

is_even = count_set_bits(the_int) % 2 == 0

...but anyone submitting this as a homework
solution had better be prepared to explain why
it works.


Mark
--
http://mail.python.org/mailman/listinfo/python-list

global name 'sqrt' is not defined

[Nick Matzke]

Hi all,

So, I can run this in the ipython shell just fine:

===
a = [12, 15, 16, 38.2]
dim = int(sqrt(size(a)))

dim
2
===


But if I move these commands to a function in another file, it freaks out:


=
a = distances_matrix.split('\t')
from LR_run_functions_v2 import make_half_square_array
d = make_half_square_array(a)


---
NameError Traceback (most recent call last)

/bioinformatics/phylocom/ipython console in module()

/bioinformatics/phylocom/_scripts/LR_run_functions_v2.py in 
make_half_square_array(linear_version_of_square_array)

   1548
   1549 a = linear_version_of_square_array
- 1550 dim = int(sqrt(size(a)))
   1551
   1552

NameError: global name 'sqrt' is not defined

=




Here's the function in LR_run_functions_v2.py

==
def make_half_square_array(linear_version_of_square_array):

a = linear_version_of_square_array
dim = int(sqrt(size(a)))
==



Any ideas?  If I do something like import math in the subfunction, 
then the error changes to global name 'math' is not defined.


Thanks!
Nick







--

Nicholas J. Matzke
Ph.D. student, Graduate Student Researcher
Huelsenbeck Lab
Center for Theoretical Evolutionary Genomics
4151 VLSB (Valley Life Sciences Building)
Department of Integrative Biology
University of California, Berkeley

Lab websites:
http://ib.berkeley.edu/people/lab_detail.php?lab=54
http://fisher.berkeley.edu/cteg/hlab.html
Dept. personal page: 
http://ib.berkeley.edu/people/students/person_detail.php?person=370

Lab personal page: http://fisher.berkeley.edu/cteg/members/matzke.html
Lab phone: 510-643-6299
Dept. fax: 510-643-6264
Cell phone: 510-301-0179
Email: mat...@berkeley.edu

Mailing address:
Department of Integrative Biology
3060 VLSB #3140
Berkeley, CA 94720-3140

-
[W]hen people thought the earth was flat, they were wrong. When people 
thought the earth was spherical, they were wrong. But if you think that 
thinking the earth is spherical is just as wrong as thinking the earth 
is flat, then your view is wronger than both of them put together.


Isaac Asimov (1989). The Relativity of Wrong. The Skeptical Inquirer, 
14(1), 35-44. Fall 1989.

http://chem.tufts.edu/AnswersInScience/RelativityofWrong.htm

--
http://mail.python.org/mailman/listinfo/python-list

Re: Use list name as string

[Tino Wildenhain]

Hi,

Vincent Davis wrote:

Sorry for not being clear
I would have something like this
x = [1, 2, 3,5 ,6 ,9,234]

Then
def savedata(dataname): ..

savedata(x)

this would save a to a file called x.csv This is my problem, getting the 
name to be x.csv which is the same as the name of the list.


and the data in the file would be 
1,2,3,5,6,9,234 this parts works


the problem you are facing comes from a little misunderstanding.
To clarify:

python objects are nameless.

You can bind them to any number of names (aka variables)

 1 # unnamed integer object with value 1
1

 a=1 # bind the integer object to name 'a'
 b=a # bind the same integer object referred to by name a to name b

therefore in your above example, which name should your
savedata pick up for the filename? the 'x' of the first
assignment or the 'dataname' of the assignment in the function
call?

The only solution I see would be to add a property to your datastore
to give it its own unique name. (By subclassing and providing
a name attribute or property) - and while you are at it, maybe
you want to put the 'write to file' part into the class as well.

Regards
Tino



smime.p7s
Description: S/MIME Cryptographic Signature
--
http://mail.python.org/mailman/listinfo/python-list

Re: global name 'sqrt' is not defined

[Chris Rebert]

On Thu, Feb 5, 2009 at 1:08 AM, Nick Matzke mat...@berkeley.edu wrote:
 Hi all,

 So, I can run this in the ipython shell just fine:

 ===
 a = [12, 15, 16, 38.2]
 dim = int(sqrt(size(a)))

sqrt() is not a builtin function, it's located in the 'math' module.
You must have imported it at some point.
Or it could be be getting imported without your direct knowledge
through ~/.pythonrc.py, $PYTHONSTARTUP, and/or by doing 'import user'
For that matter, size() is not built into Python either, so you're
definitely importing these things somewhere.

Adding explicit imports of the modules/functions in question to the
top of the file(s) in question should fix the problem.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

RE: How to find wxPython method documentation??

[Barak, Ron]

Hi Len,
First off, there's the wxPython mailing list 
(To subscribe or unsubscribe via the World Wide Web, visit 
http://lists.wxwidgets.org/mailman/listinfo/wxpython-users or, via email, send 
a message with subject or body 'help' to 
wxpython-users-requ...@lists.wxwidgets.org)
I also find Google invaluable in searching for wxPython knowledge, e.g., try 
http://www.google.com/search?q=wxpython+dialog.
Bye,
Ron.
--
http://mail.python.org/mailman/listinfo/python-list

Re: time: Daylight savings confusion

[Matt Nordhoff]

Tim H wrote:
 On Win XP 64bit, Python 2.6.1 64bit
 
 I am trying to rename files by their creation time.
 
 It seems the time module is too smart for its own good here.
 
 time.localtime(os.path.getctime(f)) returns a value one hour off from
 what windows reports for files that were created when Daylight savings
 time was in effect (ie when the tm_isdst field is one).  I can kludge
 it, but am I missing the right way to do it?
 
 Tim
 
 full code:

snip full code

From what I remember, this is a bug in the Win32 API. Python probably
could hack around it, but, well, I guess it doesn't.

Googling a bit, Perl's Win32::UTCFileTime module [1] provides
workarounds (not for Python, of course, but it could probably be ported)
and explains all the gory details (which I have not read, so maybe I
misunderstand ;-).

[1]
http://search.cpan.org/~shay/Win32-UTCFileTime-1.50/lib/Win32/UTCFileTime.pm
http://xrl.us/beezsv
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing two book chapters (text files)

[andrew cooke]

On Feb 4, 10:20 pm, Nick Matzke mat...@berkeley.edu wrote:
 So I have an interesting challenge.  I want to compare two book
 chapters, which I have in plain text format, and find out (a) percentage
 similarity and (b) what has changed.

no idea if it will help, but i found this yesterday - http://www.nltk.org/

it's a python toolkit for natural language processing.  there's a book
at http://www.nltk.org/book with much more info.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: How do i add body to email.mime.multipart.MIMEMultipart instance?

[Justin Ezequiel]

On Feb 4, 2:48 pm, srinivasan srinivas sri_anna...@yahoo.co.in
wrote:
 Hi,
 Could someone tell me the way to add body to the instance 
 email.mime.multipart.MIMEMultipart instance which has attachments?

 Thanks,

msg = MIMEMultipart()
msg.preamble = 'This is a multi-part message in MIME format.\n'
msg.epilogue = ''

body = MIMEMultipart('alternative')
body.attach(MIMEText(text))
body.attach(MIMEText(html, 'html'))

msg.attach(body)

attachment = Message()
attachment.add_header('Content-type', content_type,
  name=basename)
attachment.add_header('Content-transfer-encoding', 'base64')
attachment.add_header('Content-Disposition', 'attachment',
  filename=basename)
attachment.set_payload(b.getvalue())

msg.attach(attachment)

msg.add_header('Message-ID', generate_message_id(sender[1]))
msg.add_header('Date', strftime(DATEFORMAT, gmtime()))

msg.add_header('From', formataddr(sender))
msg.add_header('To', format_addresses(TOs))
msg.add_header('Cc', format_addresses(CCs))
msg.add_header('Subject', subj)

--
http://mail.python.org/mailman/listinfo/python-list

Re: sys.float_info.epsilon

[Mark Dickinson]

On Feb 4, 9:44 pm, Tim Rowe digi...@gmail.com wrote:
 That just leaves me puzzled as to why Mark Summerfield used it instead
 of a check against zero on user input.

No idea:  you'd have to ask Mark Summerfield.  If there's
an email address published in his book, I'm sure he
wouldn't object to the question.

 So my next question is whether there is any x that can
 be returned by float() such that x != 0 but some_number / (2 * x)
 raises a ZeroDivisionError?

Nope.  If x is nonzero, then 2*x is definitely nonzero.
It could be an infinity, or a nan, or the '2*x'
computation could raise an exception, or perhaps
cause the interpreter to crash (it shouldn't, but
you never know...), but it's never going to be zero.

Well, okay, *never* is a strong word:  if 2*x is
subnormal, and you're operating on a platform that
for whatever reasons flushes subnormal results
to zero, then it could happen.  But not in *real*
life. :)

Mark
--
http://mail.python.org/mailman/listinfo/python-list

Ordered dict by default

[bearophileHUGS]

Choosing the right data structure is usually a matter of compromises,
and sometimes the best you can do is to change some data structures
and look for the faster running time. To do this it helps to have a
language that allows you to swap data structures in the more
transparent way possible.

It's not just a matter of the specific program, it's also a matter of
the nature of the programming language. Python isn't meant to be high-
performance in running time, it's meant to be easy to use, flexible,
and easy to understand. (Ruby shares almost the same purposes, but it
looks a bit less easy, a bit more flexible, and offers a bit more
freedom, and it used to be a little slower).

Now Ruby dicts are ordered by default:
http://www.igvita.com/2009/02/04/ruby-19-internals-ordered-hash/

Of course Python can grow odicts into its collections module, that can
be used everywhere a normal dict is used. But while this may be a good
solution for faster languages like Java, C# and D, for me it doesn't
sound like the best solution for Python.

That page about Ruby dicts show a higher traversal speed (probably
just because the CPU has to scan less memory, but I am not sure,
because mordern CPUs are very complex) but lower insertion speed (I
think mostly not because the added management of two pointers, but
because the memory used increases, so there are more cache misses. If
this turns out as true, then using the xor trick may halve the extra
memory needed). I have no idea if such different balance of
performance will lead to on average faster or slower Python programs,
this has to be tested. But even if the average performance becomes a
little worse I think making the default Python dict as ordered is a
positive change for Python too, because built-ins are meant to be as
flexible as possible, even if they aren't the fastest ones or the less
memory hungry ones (and keeping dicts ordered decreases the surprises
a newbie finds, makes some code cleaner, and doctests become simpler
to write because the order of items may be more defined).

Once the default dicts are ordered, it can be possible to add an
unordereddict to the collections module to be used by programmers when
max performance or low memory usage is very important :-)

Namespaces and other internal things of Python can keep using the un-
ordereddicts.

I don't know what to do regarding Python sets yet. In mathematics sets
aren't ordered, so they may be kept as they are now.

Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list

Re: Use list name as string

[Tino Wildenhain]

Hendrik van Rooyen wrote:
  MRAB goo...@mrett.plus.com wrote:

 The actual names of the variables and functions shouldn't matter to the
 outside world; the name of an output file shouldn't depend on the name
 of a variable.

 That is a matter of opinion.
 It is however, an interesting problem, namely:

 How does one get hold of the actual name by which some parameter
 is passed?

 you may want to print, as a debug thingy:

 print the name passed in was: , ImpossibleThingYieldingName
 print and it evaluates to: , ArgumentPassed

This is possible to some degree:

import inspect

def F(a):
frame_obj,filename,line_no,
func_name,contextlines,
contextindex=(inspect.getouterframes(inspect.currentframe()))[1]
print F(%s) called from '%s' within '%s' line %d %
  (repr(a),filename,func_name,line_no)
for ln,srcline in enumerate(contextlines or []):
print %3s : %s % ('*' if ln==contextindex else '',srcline)


just play around calling the above function from different
places and you should see what I mean :-)

Regards
Tino


smime.p7s
Description: S/MIME Cryptographic Signature
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[Tim Rowe]

2009/2/5 Duncan Booth duncan.bo...@invalid.invalid:
 Mark Dickinson dicki...@gmail.com wrote:

 def count_set_bits(n):
 # make sure we include an if, to
 # satisfy OP's requirements:
 if n  0:
 raise ValueError
 count = 0
 while n:
 count += 1
 n = n-1
 return count

 is_even = count_set_bits(the_int) % 2 == 0

 ...but anyone submitting this as a homework
 solution had better be prepared to explain why
 it works.


 I remember a programming exercise when I was an undergraduate and anyone
 who *didn't* use that trick got marked down for writing inefficient code.

Is adding and a modulus *really^ more efficient than flipping a bool
as I suggested? I think I'd want to see measurements!


-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing two book chapters (text files)

[Tino Wildenhain]

andrew cooke wrote:

On Feb 4, 10:20 pm, Nick Matzke mat...@berkeley.edu wrote:

So I have an interesting challenge.  I want to compare two book
chapters, which I have in plain text format, and find out (a) percentage
similarity and (b) what has changed.


no idea if it will help, but i found this yesterday - http://www.nltk.org/

it's a python toolkit for natural language processing.  there's a book
at http://www.nltk.org/book with much more info.


Also there is difflib in the standard package which can be used
depending on exact definition of similarity.

Regards
Tino


smime.p7s
Description: S/MIME Cryptographic Signature
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Steve Holden]

bearophileh...@lycos.com wrote:
[a somewhat feeble case for ordered dicts]
 Once the default dicts are ordered, it can be possible to add an
 unordereddict to the collections module to be used by programmers when
 max performance or low memory usage is very important :-)
 
I have no real idea why you think it's desirable to turn this important
feature on its head, but if you've followed the various past threads on
ordered dicts you would know that one of the prime issues every time
this comes up is the inability of the various proponents to agree no ow
dicts should actually be ordered.

 Namespaces and other internal things of Python can keep using the un-
 ordereddicts.
 
 I don't know what to do regarding Python sets yet. In mathematics sets
 aren't ordered, so they may be kept as they are now.
 
In mathematics mappings aren't ordered either, and a pure dict is pretty
much a mapping. So leave them alone, they are fine as they are!

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Paul Rubin]

Steve Holden st...@holdenweb.com writes:
 In mathematics mappings aren't ordered either, and a pure dict is pretty
 much a mapping. So leave them alone, they are fine as they are!

Ehhh, an ordered dict has to support a comparison operation on the
keys, while a Python dict has to support a hashing operation on them.
Neither really captures the idea of a pure mapping, which shouldn't
have to support any operations (even equality) on the keys.

There have been several times when I've wished for persistent ordered
dicts in Python.  I started writing a big post about it a while ago
and maybe I'll finish it someday.
--
http://mail.python.org/mailman/listinfo/python-list

Re: parse date/time from a log entry with only strftime (and no regexen)

[Simon Mullis]

That, my friend, is ingenious...!

Thankyou

SM

2009/2/3 andrew cooke and...@acooke.org

   ValueError: unconverted data remains:  this is the remainder of the log

   line
   that I do not care about

 you could catch the ValueError and split at the ':' in the .args
 attribute to find the extra data.  you could then find the extra data
 in the original string, use the index to remove it, and re-parse the
 time.

 ugly, but should work.
 andrew
 --
 http://mail.python.org/mailman/listinfo/python-list




-- 
Simon Mullis
_
si...@mullis.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Should open(sys.stdin) and open(file, 'r') be equivalent?

[Simon Mullis]

Hi All

I've written a simple python script that accepts both stdin and a glob (or
at least, that is the plan).
Unfortunately, the glob part seems to hang when it's looped through to the
end of the filehandle.

And I have no idea why... ;-)

sys.stdin and a normal file opened with open seem to both be identical
filehandles.

 import sys
 foo = sys.stdin
 type(foo)
type 'file'
 repr(foo)
open file 'stdin', mode 'r' at 0x16020

 bar = open(test_file, 'r')
 type(bar)
type 'file'
 repr(bar)
open file 'test_file', mode 'r' at 0x3936e0

The stdin version is fine. I want to re-use the code for scan_data (and all
of the other processing methods) so I'd like to be able to iterate over one
line at a time, independently of the source (i.e. either file or stdin)

Code that illustrates the issue follows:

# cat test_fh.py

#!/usr/bin/env python
import glob, os, sys
class TestParse(object):
def __init__(self):
if (options.stdin):
  self.scan_data(sys.stdin)
if (options.glob):
  self.files = glob.glob(options.glob)
  for f in files:
  fh = open(f, 'r')
  self.scan_data(fh)
  fh.close()

def scan_data(self,fileobject):
i = int()
for line in fileobject:
print i
i += 1
# do stuff with the line...
pass
   print finished file

def main():
T = TestParse()

if __name__ == __main__:
from optparse import OptionParser
p = OptionParser(__doc__, version=testing 1 2 3)
p.add_option(--glob, dest=glob)
p.add_option(--stdin, dest=stdin, action=store_true,
default=False)
(options, args) = p.parse_args()
main()

#EOF

Running this against stdin outputs a count of lines and then exits fine
(exit code 0).

# cat test_file | ./test-fh.py --stdin
...output...
# echo $?
0

Running against --glob test_file just hangs.

# ./test_fh.py --glob test_file
wait 20 seconds or so...
^CTraceback (most recent call last):
  File ./test_fh.py, line 35, in module
main()
  File ./test_fh.py, line 26, in main
T = TestParse()
  File ./test_fh.py, line 8, in __init__
self.scan_data(sys.stdin)
  File ./test_fh.py, line 18, in scan_data
for line in fileobject:
KeyboardInterrupt
 # echo $?
1

So, what am I doing wrong?

Thanks in advance

SM

-- 
Simon Mullis
_
si...@mullis.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Duncan Booth]

Paul Rubin http://phr...@nospam.invalid wrote:

 bearophileh...@lycos.com writes:
 Now Ruby dicts are ordered by default:
 http://www.igvita.com/2009/02/04/ruby-19-internals-ordered-hash/
 
 Maybe I didn't read that carefully enough, but it looks like ordered
 means the dict records come out in the same order you inserted them
 in.  That is if you insert B,A,D,C in that order, you get them out in
 that order.  I would have thought an ordered dict meant you get A,B,C,D,
 which seems a lot more useful.
 
If you want to write doctests then any stable order in the default dict 
type would be helpful no matter whether it means that keys are in original 
insertion or latest insertion order or sorted.

There are other use cases where maintaining insertion order is important. 
For example any automated code generator which parses its previous output 
and then regenerates it maintaining edits. I ran into that before with one 
which just stored the methods in a Python dict so that every so often you'd 
find the entire source file had rearranged itself making a mess of version 
control.

It would certainly be an interesting experiment to mimic the Ruby dict 
implementation in Python's dict code and see what effect it has on 
performance: the Ruby page claims that while inserts are slower traversal 
is much faster. I think it would need some pretty convincing arguments 
though to become the default.

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding descriptors

[Bruno Desthuilliers]

Brian Allen Vanderburg II a écrit :
I'm trying to better understand descriptors and I've got a few questions 
still after reading some sites.  Here is what I 'think', but please let 
me know if any of this is wrong as I'm sure it probably is.



First when accessing an attribute on a class or instance it must be 
found.  For an instance, it's __dict__ is first search.


Actually, this is not true - binding descriptors are searched (in the 
class and it's mro) _before_ the instance's __dict__ :


 class Foo(object):
... @apply
... def bar():
... def fget(self):
... return self._bar
... def fset(self, v):
... self._bar = v
... return property(**locals())
...
 f = Foo()
 f.bar = 42
 f.bar
42
 f.__dict__['bar'] = boo
 f.bar
42
 f.__dict__['bar']
'boo'


Note that this works a bit differently for non-binding descriptors 
(descriptors that only implement the __get__ method), which are looked 
up after the instance's __dict__


So the lookup chain is:

1/ lookup the class and bases for a binding descriptor
2/ then lookup the instance's __dict__
3/ then lookup the class and bases for a non-binding descriptor or plain 
attribute

4/ then class __getattr__

Also and FWIW, there's a step zero : calls __getattribute__. All the 
above lookup mechanism is actually implemented by object.__getattribute__.


 If not found 
the class and base class __dict__ are searched.  For a class, the 
__dict__ and base classes __dict__ are search.


If assigning, and the attribute is found and is not a descriptor


a binding descriptor

or if 
the attribute is not found, then the assignment will occur in the 
__dict__ of the class or instance.  If it is found and is a descriptor, 


a binding descriptor


then __set__ will be call.



For reading, if the attribute is found and is a descriptor, __get__ will 
be called, passing the object (if it is an instance) and class.  If it 
is not a descriptor, the attribute will be returned directly.


Class methods are just functions:


In Python, classmethod has a definite meaning - it's a method that 
takes the class (and not the instance) as first argument. So your 
assertion should be Methods are just functions which are attributes of 
the class. Which is still not quite true. Sure, the def statement 
always create function objects, regardless of whether it happens within 
a class statement or not. OTHO, what you get when looking up the 
corresponding attribute is definitly a method object. So the correct 
formulation here is (IMHO of course) that methods are implemented by 
functions that are attributes of the class. Also, this is not restricted 
to functions : any callable object correctly implementing the descriptor 
protocol can be used (cf the classmethod and staticmethod objects).



class C(object):
   def F(self):
  pass

C.__dict__['F'] # function object ...


yes.


But functions are descriptors:


yes.


C.__dict__['F'].__get__ # method wrapper ...

def f1():
   pass

f1.__get__ # method wrapper ...

When a lookup is done it uses this descriptor to make a bound or unbound 
method:


yes.


c=C()

C.F # unbound method object, expects explicit instance when calling the 
function
c.F # bound method object provides instance implicitly when calling the 
function


This is also done when adding to the classes:


Indeed. Whether the def statement happens within the class statement or 
not is irrelevant here.



C.f1 = f1

f1 # function
C.f1 # unbound method
c.f1 # bound method

To prevent this it has to be decorated so the descriptor doesn't cause 
the binding:


C.f2 = staticmethod(f1)
C.f2 # functon
c.f2 # function


yes.


Here is a question, why don't instance attributes do the same thing?


Because the descriptor protocol is only invoked on class attributes.


c.f3 = f1
c.f3 # function, not bound method

So it is not calling the __get__ method for c.f3  After it finds c.f3 in 
c.__dict__, and since it has a getter, shouldn't it call the __get__ to 
return the bound method.


Nope, cf above.

 It is good that it doesn't I know, but I just 
want to know why it doesn't from an implementation view.


If by from an implementation view, you mean how it works, then the 
answer is above. If your question was about why this design choice, 
I'll leave the answer to someone else...

--
http://mail.python.org/mailman/listinfo/python-list

Python 3.0 slow file IO

[thomasvang...@gmail.com]

I just recently learned python, I'm using it mainly to process huge
5GB txt files of ASCII information about DNA. I've decided to learn
3.0, but maybe I need to step back to 2.6?

I'm getting exceedingly frustrated by the slow file IO behaviour of
python 3.0. I know that a bug-report was submitted here:
http://bugs.python.org/issue4533. And a solution was posted.
However, i don't know how to apply this patch. I've searched the
forums and tried:
C:\python30 patch -p0  fileio_buffer.patch
The patch command is not recognized..

Any help on implementing this patch, or advice on moving back to the
older version is appreciated.
Kind regards,
Thomas
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Paul Rubin]

Duncan Booth duncan.bo...@invalid.invalid writes:
 If you want to write doctests then any stable order in the default dict 
 type would be helpful no matter whether it means that keys are in original 
 insertion or latest insertion order or sorted.

Just use sorted in the test code:
   print sorted(dict((a**2,a) for a in xrange(5)).keys())
  [0, 1, 4, 9, 16]

 There are other use cases where maintaining insertion order is
 important.  For example any automated code generator which parses
 its previous output and then regenerates it maintaining edits. I ran
 into that before with one which just stored the methods in a Python
 dict so that every so often you'd find the entire source file had
 rearranged itself making a mess of version control.

Still doesn't sound so bad: just include a counter with the value.
Something like (untested):

   odict = dict(zip(methodnames, zip(methodbodies, itertools.count(
   ...
   gen_code(sorted(odict.iteritems(), key=lambda(name,(body,n)): n))

 It would certainly be an interesting experiment to mimic the Ruby dict 
 implementation in Python's dict code and see what effect it has on 
 performance: the Ruby page claims that while inserts are slower traversal 
 is much faster. I think it would need some pretty convincing arguments 
 though to become the default.

In my usage of dicts, traversal is rather rare, but YMMV.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 slow file IO

[Ulrich Eckhardt]

thomasvang...@gmail.com wrote:
 C:\python30 patch -p0  fileio_buffer.patch
 The patch command is not recognized..

You need the 'patch' program first. Further, you will need a C compiler. If
you don't know how to compile from sources, I would postpone patching
sources to after learning that.

 [...] advice on moving back to the older version is appreciated.

Just install the latest 2.6 release. I think you can easily install it
alongside 3.0, all you need to pay attention to is the PATH environment
variable.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[Tim Rowe]

2009/2/5 Duncan Booth duncan.bo...@invalid.invalid:

 I remember a programming exercise when I was an undergraduate and
 anyone who *didn't* use that trick got marked down for writing
 inefficient code.

 Is adding and a modulus *really^ more efficient than flipping a bool
 as I suggested? I think I'd want to see measurements!


 I meant the bitwise twiddling, but actually I misread it. I thought Mark
 was using the n~n+1 trick to pull out bits from least significant upwards
 when of course he's just clearing the low bit not extracting it.

Ah, ok. Yes, that's faster than my bit shifting. A million bytes with
your method on my computer with your bit-twiddling takes 1.047
seconds, with my bit-shifting it takes 1.578. On the other hand,
unless that was demonstrated to be a bottleneck I'd still go for the
bit shifting because I think it's clearer. And I'd want a word with a
tutor who insisted on premature optimisation (The root of all evil,
according to C A R Hoare), especially in a scripting language!

-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: JDBC in CPYTHON

[M.-A. Lemburg]

On 2009-02-05 03:49, KMCB wrote:
 Thanks Simon and Marc,
 
 I currently have an app on OSX that I wanted to migrate to NIX, it
 uses a ODBC DBAPI interface to communicate with Filemaker.
 Unfortunately, FMP does not support linux drivers.  They do have a
 JDBC driver that looks like it may work.  My preference was to run
 that app on one machine.
 
 I may not be understand your suggestion Marc, can the app and bridge
 run on the same machine.  Can I specify the ODBC to a localhost, being
 the bridge?  Then have the JDBC driver installed.

The setup would work as follows:

Python - mxODBC - ODBC-JDBC bridge - JDBC driver - Filemaker

[--- Linux ]   [Mac OS X]

I am not sure how you'd have to configure the EasySoft bridge.
Please check their documentation for details.

We are currently working on releasing mxODBC Connect, our client server
Python database interface, for Mac OS X. You will then be able
to use a much simpler and more efficient setup:

Python - mxODBC Connect Client -
[--- Linux ]

mxODBC Connect Server - ODBC driver - Filemaker
[--- Mac OS X ---]

and because the mxODBC Connect Cient is highly portable, moving
your application to other platforms would be easily possible
as well.

Plus, you'd avoid having to carry around the whole JVM in your
Python application: the ODBC-JDBC bridge essentially embeds
the JVM in your application and then uses JNI to interface
from C to Java and from there to the JDBC driver.

 Also, your second link does not seam to have a download or anything in
 the CVS.

Sorry, I should have looked closer... the SF project page reveals
that the PROJECT IS DEAD!:

   http://sourceforge.net/projects/odbcjdbc/

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 05 2009)
 Python/Zope Consulting and Support ...http://www.egenix.com/
 mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[Duncan Booth]

Tim Rowe digi...@gmail.com wrote:

 2009/2/5 Duncan Booth duncan.bo...@invalid.invalid:
 Mark Dickinson dicki...@gmail.com wrote:

 def count_set_bits(n):
 # make sure we include an if, to
 # satisfy OP's requirements:
 if n  0:
 raise ValueError
 count = 0
 while n:
 count += 1
 n = n-1
 return count

 is_even = count_set_bits(the_int) % 2 == 0

 ...but anyone submitting this as a homework
 solution had better be prepared to explain why
 it works.


 I remember a programming exercise when I was an undergraduate and
 anyone who *didn't* use that trick got marked down for writing
 inefficient code. 
 
 Is adding and a modulus *really^ more efficient than flipping a bool
 as I suggested? I think I'd want to see measurements!
 
 
I meant the bitwise twiddling, but actually I misread it. I thought Mark 
was using the n~n+1 trick to pull out bits from least significant upwards 
when of course he's just clearing the low bit not extracting it.


-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[Duncan Booth]

Mark Dickinson dicki...@gmail.com wrote:

 def count_set_bits(n):
 # make sure we include an if, to
 # satisfy OP's requirements:
 if n  0:
 raise ValueError
 count = 0
 while n:
 count += 1
 n = n-1
 return count
 
 is_even = count_set_bits(the_int) % 2 == 0
 
 ...but anyone submitting this as a homework
 solution had better be prepared to explain why
 it works.
 

I remember a programming exercise when I was an undergraduate and anyone 
who *didn't* use that trick got marked down for writing inefficient code.

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: string replace for back slash

[Chris Rebert]

On Thu, Feb 5, 2009 at 3:40 AM, S.Selvam Siva s.selvams...@gmail.com wrote:
 Hi all,

 I tried to do a string replace as follows,

 s=hi  people
 s.replace(,\)
 'hi \\ people'


 but i was expecting 'hi \ people'.I dont know ,what is something different
 here with escape sequence.

The Python interactive interpreter does an implicit repr() (consult
the docs if you're unfamiliar with the repr() function) on the return
value of the expression. Note the single quotes in the output, which
obviously wouldn't be present in the string itself; same thing with
the doubling of the backslash.
If you instead do `print s.replace(,\)`, you'll see that the
outputted string does indeed only contain 1 backslash.

It's a common newbie confuser, there really should be a FAQ about it;
pity there isn't.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing two book chapters (text files)

[M.-A. Lemburg]

On 2009-02-05 02:20, Nick Matzke wrote:
 Hi all,
 
 So I have an interesting challenge.  I want to compare two book
 chapters, which I have in plain text format, and find out (a) percentage
 similarity and (b) what has changed.
 
 Some features make this problem different than what seems to be the
 standard text-matching problem solvable with e.g. difflib.  Here is what
 I mean:
 
 * there is no guarantee that single lines from each file will be
 directly comparable -- e.g., if a few words are inserted into a
 sentence, then a chunk of the sentence will be moved to the next line,
 then a chunk of that line moved to the next, etc.
 
 * Also, there are cases where paragraphs have been moved around,
 sections re-ordered, etc.  So it can't just be a linear match.
 
 I imagine this kind of thing can't be all that hard in the grand scheme
 of things, but I couldn't find an easily applicable solution readily
 available.  I have advanced beginner python skills but am not quite
 where I could do this kind of thing from scratch without some guidance
 about the likely functions, libraries etc. to use.
 
 PS: I am going to have to do this for multiple book chapters so various
 software packages, e.g. for windows, are not really usable.
 
 Any help is much appreciated!!

difflib is in the Python stdlib and provides many ways to implement
difference detection:

http://docs.python.org/library/difflib.html

Here's a script that I use for diff'ing text files on a word
basis, called tdiff.py:

http://downloads.egenix.com/python/tdiff.py

It helps a lot with text that gets word wrapped or reformatted.

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 05 2009)
 Python/Zope Consulting and Support ...http://www.egenix.com/
 mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list

Re: global name 'sqrt' is not defined

[M.-A. Lemburg]

On 2009-02-05 10:08, Nick Matzke wrote:
 Hi all,
 
 So, I can run this in the ipython shell just fine:
 
 ===
 a = [12, 15, 16, 38.2]
 dim = int(sqrt(size(a)))
 
 dim
2
 ===
 
 
 But if I move these commands to a function in another file, it freaks out:

You need to add:

from math import sqrt

 =
 a = distances_matrix.split('\t')
 from LR_run_functions_v2 import make_half_square_array
 d = make_half_square_array(a)
 

 ---
 NameError Traceback (most recent call last)
 
 /bioinformatics/phylocom/ipython console in module()
 
 /bioinformatics/phylocom/_scripts/LR_run_functions_v2.py in
 make_half_square_array(linear_version_of_square_array)
1548
1549 a = linear_version_of_square_array
 - 1550 dim = int(sqrt(size(a)))
1551
1552
 
 NameError: global name 'sqrt' is not defined
 
 =
 
 
 
 
 Here's the function in LR_run_functions_v2.py
 
 ==
 def make_half_square_array(linear_version_of_square_array):
 
 a = linear_version_of_square_array
 dim = int(sqrt(size(a)))
 ==
 
 
 
 Any ideas?  If I do something like import math in the subfunction,
 then the error changes to global name 'math' is not defined.
 
 Thanks!
 Nick
 
 
 
 
 
 
 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 05 2009)
 Python/Zope Consulting and Support ...http://www.egenix.com/
 mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding descriptors

[Brian Allen Vanderburg II]

bruno.42.desthuilli...@websiteburo.invalid wrote:


So the lookup chain is:

1/ lookup the class and bases for a binding descriptor
2/ then lookup the instance's __dict__
3/ then lookup the class and bases for a non-binding descriptor or 
plain attribute

4/ then class __getattr__

Also and FWIW, there's a step zero : calls __getattribute__. All the 
above lookup mechanism is actually implemented by 
object.__getattribute__.


Okay, so instance attributes never use their __get__/__set__/etc when 
looking up.


A binding descriptor is one that has a __set__ (even if it doesn't do 
anything) and it takes priority over instance variables.  Properties are 
binding descriptors even if they don't have a set function specified.  A 
non-binding descriptor doesn't have __set__ and instance variables take 
priority over them. 


For reading:

1. Lookup in the class/bases for a binding descriptor and if found use 
its __get__

2. If instance, look up in instance __dict__ and if found return it
3. Lookup in the class/bases
   a. if found and a descriptor use it's __get__
   b. if found and not a descriptor return it
4. Use __getattr__ (if instance?)

For writing:

1. If instance
   a. lookup in the class/bases for a binding descriptor and if found 
use its __set__

   b. write to instance __dict__
2. If class, write in class __dict__


I think I understand it now.  Thanks.

Brian Vanderburg II
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Duncan Booth]

Paul Rubin http://phr...@nospam.invalid wrote:

 Duncan Booth duncan.bo...@invalid.invalid writes:
 If you want to write doctests then any stable order in the default
 dict type would be helpful no matter whether it means that keys are
 in original insertion or latest insertion order or sorted.
 
 Just use sorted in the test code:
   print sorted(dict((a**2,a) for a in xrange(5)).keys())
   [0, 1, 4, 9, 16]
 

No, that only works if you have control over the dict iteration. There are 
plenty of situations when tests become unstable because dicts are unstable 
and you don't have control over the order of iteration. Also, if you are 
using doctests as a form of documentation then you just want to be able to 
show the value, or at most pprint it, you don't want to obscure the 
documentation with calls to sort.

For example if you have something that creates xml then almost certainly 
the library you are using stores the attributes for a tag in a dictionary 
and just outputs them in whatever order they iterate over. That's fine for 
creating XML but when you want a test like:

 mydata.asXml()
... some xml here except we don't know what order the attributes will 
appear in ...

then you are stuffed. Actually it's also a problem if you want to store the 
output XML in Subversion. As with my round-trip coding problem you could 
obviously solve it by changing the code to sort the output, but stabilising 
the order of dict iteration would solve the problem for all such cases.

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

string replace for back slash

[S.Selvam Siva]

Hi all,

I tried to do a string replace as follows,

 s=hi  people
 s.replace(,\)
'hi \\ people'


but i was expecting 'hi \ people'.I dont know ,what is something different
here with escape sequence.

-- 
Yours,
S.Selvam
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 slow file IO

[Christian Heimes]

thomasvang...@gmail.com schrieb:
 I just recently learned python, I'm using it mainly to process huge
 5GB txt files of ASCII information about DNA. I've decided to learn
 3.0, but maybe I need to step back to 2.6?
 
 I'm getting exceedingly frustrated by the slow file IO behaviour of
 python 3.0. I know that a bug-report was submitted here:
 http://bugs.python.org/issue4533. And a solution was posted.
 However, i don't know how to apply this patch. I've searched the
 forums and tried:
 C:\python30 patch -p0  fileio_buffer.patch
 The patch command is not recognized..

You need the Python sources, a patch utility and Microsoft Visual Studio
2008 in order to compile Python yourself.
 
 Any help on implementing this patch, or advice on moving back to the
 older version is appreciated.

I suggest you stick with Python 2.5 or 2.6. Python 3.0 isn't as mature
as the 2.x series. It's brand new, lot's of things have changed, too.

--
http://mail.python.org/mailman/listinfo/python-list

string replace for back slash

[rdmurray]

S.Selvam Siva s.selvams...@gmail.com wrote:
 I tried to do a string replace as follows,
 
  s=hi  people
  s.replace(,\)
 'hi \\ people'
 
 
 but i was expecting 'hi \ people'.I dont know ,what is something different
 here with escape sequence.

You are running into the difference between the 'repr' of a string (which
is what is printed by default at the python prompt) and the actual
contents of the string.  In the repr the backslash needs to be escaped
by prefixing it with a backslash, just as you would if you wanted to
enter a backslash into a string in your program.  If you print the string,
you'll see there is only one backslash.  Note that you didn't need to
double the backslash in your replacement string only because it wasn't
followed by a character that forms an escape...but the repr of that
string will still have the backslash doubled, and that is really the
way you should write it in your program to begin with for safety's sake.

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13) 
[GCC 4.3.2] on linux2
Type help, copyright, credits or license for more information.
 s=hi  people
 replacementstring = \
 replacementstring
'\\'
 print replacementstring
\
 x = s.replace(,\\)
 x
'hi \\ people'
 print x
hi \ people

--
http://mail.python.org/mailman/listinfo/python-list

Re: Should open(sys.stdin) and open(file, 'r') be equivalent?

[Simon Mullis]

Hi Chris

2009/2/5 Chris Rebert c...@rebertia.com

 I'd add some print()s in the above loop (and also the 'for f in files'
 loop) to make sure the part of the code you didn't want to share (do
 stuff with the line) works correctly, and that nothing is improperly
 looping in some unexpected way.

The point is that even with the very, very simple script I posted
above the behavior of open(sys.stdin) and open(filename, 'r') is
different.

The object foo (where foo = sys.stdin) allows me to iterate then hands
back after the loop is finished.

The object bar (where bar = open(filename, 'r')) does not.

Both foo and bar have the same type, methods, repr etc.

 Also, there are several series of lines with invalid indentation;
 could be an email artifact or could be the cause of your problem. If
 the print()s don't yield any useful insight, repost the code again
 with absolutely correct indentation.

(code posted again to fix indents)
#!/usr/bin/env python

import glob, os, sys

class TestParse(object):
def __init__(self):
if options.stdin:
self.scan_data(sys.stdin)
if options.glob:
self.files = glob.glob(options.glob)
for f in files:
fh = open(f, 'r')
self.scan_data(fh)
fh.close()

def scan_data(self,fileobject):
i = 0
for line in fileobject:
print i
i += 1
# do stuff with the line...
pass
print finished file

def main():
T = TestParse()

if __name__ == __main__:
from optparse import OptionParser
p = OptionParser(__doc__, version=testing 1 2 3)
p.add_option(--glob, dest=glob)
p.add_option(--stdin, dest=stdin, action=store_true, default=False)
(options, args) = p.parse_args()
main()


(The code I'm actually using is much more complex than this. I tried
to create the most simple example of it _not_ working as expected...)

 Finally, some stylistic points:
 - don't do 'if (foo):'   use the less noisy 'if foo:' instead
 - don't do 'i = int()'   use the more obvious 'i = 0' instead

ok.

My question again, to be more explicit: Should the objects created by
sys.stdin and open(filename, 'r') have the same behavior when iterated
over? They both have __iter__ methods

Thanks in advance for any suggestions

SM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Should open(sys.stdin) and open(file, 'r') be equivalent?

[Chris Rebert]

On Thu, Feb 5, 2009 at 2:58 AM, Simon Mullis si...@mullis.co.uk wrote:
 Hi All

 I've written a simple python script that accepts both stdin and a glob (or
 at least, that is the plan).
 Unfortunately, the glob part seems to hang when it's looped through to the
 end of the filehandle.

 And I have no idea why... ;-)

 sys.stdin and a normal file opened with open seem to both be identical
 filehandles.

 import sys
 foo = sys.stdin
 type(foo)
 type 'file'
 repr(foo)
 open file 'stdin', mode 'r' at 0x16020

 bar = open(test_file, 'r')
 type(bar)
 type 'file'
 repr(bar)
 open file 'test_file', mode 'r' at 0x3936e0

 The stdin version is fine. I want to re-use the code for scan_data (and all
 of the other processing methods) so I'd like to be able to iterate over one
 line at a time, independently of the source (i.e. either file or stdin)

 Code that illustrates the issue follows:

 # cat test_fh.py

 #!/usr/bin/env python
 import glob, os, sys
 class TestParse(object):
 def __init__(self):
 if (options.stdin):
   self.scan_data(sys.stdin)
 if (options.glob):
   self.files = glob.glob(options.glob)
   for f in files:
   fh = open(f, 'r')
   self.scan_data(fh)
   fh.close()

 def scan_data(self,fileobject):
 i = int()
 for line in fileobject:
 print i
 i += 1
 # do stuff with the line...
 pass
print finished file

I'd add some print()s in the above loop (and also the 'for f in files'
loop) to make sure the part of the code you didn't want to share (do
stuff with the line) works correctly, and that nothing is improperly
looping in some unexpected way.

Also, there are several series of lines with invalid indentation;
could be an email artifact or could be the cause of your problem. If
the print()s don't yield any useful insight, repost the code again
with absolutely correct indentation.

Finally, some stylistic points:
- don't do 'if (foo):'   use the less noisy 'if foo:' instead
- don't do 'i = int()'   use the more obvious 'i = 0' instead

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: x64 speed

[Robin Becker]

Martin v. Löwis wrote:

Is it the
x64 working faster at its design sizes


Another guess (still from the darkness of not having received the
slightest clue what the test actually does): if it creates integers
in range(2**32, 2**64), then they fit into a Python int on AMD64-Linux,
but require a Python long on 32-bit Windows; long operations are much
slower than int operations.


..
I don't think we're doing a lot of bignum arithmetic, some masking operations 
etc etc.


--
Robin Becker

--
http://mail.python.org/mailman/listinfo/python-list

Arguments for map'ped functions

[mk]

Hello everyone,

So I have this function I want to map onto a list of sequences of 
*several* arguments (while I would want to pass those arguments to each 
function in the normal fashion). I realize this is contrived, maybe an 
example would make this clear:


params = [ ('comp.lang.python', ['rtfm', 'shut']), ('comp.lang.perl', 
['rtfm','shut']) ]


qurls = map(consqurls, params)

def consqurls(args):
ggroup, gkeywords = args
...

Since the argument passed to map'ped function is a single tuple of 
arguments, I have to unpack them in the function. So the question is how 
to pass several arguments to a map'ped function here?



Code in question:

def fillurlfmt(args):
urlfmt, ggroup, gkw = args
return {'group':ggroup, 'keyword':gkw, 'url': urlfmt % (gkw, ggroup)}

def consqurls(args):
ggroup, gkeywords = args
urlfmt = 
'http://groups.google.com/groups/search?as_q=%sas_epq=as_oq=as_eq=num=10scoring=lr=as_sitesearch=as_qdr=as_drrb=bas_mind=1as_minm=1as_miny=1999as_maxd=1as_maxm=1as_maxy=2009as_ugroup=%sas_usubject=as_uauthors=safe=off'

qurls = map(fillurlfmt, [ (urlfmt, ggroup, gkw) for gkw in gkeywords])
return qurls

if __name__ == __main__:
gkeywords = ['rtfm', 'shut']
ggroups = ['comp.lang.python', 'comp.lang.perl']
params = [(ggroup, gkeywords) for ggroup in ggroups]
qurls = map(consqurls, params)
print qurls

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[andrew cooke]

so what is happening with pep 372?

http://www.python.org/dev/peps/pep-0372/
--
http://mail.python.org/mailman/listinfo/python-list

Re: x64 speed

[Robin Becker]

...

--
Ran 193 tests in 27.841s

OK

real0m28.150s
user0m26.606s
sys 0m0.917s
[rpt...@localhost tests]$

magical how the total python time is less than the real time.


time(1) also measures the Python startup and shutdown time, so
I don't quite see the magic :-(



yes stupid me :(


FWIW: VMware VMs need the VMware tools installed to make their
clocks work more or less. With Linux, you need some extra tweaks
as well, otherwise the clocks are just completely unreliable.



I do have the tools installed and from what I can see the clock isn't so far 
off. At least when I run the two tests side by side the vm run always finishes 
first. Of course that could be because vmware is stealing cpu somehow.



See these notes:

http://kb.vmware.com/selfservice/viewContent.do?language=en_USexternalId=1420
http://communities.vmware.com/message/782173




--
Robin Becker

--
http://mail.python.org/mailman/listinfo/python-list

python 3 error i know the file works in python 2.6

[garywood]

can someone help me please 


#open file and read last names
filename = input('name file')
file = open(filename, 'r')
names_list = file.readlines()
file.close()
#open a file for saving passwords
outfile_name = input('Save passwords')
outfile = open(outfile_name, 'a')


#create a password for each name in list
import random, string
name = ('')
for name in names_list:
name_prefix = name[0:2]
number = random.randrange(100,999)
name_prefix = str.upper(name_prefix)
password = name_prefix + str(number)
whole_line = (password)
print (password)
outfile_name.write(whole_line)
outfile_name.close()

print (password)

error
Traceback (most recent call last):
  File C:\Documents and Settings\Gary\Desktop\python\bembry\pa2.i.py, line 
21, in module
outfile_name.write(whole_line)
AttributeError: 'str' object has no attribute 'write'--
http://mail.python.org/mailman/listinfo/python-list

Re: Arguments for map'ped functions

[Peter Otten]

mk wrote:

 So I have this function I want to map onto a list of sequences of
 *several* arguments (while I would want to pass those arguments to each
 function in the normal fashion). I realize this is contrived, maybe an

You can either use a list comprehension

[f(*args) for args in seq]

or starmap()

list(itertools.starmap(f, seq))

Peter
--
http://mail.python.org/mailman/listinfo/python-list

[SOLVED] Re: Should open(sys.stdin) and open(file, 'r') be equivalent?

[Simon Mullis]

Forget it all... I was being very very daft!

The default = 'False' in the options for stdin was not being
evaluated as I thought, so the script was waiting for stdin even when
there was the glob switch was used...No stdin equals the script
seeming to hang.

Ah well.

SM
--
http://mail.python.org/mailman/listinfo/python-list

Flattening lists

[mk]

Hello everybody,

Any better solution than this?

def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
else:
res.append(el)
return res

a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
print flatten(a)


[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Christian Heimes]

andrew cooke schrieb:
 so what is happening with pep 372?
 
 http://www.python.org/dev/peps/pep-0372/

It's still a draft and hasn't been implemented yet. Now is the time to
get it ready for Python 3.1 and 2.7.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 slow file IO

[rdmurray]

Quoth Christian Heimes li...@cheimes.de:
 thomasvang...@gmail.com schrieb:
  I just recently learned python, I'm using it mainly to process huge
  5GB txt files of ASCII information about DNA. I've decided to learn
  3.0, but maybe I need to step back to 2.6?
  
  I'm getting exceedingly frustrated by the slow file IO behaviour of
  python 3.0. I know that a bug-report was submitted here:
  http://bugs.python.org/issue4533. And a solution was posted.
[snip stuff about patching]
 
 I suggest you stick with Python 2.5 or 2.6. Python 3.0 isn't as mature
 as the 2.x series. It's brand new, lot's of things have changed, too.

3.1 will come out much sooner than a normal next dot release python
would, and will contain very significant improvements in the IO speed.
In the meantime, in your case at least, to get real work done you'll
be better off using 2.6.  You can use the 'from __future__ import' to
bring various 3.0 features into 2.6 and make 2.6 look more like 3.0,
and you can use the -3 flag to check your code for 3.0 compatibility, so
that you will be ready to move back to the 3 series when 3.1 comes out.
(Assuming you don't end up needing any 3rd party libraries that haven't
been ported yet by then, that is.)

Unless, of course, you are comfortable with compiling from source
and would like to help _test_ the 3.1 IO library :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

RE: python 3 error i know the file works in python 2.6

[Andreas Tawn]

#open file and read last names
filename = input('name file')
file = open(filename, 'r')
names_list = file.readlines()
file.close()
#open a file for saving passwords
outfile_name = input('Save passwords')
outfile = open(outfile_name, 'a')


#create a password for each name in list
import random, string
name = ('')
for name in names_list:
name_prefix = name[0:2]
number = random.randrange(100,999)
name_prefix = str.upper(name_prefix)
password = name_prefix + str(number)
whole_line = (password)
print (password)
outfile_name.write(whole_line)
outfile_name.close()

print (password)
 
error
Traceback (most recent call last):
  File C:\Documents and Settings\Gary\Desktop\python\bembry\pa2.i.py,
line 21, in module
outfile_name.write(whole_line)
AttributeError: 'str' object has no attribute 'write'

You're trying to output to the output filename string rather that the
output file you opened with that filename.

Cheers,

Drea
--
http://mail.python.org/mailman/listinfo/python-list

Re: Should open(sys.stdin) and open(file, 'r') be equivalent?

[Simon Mullis]

Last try at getting the indenting to appear correctly..

#!/usr/bin/env python

import glob, os, sys

class TestParse(object):
def __init__(self):
if options.stdin:
self.scan_data(sys.stdin)
if options.glob:
self.files = glob.glob(options.glob)
for f in files:
fh = open(f, 'r')
self.scan_data(fh)
fh.close()

def scan_data(self,fileobject):
i = 0
for line in fileobject:
print i
i += 1
# do stuff with the line...
pass
print finished file

def main():
T = TestParse()

if __name__ == __main__:
from optparse import OptionParser
p = OptionParser(__doc__, version=testing 1 2 3)
p.add_option(--glob, dest=glob, help=use this glob)
p.add_option(--stdin, dest=stdin, action=store_true,
default=False, help=use stdin)
(options, args) = p.parse_args()
main()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Brian Allen Vanderburg II]

mrk...@gmail.com wrote:

Hello everybody,

Any better solution than this?

def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
else:
res.append(el)
return res

a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
print flatten(a)


[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list


I think it may be just a 'little' more efficient to do this:

def flatten(x, res=None):
   if res is None:
  res = []

   for el in x:
  if isinstance(el, (tuple, list)):
 flatten(el, res)
  else:
 res.append(el)

   return res

Brian Vanderburg II
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Paul Rubin]

bearophileh...@lycos.com writes:
 Now Ruby dicts are ordered by default:
 http://www.igvita.com/2009/02/04/ruby-19-internals-ordered-hash/

Maybe I didn't read that carefully enough, but it looks like ordered
means the dict records come out in the same order you inserted them
in.  That is if you insert B,A,D,C in that order, you get them out in
that order.  I would have thought an ordered dict meant you get A,B,C,D,
which seems a lot more useful.

Red-black trees as described at the bottom of the page are a good way
to get A,B,C,D.  They also make it simple to have persistent or
functional dictionaries, which I've written about a few times before.
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to find wxPython method documentation??

[Egon Frerich]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

The documentation is here:

http://www.wxpython.org/onlinedocs.php

In Alphabetical class reference you can find for example wxButton with
its methods. Yes - there is no function 'SetBackgroundColour'. But there
is a list with classes from which wxButton is derived.
'SetBackgroundColour' is a method for all widgets so it is described in
~ class wxWindow.

Egon
len schrieb:
| Hi
|
| I am going through the wxPython in Action book by Noel Rappin and
| Robin Dunn.
|
| I have been typing in the example programs as I go and play with
| modifing the code.
| Thought I should start trying to find my way around the documentation
| found on the wxPython web site.
|
| The problem I have been having is I can't find the methods or more
| specifically
|
| 'SetBackgroundColour' or 'AddSimpleTool' for example.
|
| How does one find the methods that are available in the classes.
|
| I tried looking at the wxWidgets web site and still had the same
| problem.
|
| Could someone give me a little direction in this?
|
| Thanks
| Len
| --
| http://mail.python.org/mailman/listinfo/python-list

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJirGrZRiDo9Iq4qIRAoReAJ49ojnUSI9tlCLXn3ynrUDqERykywCePuYB
FOFMykSlCTgON4Ta3eGLPCM=
=0Xq3
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: string replace for back slash

[S.Selvam Siva]

On Thu, Feb 5, 2009 at 5:59 PM, rdmur...@bitdance.com wrote:

 S.Selvam Siva s.selvams...@gmail.com wrote:
  I tried to do a string replace as follows,
 
   s=hi  people
   s.replace(,\)
  'hi \\ people'
  
 
  but i was expecting 'hi \ people'.I dont know ,what is something
 different
  here with escape sequence.

 You are running into the difference between the 'repr' of a string (which
 is what is printed by default at the python prompt) and the actual
 contents of the string.  In the repr the backslash needs to be escaped
 by prefixing it with a backslash, just as you would if you wanted to
 enter a backslash into a string in your program.  If you print the string,
 you'll see there is only one backslash.  Note that you didn't need to
 double the backslash in your replacement string only because it wasn't
 followed by a character that forms an escape...but the repr of that
 string will still have the backslash doubled, and that is really the
 way you should write it in your program to begin with for safety's sake.

 Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13)
 [GCC 4.3.2] on linux2
 Type help, copyright, credits or license for more information.
  s=hi  people
  replacementstring = \
  replacementstring
 '\\'
  print replacementstring
 \
  x = s.replace(,\\)
  x
 'hi \\ people'
  print x
 hi \ people

 --
 http://mail.python.org/mailman/listinfo/python-list



Thank you all for your response,

Now i understood the way python terminal expose '\'.



-- 
Yours,
S.Selvam
--
http://mail.python.org/mailman/listinfo/python-list

os.system issues

[Youri Lammers]

Ok,
 
I want to run a program called 'muscle' with my python script,
muscle uses the following command:
'muscle.exe -in filename -out filename'
so far I got:
 
import os
args = ['-in filename', '-out filename']
os.system('E:\Programs\muscle\muscle.exe args')
 
However, when I run this nothing happends, no error message, nothing.
 
So what am I doing wrong here?
 
Youri
_
De leukste online filmpjes vind je op MSN Video!
http://video.msn.com/video.aspx?mkt=nl-nl--
http://mail.python.org/mailman/listinfo/python-list

Re: How to find wxPython method documentation??

[Mike Driscoll]

On Feb 4, 5:49 pm, andrew cooke and...@acooke.org wrote:
 On Feb 4, 8:06 pm, len lsumn...@gmail.com wrote:

  How does one find the methods that are available in the classes.

 heh.  welcome to the wonderful world of wxpython :o(

 if you use eclipse to edit your code, then (providing the wind is in
 the right direction and the file you are editing doesn't have any
 syntax errors) pressing F3 when you are on a particular method will
 take you to the definition.  or, at least, one of the definitions with
 that name.

 and if you want to see a list of available methods the simplest way is
 to use tab completion (or dir() in python itself).

 if you're not using eclipse (with pydev) check to see if the ide/
 editor you are using has something similar.

 also, wxpython comes with some examples, all packaged in a demo
 program.  that is your best source of documentation.  go through all
 the examples in there and look at the code (the demo program will show
 you the code and even let you edit it and see the results of your
 changes).

 to be honest, wxpython is a bit of a nightmare (imho). but the results
 can be worth it.

 good luck,
 andrew

I don't think the docs are any worse than any other complicated third
party package. Reportlab comes to mind...and the wxPython list members
are very helpful.

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to find wxPython method documentation??

[Mike Driscoll]

On Feb 4, 5:06 pm, len lsumn...@gmail.com wrote:
 Hi

 I am going through the wxPython in Action book by Noel Rappin and
 Robin Dunn.

 I have been typing in the example programs as I go and play with
 modifing the code.
 Thought I should start trying to find my way around the documentation
 found on the wxPython web site.

 The problem I have been having is I can't find the methods or more
 specifically

 'SetBackgroundColour' or 'AddSimpleTool' for example.

 How does one find the methods that are available in the classes.

 I tried looking at the wxWidgets web site and still had the same
 problem.

 Could someone give me a little direction in this?

 Thanks
 Len

In addition to what the others have said, there's also an alternate
doc location done up by one guy who seems to have a lot of free time:

http://xoomer.virgilio.it/infinity77/wxPython/index.html

The wiki is also helpful:

http://wiki.wxpython.org/

But I concur that the mailing list is definitely THE place to be to
learn the most once you've gotten comfortable with the toolkit.

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Brian Allen Vanderburg II wrote:
 def flatten(x):
 res = []
 for el in x:
 if isinstance(el,list):
 res.extend(flatten(el))
 else:
 res.append(el)
 return res


 I think it may be just a 'little' more efficient to do this:

 def flatten(x, res=None):
if res is None:
   res = []

for el in x:
   if isinstance(el, (tuple, list)):
  flatten(el, res)
   else:
  res.append(el)

return res


Hmm why should it be more efficient? extend operation should not be very
costly?

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: os.system issues

[Christian Heimes]

Youri Lammers schrieb:
 Ok,
  
 I want to run a program called 'muscle' with my python script,
 muscle uses the following command:
 'muscle.exe -in filename -out filename'
 so far I got:
  
 import os
 args = ['-in filename', '-out filename']
 os.system('E:\Programs\muscle\muscle.exe args')
  
 However, when I run this nothing happends, no error message, nothing.
  
 So what am I doing wrong here?

For starters you should use the subprocess module instead of os.system.
Next you have either write E:\\... or rE:\... (note the trailing 'r').

Try this:

import subprocess
subprocess.check_call(['E:\\Programs\\muscle\\muscle.exe', '-in',
'filename', '-out', 'filename'])

Christian

--
http://mail.python.org/mailman/listinfo/python-list

os.system issues

[rdmurray]

Youri Lammers youri_lammers...@hotmail.com writes:
 I want to run a program called 'muscle' with my python script=2C
 muscle uses the following command:
 'muscle.exe -in filename -out filename'
 so far I got:
 
 import os
 args = ['-in filename', '-out filename']
 os.system('E:\Programs\muscle\muscle.exe args')
 
 However when I run this nothing happends no error message nothing.
 
 So what am I doing wrong here?

You are calling the program muscle with the argument 'args'.

If you want to put your arguments into the system call, you will
want to write it like this:

os.system('E:\Programs\muscle\muscle.exe -in filename -out filename')

If you need to substitute args into the string passed to
os.system, then take a look at python string handling in the
tutorial.

You also will probably want to look at the 'subprocess' module
at some point, which is more flexible than os.system.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Baolong zhen]

less list creation.

On Thu, Feb 5, 2009 at 10:17 PM, mk mrk...@gmail.com wrote:

 Brian Allen Vanderburg II wrote:
  def flatten(x):
  res = []
  for el in x:
  if isinstance(el,list):
  res.extend(flatten(el))
  else:
  res.append(el)
  return res

 
  I think it may be just a 'little' more efficient to do this:
 
  def flatten(x, res=None):
 if res is None:
res = []
 
 for el in x:
if isinstance(el, (tuple, list)):
   flatten(el, res)
else:
   res.append(el)
 
 return res


 Hmm why should it be more efficient? extend operation should not be very
 costly?


 Regards,
 mk

 --
 http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Brian Allen Vanderburg II wrote:

def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
else:
res.append(el)
return res




I think it may be just a 'little' more efficient to do this:

def flatten(x, res=None):
   if res is None:
  res = []

   for el in x:
  if isinstance(el, (tuple, list)):
 flatten(el, res)
  else:
 res.append(el)

   return res



Hmm why should it be more efficient? extend operation should not be very 
costly?


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Shane Geiger]

These functions come from goopy:


def flatten1(seq):
 
 Return a list with the contents of SEQ with sub-lists and tuples 
exploded.

 This is only done one-level deep.
 

 lst = []
 for x in seq:
   if type(x) is list or type(x) is tuple:
 for val in x:
   lst.append(val)
   else:
 lst.append(x)
 return lst

def flatten(seq):
 
 Returns a list of the contents of seq with sublists and tuples exploded.
 The resulting list does not contain any sequences, and all inner sequences
 are exploded.  For example:

  flatten([7,(6,[5,4],3),2,1])
 [7,6,5,4,3,2,1]
 
 lst = []
 for el in seq:
   if type(el) == list or type(el) is tuple:
 lst.extend(flatten(el))
   else:
 lst.append(el)
 return lst




Brian Allen Vanderburg II wrote:

mrk...@gmail.com wrote:

Hello everybody,

Any better solution than this?

def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
else:
res.append(el)
return res

a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
print flatten(a)


[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list


I think it may be just a 'little' more efficient to do this:

def flatten(x, res=None):
   if res is None:
  res = []

   for el in x:
  if isinstance(el, (tuple, list)):
 flatten(el, res)
  else:
 res.append(el)

   return res

Brian Vanderburg II
--
http://mail.python.org/mailman/listinfo/python-list




--
Shane Geiger, IT Director
Council For Economic Education / www.councilforeconed.org
sgei...@councilforeconed.org  / 402-438-8958

Teaching Opportunity

--
http://mail.python.org/mailman/listinfo/python-list

Re: os.system issues

[Grant Edwards]

On 2009-02-05, Christian Heimes li...@cheimes.de wrote:
 Youri Lammers schrieb:
 Ok,
  
 I want to run a program called 'muscle' with my python script,
 muscle uses the following command:
 'muscle.exe -in filename -out filename'
 so far I got:
  
 import os
 args = ['-in filename', '-out filename']
 os.system('E:\Programs\muscle\muscle.exe args')
  
 However, when I run this nothing happends, no error message, nothing.
  
 So what am I doing wrong here?

 For starters you should use the subprocess module instead of os.system.
 Next you have either write E:\\... or rE:\... (note the trailing 'r').

Leading 'r'.

Or just skip cmd.exe and use forward slashes like god intended.

-- 
Grant Edwards   grante Yow! I'd like MY data-base
  at   JULIENNED and stir-fried!
   visi.com
--
http://mail.python.org/mailman/listinfo/python-list

Cheetah and hungarian charset...

[durumdara]

Hi!

I wanna ask that have anyone some exp. with Cheetah and the non-ascii chars?

I have a site. The html template documents are saved in ansi format, psp 
liked them.
But the cheetah parser makes ParseError on hungarian characters, like 
á, é, í, etc. When I remove them, I got good result, but I don't 
want to remove them all...


I cannot parse them. Please help me, how to force cheetah to eat them all?

Thanks for your help:
dd

--
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter

[Luke]

Thanks, Its working smoothly now
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Mark Dickinson]

On Feb 5, 1:17 pm, mk mrk...@gmail.com wrote:
 Hello everybody,

 Any better solution than this?

 def flatten(x):

Just out of interest, how often do people really need
such a recursive flatten, as opposed to a single-level
version?

I often find myself needing a 'concat' method that
turns a list of lists (or iterable of iterables) into
a single list;  itertools.chain does this quite nicely.
But I don't think I've ever encountered a need for the
full recursive version.

Mark

--
http://mail.python.org/mailman/listinfo/python-list

Re: [Web 2.0] Added-value of frameworks?

[J Kenneth King]

Bruno Desthuilliers bdesth.quelquech...@free.quelquepart.fr writes:

 Gilles Ganault a écrit :
 Hello

 If I wanted to build some social web site such as Facebook, what do
 frameworks like Django or TurboGears provide over writing a site from
 scratch using Python?

 Quite a lot of abstractions and factorisation of the boilerplate code,
 a known way to organize your application, and possibly a good
 integration of the usual components (and wrt/ Django, a customizable
 yet fairly usable OOTB admin interface). For simple to mediumly
 complex applications, this can mean more than 80% of the grunt
 work. The counterpart is mostly learning and understanding the
 framework, which means you may not save that much time on a first
 project - but it can really pay off then. One of my coworker started
 this morning a (really simple...) project which acceptance tests are
 to begin on monday, and we are all quite confident he'll deliver on
 time, thanks to Django.

Well the big negative is when you application design starts expanding
past the framework design. A percolator makes a decent cup of coffee but
it really isn't meant for making fancy lattes. This is where the 90/10
rule will catch you if you're not careful.

90% of the effort will be spent on the last 10% of the work.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Baolong zhen wrote:

less list creation.


At the cost of doing this at each 'flatten' call:

if res is None:
   res = []

The number of situations of executing above code is the same as the 
number of list creations (once for each 'flatten' call, obviously).


Is list creation really more costly than above?

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: time: Daylight savings confusion

[MRAB]

Matt Nordhoff wrote:

Tim H wrote:

On Win XP 64bit, Python 2.6.1 64bit

I am trying to rename files by their creation time.

It seems the time module is too smart for its own good here.

time.localtime(os.path.getctime(f)) returns a value one hour off
from what windows reports for files that were created when Daylight
savings time was in effect (ie when the tm_isdst field is one).  I
can kludge it, but am I missing the right way to do it?

Tim

full code:


snip full code


From what I remember, this is a bug in the Win32 API. Python
probably could hack around it, but, well, I guess it doesn't.


Googling a bit, Perl's Win32::UTCFileTime module [1] provides 
workarounds (not for Python, of course, but it could probably be

ported) and explains all the gory details (which I have not read, so
maybe I misunderstand ;-).

[1] 
http://search.cpan.org/~shay/Win32-UTCFileTime-1.50/lib/Win32/UTCFileTime.pm

 http://xrl.us/beezsv


Windows does apply the current value of DST when converting from UTC to 
local time.


The problem is that the rules for when DST starts and ends might change 
over time and from place to place, and local time depends on the

timezone too.

Probably the best you can do is just to apply the current rules and 
accept that the result might not match what the local time actually was.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Michele Simionato]

On Feb 5, 2:17 pm, mk mrk...@gmail.com wrote:
 Hello everybody,

 Any better solution than this?

 def flatten(x):
      res = []
      for el in x:
          if isinstance(el,list):
              res.extend(flatten(el))
          else:
              res.append(el)
      return res

 a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
 print flatten(a)

 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

 Regards,
 mk

Looks fine to me. In some situations you may also use hasattr(el,
'__iter__') instead of isinstance(el, list) (it depends if you want to
flatten generic iterables or only lists).
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.

[MRAB]

Mark Dickinson wrote:

On Feb 5, 1:18 am, Chris Rebert c...@rebertia.com wrote:

For an integer:
is_even = bin(the_int)[2:].count('1') % 2 == 0


But the OP has to use if and while.  How about:

while 2+2 != 5:
if 'wkw' in 'just being awkward':
is_even = bin(the_int)[2:].count('1') % 2 == 0
break

or (Python 2.5 compatible):

def count_set_bits(n):
# make sure we include an if, to
# satisfy OP's requirements:
if n  0:
raise ValueError
count = 0
while n:
count += 1
n = n-1
return count

is_even = count_set_bits(the_int) % 2 == 0

...but anyone submitting this as a homework
solution had better be prepared to explain why
it works.


I haven't seen that algorithm before. Very clever! :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Cheetah and hungarian charset...

[Diez B. Roggisch]

durumdara schrieb:

Hi!

I wanna ask that have anyone some exp. with Cheetah and the non-ascii 
chars?


I have a site. The html template documents are saved in ansi format, psp 
liked them.
But the cheetah parser makes ParseError on hungarian characters, like 
á, é, í, etc. When I remove them, I got good result, but I don't 
want to remove them all...


I cannot parse them. Please help me, how to force cheetah to eat them all?


Please provide tracebacks and code and data examples.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[jason-sage]

mk wrote:

Hello everybody,

Any better solution than this?

def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
else:
res.append(el)
return res

a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
print flatten(a)


It depends on what you mean by better.  More features?  Here is the 
function from Sage (http://www.sagemath.org), which is a modified 
version of a more standard implementation to give a max_level argument.


The first few lines of documentation:

def flatten(in_list, ltypes=(list, tuple), max_level=sys.maxint):
  
  Flattens a nested list.

  INPUT:
  in_list -- a list or tuple
  ltypes -- optional list of particular types to flatten
  max_level -- the maximum level to flatten

  OUTPUT:
  a flat list of the entries of in_list

(lots of examples follow this documentation)

The implementation:  
http://www.sagemath.org/hg/sage-main/file/b0aa7ef45b3c/sage/misc/flatten.py



Jason

--
http://mail.python.org/mailman/listinfo/python-list

Re: global name 'sqrt' is not defined

[Scott David Daniels]

M.-A. Lemburg wrote:

On 2009-02-05 10:08, Nick Matzke wrote:

..., I can run this in the ipython shell just fine:
a = [12, 15, 16, 38.2]
dim = int(sqrt(size(a)))
...But if I move these commands to a function in another file, it freaks out:

You need to add:

from math import sqrt

or:
from cmath import sqrt
or:
from numpy import sqrt

Each with their own, slightly different, meaning.
Hence the reason many of us prefer to import the module
and reference the function as a module attribute.

Note that _many_ (especially older) package documents describe
their code without the module name.  I believe that such behavior
is because, when working to produce prose about a package, it
feels too much like useless redundancy when describing each function
or class as package.name.


--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Is c.l.py becoming less friendly?

[mk]

(duck)

542 comp.lang.python rtfm

467 comp.lang.python shut+up

263 comp.lang.perl rtfm

45 comp.lang.perl shut+up




Code:

import urllib2
import re
import time

def fillurlfmt(args):
urlfmt, ggroup, gkw = args
return {'group':ggroup, 'keyword':gkw, 'url': urlfmt % (gkw, ggroup)}

def consqurls(args):
ggroup, gkeywords = args
urlfmt = 
'http://groups.google.com/groups/search?as_q=%sas_epq=as_oq=as_eq=num=10scoring=lr=as_sitesearch=as_drrb=qas_qdr=as_mind=1as_minm=1as_miny=1999as_maxd=1as_maxm=1as_maxy=2009as_ugroup=%sas_usubject=as_uauthors=safe=off'

qurls = map(fillurlfmt, [ (urlfmt, ggroup, gkw) for gkw in gkeywords ])
return qurls

def flatten_list(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten_list(el))
else:
res.append(el)
return res

def ggsearch(urldict):
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0 (Windows; U; 
Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 (CK-IBM) 
Firefox/2.0.0.20')]

time.sleep(0.1)
urlf = opener.open(urldict['url'])
resdict = {'result': urlf.read()}
resdict.update(urldict)
urlf.close()
return resdict

def extrclosure(resregexp, groupno):
def extrres(resdict):
txtgr = resregexp.search(resdict['result'])
resdict['result']=txtgr.group(groupno)
return resdict
return extrres

def delcomma(x):
x['result'] = x['result'].replace(',','')
return x

if __name__ == __main__:
gkeywords = ['rtfm', 'shut+up']
ggroups = ['comp.lang.python', 'comp.lang.perl']
params = [(ggroup, gkeywords) for ggroup in ggroups]
qurls = map(consqurls, params)
qurls = flatten_list(qurls)
gresults = map(ggsearch, qurls)
resre = re.compile('Results \b\1\/b\ - \b\.+?\/b\ of about 
\b\(.+?)\/b\')

gextrsearchresult = extrclosure(resre,1)
gresults = map(gextrsearchresult, gresults)
gresults = map(delcomma, gresults)
for el in gresults:
print el['result'], el['group'], el['keyword']
print


This was inspired by 
http://mail.python.org/pipermail/python-list/2002-November/172466.html


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Stephen Hansen]

 That page about Ruby dicts show a higher traversal speed (probably
 just because the CPU has to scan less memory, but I am not sure,
 because mordern CPUs are very complex) but lower insertion speed (I
 think mostly not because the added management of two pointers, but
 because the memory used increases, so there are more cache misses. If
 this turns out as true, then using the xor trick may halve the extra
 memory needed). I have no idea if such different balance of
 performance will lead to on average faster or slower Python programs,
 this has to be tested. But even if the average performance becomes a
 little worse I think making the default Python dict as ordered is a
 positive change for Python too, because built-ins are meant to be as
 flexible as possible, even if they aren't the fastest ones or the less
 memory hungry ones (and keeping dicts ordered decreases the surprises
 a newbie finds, makes some code cleaner, and doctests become simpler
 to write because the order of items may be more defined).

Er, doing a cursory review of my code has an overwhelmingly higher incidence
of insertions then traversals with my dictionaries: many dictionaries are
never traversed and when they are it is only once, and often had tens to
thousands of insertions before that traversal happens.

So a slight decrease in insertion speed at the cost of faster traversal seems
like a complete loss to me.

Now, I also do recognize the utility of ordered dictionaries in some cases, but
exactly what you mean by ordered varies. I have two cases where ordered
has the keys are in a specific custom order. I have four cases where ordered
means maintaining insertion order. For the former I do sorted(dict) on access
when I need to do something order-related. For the latter I have a simple
ordered dictionary implementation that maintains the keys as a separate list
attached to the dictionary.

But interestingly enough to me, in all the cases where I use an ordered dict
in my code, performance is completely irrelevant. Its tens to hundreds of
items usually, and the best optimization in the world wouldn't add more then
an imperceptible fraction of a second, and the memory lost to the extra
list adds up to nothing relevant.

I'm not claiming my use cases are everyones, not by any means. But: this
seems like a complete lose-lose for an idea from my perspective. Sure you
say an unordered dictionary could then be added for my needs, but... I like
my syntactic sugar, thank you :)

Ordered dictionaries can be useful, its good to see a standard implementation
going into the core, but the default? I hope not!

--Stephen
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is c.l.py becoming less friendly?

[Dan Upton]

On Thu, Feb 5, 2009 at 11:00 AM, mk mrk...@gmail.com wrote:

 (duck)

 542 comp.lang.python rtfm

 467 comp.lang.python shut+up

 263 comp.lang.perl rtfm

 45 comp.lang.perl shut+up

But over how many messages for each group?  Wouldn't the percentage of
messages containing those be more interesting than the raw number? ;p
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[Stephen Hansen]

 Now, I also do recognize the utility of ordered dictionaries in some cases, 
 but
 exactly what you mean by ordered varies. I have two cases where ordered
 has the keys are in a specific custom order. I have four cases where ordered
 means maintaining insertion order. For the former I do sorted(dict) on access
 when I need to do something order-related. For the latter I have a simple
 ordered dictionary implementation that maintains the keys as a separate list
 attached to the dictionary.

Ooh, as an addendum... I found one case where I want insertion-and-update
order: meaning that its an ordered dictionary that maintains insertion
order, but
an update to a particular item moves that item to the back so an update behaves
like del d[key]; d[key] = value in terms of the key order.

Ordered Dictionaries are a bit fuzzy. :)

--S
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Mark Dickinson wrote:

I often find myself needing a 'concat' method that
turns a list of lists (or iterable of iterables) into
a single list;  itertools.chain does this quite nicely.
But I don't think I've ever encountered a need for the
full recursive version.



You're most probably right in this; however, my main goal here was 
finding 'more Pythonic' way of doing this and learning this way rather 
than the practical purpose of flattening deeply nested lists.


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Michele Simionato wrote:


Looks fine to me. In some situations you may also use hasattr(el,
'__iter__') instead of isinstance(el, list) (it depends if you want to
flatten generic iterables or only lists).


Thanks! Such stuff is what I'm looking for.

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Brian Allen Vanderburg II]

mrk...@gmail.com wrote:

Baolong zhen wrote:

less list creation.


At the cost of doing this at each 'flatten' call:

if res is None:
   res = []

The number of situations of executing above code is the same as the 
number of list creations (once for each 'flatten' call, obviously).


Is list creation really more costly than above?

Probably not.  I wrote a small test program using a list several levels 
deep, each list containing 5 sublists at each level and finally just a 
list of numbers.  Flattening 1000 times took about 3.9 seconds for the 
one creating a list at each level, and 3.2 for the one not creating the 
list at each level.


Brian Vanderburg II
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is c.l.py becoming less friendly?

[Tim Rowe]

2009/2/5 mk mrk...@gmail.com:

 (duck)

 542 comp.lang.python rtfm

 467 comp.lang.python shut+up

 263 comp.lang.perl rtfm

 45 comp.lang.perl shut+up

Yes, but is there any real traffic on comp.lang.perl nowadays?

Sorry, cheap shot ;-)

-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Couple of noobish question

[Bruno Desthuilliers]

Tim Rowe a écrit :

2009/2/4 Bruno Desthuilliers bdesth.quelquech...@free.quelquepart.fr:


# somemodule.py

import os

if os.uname()[0] == Linux:


On an MS Windows system, os.uname()[0] raises an AttributeError


Thanks for the correction - as you may have guessed, I have not used 
windows for years !-)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[MRAB]

Paul Rubin wrote:

bearophileh...@lycos.com writes:

Now Ruby dicts are ordered by default:
http://www.igvita.com/2009/02/04/ruby-19-internals-ordered-hash/


Maybe I didn't read that carefully enough, but it looks like ordered
means the dict records come out in the same order you inserted them
in.  That is if you insert B,A,D,C in that order, you get them out in
that order.  I would have thought an ordered dict meant you get A,B,C,D,
which seems a lot more useful.


[snip]
You're confusing ordered with sorted! :-)

A list is an ordered collection, for example.
--
http://mail.python.org/mailman/listinfo/python-list

Is the subprocess module robust enough in 2.4?

[skip]

The subprocess module was added in Python 2.4.  I'm running 2.4.5 at work.
I know it's seen many bugfixes since first released.  Is the version in 2.4
robust enough to use in preference to os.popen and friends?

Thx,

-- 
Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is c.l.py becoming less friendly?

[Diez B. Roggisch]

mk schrieb:


(duck)

542 comp.lang.python rtfm

467 comp.lang.python shut+up

263 comp.lang.perl rtfm

45 comp.lang.perl shut+up


It appears to me that comp.lang.perl isn't even active anymore. Or 
googles interface is just crappy.


c.l.perl.misc seems to be the place to search.

And raw numbers are nothing without actual postings. FWIW, I can't 
remember reading the last time RTFM here, but then I usually skip the 
giant-size horse-beatings about GIL removal and encapsulation...



Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[mk]

Brian Allen Vanderburg II wrote:


Is list creation really more costly than above?

Probably not.  I wrote a small test program using a list several levels 
deep, each list containing 5 sublists at each level and finally just a 
list of numbers.  Flattening 1000 times took about 3.9 seconds for the 
one creating a list at each level, and 3.2 for the one not creating the 
list at each level.


Hmm, I'm surprised by even that! Apparently list creation is more 
expensive than I thought - it seems somewhat more expensive than the 
cost of interpreting bytecode for if var is None. Either list creation 
is somewhat costly, or if var is None is really cheap.


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Stephen Hansen]

 Either list creation is somewhat
 costly, or if var is None is really cheap.

if x is y is extremely cheap, I believe. Unlike most comparisons
which are (relatively) expensive, that one is just comparing simple
object address. You can't override is so there's a whole series of
checks that don't have to get done. You don't have to go through the
richcompare machinery, check if there's a __ne__, etc, etc.

--S
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Baolong zhen netz...@gmail.com wrote:
 On Thu, Feb 5, 2009 at 10:17 PM, mk mrk...@gmail.com wrote:

  Brian Allen Vanderburg II wrote:
   def flatten(x):
   res = []
   for el in x:
   if isinstance(el,list):
   res.extend(flatten(el))
   else:
   res.append(el)
   return res
 
  
   I think it may be just a 'little' more efficient to do this:
  
   def flatten(x, res=None):
  if res is None:
 res = []
  
  for el in x:
 if isinstance(el, (tuple, list)):
flatten(el, res)
 else:
res.append(el)
  
  return res
 
 
  Hmm why should it be more efficient? extend operation should not be very
  costly?

 less list creation.

(Took me a while to find the content of your post because
you top posted it.  I've taken the liberty of correcting that.)

This is all premature optimization, except for the goopy code, which is
presumably used enough to make it worth optimizing.  And guess what?
The goopy code wins.  What the people theorizing about the speed of
extend vs list creation miss is that the things with high overhead in the
above functions are (1) isinstance and (2) the recursive function call.
The goopy code avoids this by using type and is, and by unrolling the
lowest level without a function call.  On the other hand, extend
_is_ faster than append if you aren't creating a new list, so the
goopy code can be optimized a little more.

I remembered the bit about high function call overhead, but the rest
of it measured:

temp.py
-
from __future__ import print_function
from timeit import Timer

def flatten1(x):
res = []
for el in x:
if isinstance(el, list):
res.extend(flatten1(el))
else:
res.append(el)
return res

def flatten1a(x):
res = []
for el in x:
if isinstance(el, (tuple, list)):
res.extend(flatten1a(el))
else:
res.append(el)
return res

def flatten1b(x):
res = []
for el in x:
if type(el) is list or type(el) is tuple:
res.extend(flatten1b(el))
else:
res.append(el)
return res


def flatten2(x, res=None):
if res is None:
   res = []
for el in x:
   if isinstance(el, list):
  flatten2(el, res)
   else:
  res.append(el)
return res


def flatten2a(x, res=None):
if res is None:
   res = []
for el in x:
   if isinstance(el, (tuple, list)):
  flatten2a(el, res)
   else:
  res.append(el)
return res


def flatten2b(x, res=None):
if res is None:
   res = []
for el in x:
   if type(el) is list or type(el) is tuple:
  flatten2b(el, res)
   else:
  res.append(el)
return res



def flatten3z(seq):
  lst = []
  for x in seq:
if type(x) is list or type(x) is tuple:
  for val in x:
lst.append(val)
else:
  lst.append(x)
  return lst

def flatten3(seq):
  lst = []
  for el in seq:
if type(el) == list or type(el) is tuple:
  lst.extend(flatten3z(el))
else:
  lst.append(el)
  return lst


def flatten3y(seq):
  lst = []
  for x in seq:
if type(x) is list or type(x) is tuple:
  lst.extend(x)
else:
  lst.append(x)
  return lst

def flatten3a(seq):
  lst = []
  for el in seq:
if type(el) == list or type(el) is tuple:
  lst.extend(flatten3y(el))
else:
  lst.append(el)
  return lst



l = [[1, 2, 3], 5, [7, 8], 3, [9, [10, 11, 12], 4, [9, [10, 5, 7]], 1, [5, 
4], 3], 4, 3], 3, 1, 45], 9], 10]]

cases = dict(
base = Timer(),
c1 = Timer(flatten1(l), from temp import flatten1, l),
c1a = Timer(flatten1a(l), from temp import flatten1a, l),
c1b = Timer(flatten1b(l), from temp import flatten1b, l),
c2 = Timer(flatten2(l), from temp import flatten2, l),
c2a = Timer(flatten2a(l), from temp import flatten2a, l),
c2b = Timer(flatten2b(l), from temp import flatten2b, l),
c3 = Timer(flatten3(l), from temp import flatten3, l),
c3a = Timer(flatten3a(l), from temp import flatten3a, l),
)

if __name__==__main__:
for (name, case) in sorted(cases.items()):
print({0:4s} {1}.format(name, case.timeit()))

-

It is also interesting to note that python3.0 is faster in this
particular case (unless there are timing vagrancies on my machine, which
is possible, though the results were fairly consistent over several runs
of the script).  The second run below is using python2.6.1.

src/python/Python-3.0/python temp.py
base 0.0278329849243
c1   30.4776289463
c1a  44.3886289597
c1b  32.5621030331
c2   25.6131818295
c2a  39.0944678783
c2b  27.1573381424
c3   15.346280098
c3a  14.3178970814

python temp.py 
base 0.0288269519806
c1   35.8193409443
c1a  52.3054969311
c1b  36.3652667999
c2   32.3255820274
c2a  

Re: Flattening lists

[J Kenneth King]

mk mrk...@gmail.com writes:

 Hello everybody,

 Any better solution than this?

 def flatten(x):
 res = []
 for el in x:
 if isinstance(el,list):
 res.extend(flatten(el))
 else:
 res.append(el)
 return res

 a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
 print flatten(a)


 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

 Regards,
 mk

http://mail.python.org/pipermail/python-list/2005-July/330367.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[Sion Arrowsmith]

mk  mrk...@gmail.com wrote:
Brian Allen Vanderburg II wrote:
 I think it may be just a 'little' more efficient to do this:
 
 def flatten(x, res=None):
if res is None:
   res = []
for el in x:
   if isinstance(el, (tuple, list)):
  flatten(el, res)
   else:
  res.append(el)
return res


Hmm why should it be more efficient [than
def flatten(x):
res = []
for el in x:
if isinstance(el,list):
res.extend(flatten(el))
   else:
res.append(el)
return res

]? extend operation should not be very costly?

It's not a question of extend/append, it's the fact that your
original function creates (and destroys) a new list for every
recursive call. Which, if you've got large nested lists, will
have an impact.

-- 
\S -- si...@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
   Frankly I have no feelings towards penguins one way or the other
-- Arthur C. Clarke
   her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
--
http://mail.python.org/mailman/listinfo/python-list

Re: Scanning a file character by character

[Gabriel Genellina]

En Thu, 05 Feb 2009 04:48:13 -0200, Spacebar265 spacebar...@gmail.com
escribió:


Hi. Does anyone know how to scan a file character by character and
have each character so I can put it into a variable. I am attempting
to make a chatbot and need this to read the saved input to look for
spelling mistakes and further analysis of user input.


Read the file one line at a time, and process each line one character at a
time:

with open(filename, r) as f:
   for line in f:
 for c in line:
   process(c)

But probably you want to process one *word* at a time; the easiest way
(perhaps inaccurate) is to just split on whitespace:

   ...
 for word in line.split():
   process(word)

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Structuring Modules with a Ubiquitous Base Class (Circular Dependencies)

[Gabriel Genellina]

En Wed, 04 Feb 2009 21:12:58 -0200, andrew cooke and...@acooke.org
escribió:


On Feb 4, 7:49 pm, andrew cooke and...@acooke.org wrote:

This leads to a circular dependency - the base class wants to import
the components, which in turn want to import the base class.


well, to partially answer my own question, this is certainly
possible.  in the general case it might get quite complex, but for the
structure described it is quite simple.  the 'trick' is to import the
component in the *method* of the common base class.

for example:

class Base(object):

def __init__(self):
from first import First
from second import Second
self.first = lambda *args: First(*args)
self.second = lambda *args: Second(*args)

where First, defined in first, subclasses Base in the normal way.

however, i suspect this will have a performance hit, since linking
is being done at run time rather than compile time.  anyone have
any guidance on how serious that would be?  i guess it could be
ameliorated by doing the work in the constructor (as above, which is
just that way for a compact example - in real life the components
are used in a more complex manner).


There is no linking step in Python, all is done at run time.
Once a module is imported the first time, any subsequent import is very
cheap.


and i still suspect there is a more efficient metaclass or similar
approach.


This approach is fine to me.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

HOWTO for setting up a PyQt project in Eclipse ?

[Linuxguy123]

Does anyone know of a HOWTO for setting up a PyQt project in Eclipse ?

I know about setting up a PyDev project, just wondering how to integrate
the QtDesigner parts.  

For example, should I save the QtDesigner project in the root PyDev
directory ?

Thanks

--
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess.Popen not creating a pipe

[Gabriel Genellina]

En Sun, 01 Feb 2009 18:00:36 -0200, Andrew Parker gbofs...@gmail.com
escribió:


On Sun, Feb 1, 2009 at 1:46 PM, Andrew Parker gbofs...@gmail.com wrote:

I'm having some fun with Popen.  I have the following line:

   process = subprocess.Popen(command, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
   print process.stdout

Under normal circumstances, this displays:

   open file 'stdout', mode 'w' at 0xb7f8e068

However, I have a binary that I use to kick off this script, and when
that runs, it displays:

   None

So, two questions:

   1.  What the heck is this binary doing that upsets Popen so much?
   2.  What can *my script* do to get around this problem.

Unfortunately I'm stuck using this binary, so its the python where I
have to solve this.


so, tracing through subprocess.Popen, I see that os.pipe() is being
invoked for stdout.  This is returning (0, 3), and I assume the 0 is
conflicting with what python is assuming is stdin.  Calling pipe()
before Popen gets around my problem, as the pipe the Popen gets then
returns (4,5) which Popen seems happy with.


Seems that your binary is closing stdin before running your script, so
the first available file descriptor is 0.

Sounds like a bug.  Should I report this, or is it expected/known  
behaviour?


It's hardly a Python error, and I don't think the subprocess module should
consider this very special case, but you may report it at
http://bugs.python.org/ anyway.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Is c.l.py becoming less friendly?

[Tim Chase]

(duck)

542 comp.lang.python rtfm
467 comp.lang.python shut+up
263 comp.lang.perl rtfm
45 comp.lang.perl shut+up


Is this where we tell you to shut up? gdr ;-)

As others mentioned, the raw numbers don't mean much without a 
total-volume-of-posts to demonstrate the percentage.


It would also be interesting to see how many of those posts are 
concentrated in certain threads -- for the most part, c.l.python 
has been a pretty civil place, but a few threads have degraded 
into puerile spats.


-tkc



--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Quoth rdmur...@bitdance.com:
 This is all premature optimization, except for the goopy code, which is
 presumably used enough to make it worth optimizing.  And guess what?
 The goopy code wins.  What the people theorizing about the speed of
 extend vs list creation miss is that the things with high overhead in the
 above functions are (1) isinstance and (2) the recursive function call.
 The goopy code avoids this by using type and is, and by unrolling the
 lowest level without a function call.  On the other hand, extend
 _is_ faster than append if you aren't creating a new list, so the
 goopy code can be optimized a little more.
 
 I remembered the bit about high function call overhead, but the rest
 of it measured:

Oooh, that's embarrassing.  Not only didn't I read the code carefully
enough, I didn't test the actual output of the functions.  The goopy
code doesn't flatten to arbitrary depth, so of course it is faster.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: sorting mesh data an from abaqus simulation

[Gabriel Genellina]

En Mon, 02 Feb 2009 10:10:15 -0200, Alessandro Zivelonghi
zasaconsult...@gmail.com escribió:


*Ntop = odb.rootAssembly.instances['PART-1-1'].nodeSets['TOP'].nodes
*
Problem:
1) the list of nodes Ntop contains all the node labels [2673,   2675,
2676,   2677,   2678,   3655,   3656, 119939, 124154, 127919] already
ordered in ascending order.
What I need is the same list *ordered by coordinate_x* of each node, i.e.
from left to right in the model (unfortunately, for example node 124154
cames before node 3656 in the model, if you read the model from left to
right)


Use the key= argument to the sorted() builtin function
http://docs.python.org/library/functions.html#sorted

def get_node_x(node):
   return node.coordinates[0]

sorted_nodes = sorted(Ntop, key=get_node_x)

(For a list of reasonable size, that's fine. For a giant list, this could
be written more efficiently, although less readable)


1b) I don't understand which kind of data are EL, N, Ntop (list?) and how
can I sort Ntop  with the criterium based on coordinate_x (coordinate[0])


AFAIK those are plain lists. You can try:

py print type(Ntop)

and see what happens. If you don't get type 'list', try with:

py type(Ntop).mro()

to see the chain of base classes.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list