Re: html parser , unexpected '<' char in declaration

2006-02-21 Thread Tim Roberts
"Jesus Rivero - (Neurogeek)" <[EMAIL PROTECTED]> wrote:
>
>hmmm, that's kind of different issue then.
>
>I can guess, from the error you pasted earlier, that the problem shown
>is due to the fact Python is interpreting a "<" as an expression and not
>as a char. review your code or try to figure out the exact input you're
>receving within the mta.

Well, Jesus, you are 0 for 2.  Sakcee pointed out what the exact problem
was in his original message.  The HTML he is being given is ill-formed; the
 tag
which it thinks is inside the > well probabbly I should explain more.  this is part of an email . after
>> the mta delivers the email, it is stored in a local dir.
>> After that the email is being parsed by the parser inside an web based
>> imap client at display time.
>> 
>> I dont think I have the choice of rewriting the message!? and I dont
>> want to reject the message alltogether.
>> 
>> I can either 1-fix the incoming html by tidying it up
>> or 2- strip only plain text out and dispaly that you have spam, 3 - or
>> ignore that mal-formatted tag and display the rest

If this is happening with more than one message, you could check for it
rather easily with a regular expression, or even just ''.find, and then
either insert a closing '>' or delete everything up to the  before
parsing it.
-- 
- Tim Roberts, [EMAIL PROTECTED]
  Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to break a for loop?

2006-02-21 Thread Steven D'Aprano
Petr Jakes wrote:

> zero_list=[0,0,0,0,0,1,2,3,4]
> for x in range(len(zero_list)):
> if zero_list[x]!=0: break
> nonzero_list=zero_list[x:]
> print nonzero_list
> 
> but some more "pythonic" solutions will be posted from other users I
> guess :)


That looks perfectly Pythonic to me.

You know, just because Python has for loops and 
multi-line blocks of code doesn't mean we shouldn't use 
them *wink*


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


fast text processing

2006-02-21 Thread Alexis Gallagher
(I tried to post this yesterday but I think my ISP ate it. Apologies if 
this is a double-post.)

Is it possible to do very fast string processing in python? My 
bioinformatics application needs to scan very large ASCII files (80GB+), 
compare adjacent lines, and conditionally do some further processing. I 
believe the disk i/o is the main bottleneck so for now that's what I'm 
optimizing. What I have now is roughly as follows (on python 2.3.5).

filehandle = open("data",'r',buffering=1000)

lastLine = filehandle.readline()

for currentLine in filehandle.readlines():

 lastTokens = lastLine.strip().split(delimiter)
 currentTokens = currentLine.strip().split(delimiter)

 lastGeno = extract(lastTokens[0])
 currentGeno = extract(currentTokens[0])

 # prepare for next iteration
 lastLine = currentLine

 if lastGeno == currentGeno:
table.markEquivalent(int(lastTokens[1]),int(currentTokens[1]))

So on every iteration I'm processing mutable strings -- this seems 
wrong. What's the best way to speed this up? Can I switch to some fast 
byte-oriented immutable string library? Are there optimizing compilers? 
Are there better ways to prep the file handle?

Perhaps this is a job for C, but I am of that soft generation which 
fears memory management. I'd need to learn how to do buffered reading in 
C, how to wrap the C in python, and how to let the C call back into 
python to call markEquivalent(). It sounds painful. I _have_ done some 
benchmark comparisons of only the underlying line-based file reading 
against a Common Lisp version, but I doubt I'm using the optimal 
construct in either language so I hesitate to trust my results, and 
anyway the interlanguage bridge will be even more obscure in that case.

Much obliged for any help,
Alexis
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python vs. Lisp -- please explain

2006-02-21 Thread Paul Boddie
Delaney, Timothy (Tim) wrote:
> Donn Cave wrote:
>
> > for very long.  If you give me a Python program, you have 3 choices:
> > cross your fingers and hope that I have the required Python
> > interpreter version, slip in a 25Mb Python interpreter install and
> > hope I won't notice, or come clean and tell me that your program
> > needs an interpreter and I should check to see that I have it.
>
> Or use the correct term - a Python Virtual Machine.

Well, yes, I suppose you could have something just interpreting
bytecodes and with no support for parsing and compiling Python the
language - I suppose PyMite [1, 2] and other similar works are like
that.

> The the compiler is built into the VM as opposed to a separate tool
> (like Java) is just an implementation issue. There's nothing stopping
> you from compiling Python source offline and just distributing the
> bytecode (and this is commonly done) - although in that case you are
> usually tied to a specific PVM major version.

Yes, but you can run something like gcj on Java programs and get an
executable which, if I'm not mistaken, doesn't contain any Java VM
whatsoever. There are packagers and other tools for Python which
produce executables, but most of them bundle some kind of bytecode
interpreter, with the possible exception of things like "Python to C"
and, if you're not actually writing Python, Pyrex. However, I believe
that jythonc also produces Java class files with no dependencies on an
embedded VM.

Paul

[1] http://www.python.org/pycon/papers/pymite/
[2] http://wiki.python.org/moin/PyMite

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to break a for loop?

2006-02-21 Thread Giovanni Bajo
[EMAIL PROTECTED] wrote:

>> I need to remove zeros from
>> the begining of list, but I can't :-(.
>
> I believe the following is almost a direct translation of the above
> sentence.
>
> import itertools as it
> a=[0,0,0,1,0]
> a[:]=it.dropwhile(lambda x: x is 0, a)


Usa lambda x: x==0 or simply lambda x: x

Using 'is' relies on the interpreter reusing existing instances of the
immutable object '0', which is an implementation detail.
-- 
Giovanni Bajo


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Safe Python Execution

2006-02-21 Thread Paul Boddie
[EMAIL PROTECTED] wrote:
>
> Is anyone aware of a more functional but still untrusted python?

Given that you've looked into Zope 3's security/proxy mechanisms, have
you also looked at mxProxy?

http://www.egenix.com/files/python/mxProxy.html

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: In need of a virtual filesystem / archive

2006-02-21 Thread Steven D'Aprano
Enigma Curry wrote:

> I need to store a large number of files in an archive. From Python, I
> need to be able to create an archive, put files into it, modify files
> that are already in it, and delete files already in it.
> 
> The easy solution would be to use a zip file or a tar file. Python has
> good standard modules for accessing those types. However, I would tend
> to think that modifying or deleting files in the archive would require
> rewriting the entire archive.
> 
> Is there any archive format that can allow Python to modify a file in
> the archive *in place*? That is to say if my archive is 2GB large and I
> have a small text file in the archive I want to be able to modify that
> small text file (or delete it) without having to rewrite the entire
> archive to disk.

Yes. I believe your common or garden variety file 
manager can handle this task, by storing files in an 
archive called "a directory". For example, many mail 
systems use the "maildir" archive for storing email 
while still being able to access it quickly and robustly.

Do you really need to store your files in a single 
meta-file? Do you need compression? How much overhead 
for the archive structure are you prepared to carry? Do 
you expect the archive to shrink when you delete a file 
from the middle?

I suspect you can pick any two of the following three:

1. single file
2. space used for deleted files is reclaimed
3. fast performance

Using a proper database will give you 2 and 3, but at 
the cost of a lot of overhead, and typically a 
relational database is not a single file.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python vs. Lisp -- please explain

2006-02-21 Thread Steven D'Aprano
Donn Cave wrote:

> Quoth Steven D'Aprano <[EMAIL PROTECTED]>:
> ...
> | Nobody denies that Python code running with no optimization tricks is
> | (currently) slower than compiled C code. That's a matter of objective
> | fact. Nobody denies that Python can be easily run in interactive mode.
> | Nobody denies that *at some level* Python code has to be interpreted.
> |
> | But ALL code is interpreted at some level or another. And it is equally
> | true that at another level Python code is compiled. Why should one take
> | precedence over the other?
> 
> I have no idea, what precedence?  

There seem to be two positions in this argument:

The "Python is interpreted and not compiled" camp, who 
appear to my eyes to dismiss Python's compilation stage 
as a meaningless technicality.

The "Python is both interpreted and compiled" camp, who 
believe that both steps are equally important, and to 
raise one over the other in importance is misleading.


 > All I'm saying is that Python matches
> what people think of as an interpreted language.

Most people in IT I know of still think of 
"interpreted" as meaning that every line of source code 
is parsed repeatedly every time the code is executed. 
Even when they intellectually know this isn't the case, 
old habits die hard -- they still think of 
"interpreted" as second class.

If you think that Python has to parse the line "print 
x" one hundred times in "for i in range(100): print x" 
then you are deeply, deeply mistaken.

That's why Sun doesn't describe Java as interpreted, 
but as byte-code compiled. They did that before they 
had JIT compilers to compile to machine code. 
Consequently nobody thinks of Java source having to be 
parsed, and parsed, and parsed, and parsed again.


 > You can deny it, but
> but it's going to look like you're playing games with words, and to no
> real end, since no one could possibly be deceived for very long.

Pot, meet kettle.

A simple question for you: does Python compile your 
source code before executing it? If you need a hint, 
perhaps you should reflect on what the "c" stands for 
in .pyc files.


 > If you
> give me a Python program, you have 3 choices:  cross your fingers and
> hope that I have the required Python interpreter version, slip in a
> 25Mb Python interpreter install and hope I won't notice, or come clean
> and tell me that your program needs an interpreter and I should check to
> see that I have it.

Hey Donn, here is a compiled program for the PowerPC, 
or an ARM processor, or one of IBM's Big Iron 
mainframes. Or even a Commodore 64. What do you think 
the chances are that you can execute it on your 
x86-compatible PC? It's compiled, it should just 
work!!! Right?

No of course not. If your CPU can't interpret the 
machine code correctly, the fact that the code is 
compiled makes NO difference at all.

In other words, I have three choices:

- cross my fingers and hope that you have the required 
interpreter (CPU);

- slip in an interpreter install (perhaps an emulator) 
and hope you won't notice;

- or come clean and tell you that my program needs an 
interpreter ("Hey Donn, do you have a Mac you can run 
this on?") and you should check to see that you have it.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to break a for loop?

2006-02-21 Thread bonono

Giovanni Bajo wrote:
> [EMAIL PROTECTED] wrote:
>
> >> I need to remove zeros from
> >> the begining of list, but I can't :-(.
> >
> > I believe the following is almost a direct translation of the above
> > sentence.
> >
> > import itertools as it
> > a=[0,0,0,1,0]
> > a[:]=it.dropwhile(lambda x: x is 0, a)
>
>
> Usa lambda x: x==0 or simply lambda x: x
>
> Using 'is' relies on the interpreter reusing existing instances of the
> immutable object '0', which is an implementation detail.

stand corrected. But I don't think "lambda x: x" is I want as 0 would
be false which would stop the dropwhile right away.

For this particular case one can even use operator.not_ directly, which
skip the lambda(has some peformance advantage).

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Giovanni Bajo
Suresh Jeevanandam wrote:

> In python all numbers are immutable.

True.

> This means there is one object (
> a region in the memory ) created every time we do an numeric
> operation.

False. Memory regions can be pooled in a way that "allocation" is a O(1)
operation.

> I hope there should have been some good reasons why it was
> designed this way.

Yes, so that you can obtain a pass-by-value semantic for numeric types. That
is, if you pass a number to a function, it can't modify it. You consistenly use
"a = foo(a)" to do a calculation on "a", instead of having half programs doing
"foo(a)" and the other half doing "a = foo(a)". Also, it's done so that you
don't need to special case integer literals: they are (fixed) names to the same
immutable instances and nobody can modify the value of 13 or any other literal.

> But why not have mutable numbers also in the language. A type which
> would behave as follows:
>
> a = MutableInt(12)
> b = a
>
> Now both a and b should refer to the same memory location. Any change
> in the object should get reflected in all the references.
>
> a.assign(13) # Same memory location should be updated with value 13, b
> is also 13 now.
>
> Now we can further optimize:
>
> a.incrementOne() # equivalent to a++ in C++
> a.decrementOne()
>
> and the augmented assignment operation also could be made optimized.

So you just created a numeric type which does not work with normal operators.
This is so unpythonic. I don't think anybody wants to write "incrementOne" when
"+ 1" exists and does the same it used to do in primary school. Even if you
overloaded all the operators, you still wouldn't be able to use '=' to do
assignment, which pretty much makes your class useless or very unpythonic to
use (= wrong).

> In any application most of the operation is numerical. So, i think, we
> should get a good speed advantage with the availability of mutable
> numbers. What do you think ?


You are doing premature optimization. It's so premature that it's even a brain
thing. You *presume* Python to be slow, you *presume* that this presumed
slowness comes from numbers being immutables. But you didn't *measure* it, you
didn't try it on a real-world case. You didn't even look at the Python source
code to check if your assumptions were true. Try to check your assumptions
before designing solutions to problems which do not exist. I suggest you start
writing real-world Python before doing so many assumptions.
-- 
Giovanni Bajo


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: In need of a virtual filesystem / archive

2006-02-21 Thread bonono

Steven D'Aprano wrote:
> I suspect you can pick any two of the following three:
>
> 1. single file
> 2. space used for deleted files is reclaimed
> 3. fast performance
>
> Using a proper database will give you 2 and 3, but at
> the cost of a lot of overhead, and typically a
> relational database is not a single file.
sqlite can give 1-3, it does have overhead but whether it worths it
depends on individual judgement based on features, usage pattern etc..
I think monotone use it.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: number ranges

2006-02-21 Thread Steven D'Aprano
Alex Martelli wrote:

> Colin J. Williams <[EMAIL PROTECTED]> wrote:
>...
> 
I am also open to such arguments but it will be tough to convince me
that "x to y" should mean something different from what it means in
Pascal, BASIC, and English.
>>>
>...
> 
>>1. Why not treat '2 to 5' or '(2 to 5)' as a semi-open interval?
> 
> 
> Reread the part I quoted above: at least some of the proponents of this
> syntax appear to be totally ignorant of 30 years of literature and
> practice of programming, "it will be tough to convince" them that closed
> intervals are a terrible mistake and semi-open ones the only way to go.

I intellectually understand that semi-open intervals 
are the only way to go. But reading the words, the part 
of my brain that speaks English cries out for a closed 
interval. Bad brain.

Given the overwhelming benefit of semi-closed 
intervals, I wish to amend my proposal to follow Alex's 
suggestions, namely:

for i in (1 to 10 by 3):
 print i

should print 1 4 7.

That would make (a to b by c) equivalent to 
range(a,b,c) except that it returns an iterator rather 
than a list. Hmmm... putting it that way, the new 
syntax actually costs 1 keystroke, or saves 1 if you 
put spaces after the commas in the range expression. 
Does it add enough clarity and ease of understanding to 
justify two new keywords?


> Introducing a new syntax, with semantics that "don't convince" some of
> its prominent proponents, would be self-destructive (I shudder just to
> think of the amount of time and energy we'd all be spending dealing with
> whines about it); Python is clearly much better off if such people run
> away to Ruby, with its (expletive deleted) a..b AND a...b syntaxes just
> to ensure maximum confusion;-).

Ruby uses both .. and ...? Now I'm frightened.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


DCOracle2: OCI error

2006-02-21 Thread Chris
Hello

I'm using DCOracle2 with Python 2.1 and Oracle 8.1.7 on Solaris and
while it's working most of the time, I have random ORA-01084 invalid
argument in OCI call errors that occur on INSERT.
Out of 18000 queries all using the same DB connection I have like 15
errors of this type. It's always the same SQL queries (2 different
INSERT and some other SELECT), only the parameters change and they're
always valid. When the error occur I can simply retry the statement and
it works, so nothing's wrong with the query and its parameters.
If I launch all the queries again the errors will occur again but never
at the same place, it's really random except for the fact that it only
occurs on INSERT.
There are commits after each INSERT. When I have an ORA-1084 I rollback
and go on with the next queries and they work.

So I'm thinking it may be a bug in DCOracle2.
I've activated the DCOracle traceback and I've noticed that the error
always occurs on the binding of NULL (None) values.
My guess at the moment is that the bindObject function in dco2.c should
return 0x for bind->valuep when the value is None and indeed
it's what it does most of the time but when the error occurs I can see
in the traceback that 0x is returned and this make the
following call to OCIBindByPos fail with the above error.
Why would 0x be returned? Any help on this is appreciated.
I'm especially interested in the inner working of dco2.c OCI calls so I
can investigate more because for now I'm stuck.
I'll try to add disconnect / reconnect every now and then to see if it
helps.

Regards
Chris

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: fast text processing

2006-02-21 Thread Steve Holden
Alexis Gallagher wrote:
> (I tried to post this yesterday but I think my ISP ate it. Apologies if 
> this is a double-post.)
> 
> Is it possible to do very fast string processing in python? My 
> bioinformatics application needs to scan very large ASCII files (80GB+), 
> compare adjacent lines, and conditionally do some further processing. I 
> believe the disk i/o is the main bottleneck so for now that's what I'm 
> optimizing. What I have now is roughly as follows (on python 2.3.5).
> 
> filehandle = open("data",'r',buffering=1000)

This buffer size seems, shall we say, unadventurous? It's likely to slow 
things down considerably, since the filesystem is probably going to 
naturally wnt to use a rather larger value. I'd suggest a 64k minumum.
> 
> lastLine = filehandle.readline()
> 
I'd suggest

lastTokens = filehandle.readline().strip().split(delimiter)

here. You have no need of the line other than to split it into tokens.

> for currentLine in filehandle.readlines():
> 
Note that this is going to read the whole file in to (virtual) memory 
before entering the loop. I somehow suspect you'd rather avoid this if 
you could. I further suspect your testing has been with smaller files 
than 80GB ;-). You might want to consider

for currentLine in filehandle:

as an alternative. This uses the file's generator properties to produce 
the next line each time round the loop.

>  lastTokens = lastLine.strip().split(delimiter)

The line above goes away if you adopt the loop initialization suggestion 
above. Otherwise you are repeating the splitting of each line twice, 
once as the current line then again as the last line.

>  currentTokens = currentLine.strip().split(delimiter)
> 
>  lastGeno = extract(lastTokens[0])
>  currentGeno = extract(currentTokens[0])
> 
If the extract() operation is stateless (in other words if it always 
produces the same output for a given input) then again you are 
unecessarily repeating yourself here by running extract() on the same 
data as the current first token and the last first token (if you see 
what I mean).

I might also observe that you seem to expect only two tokens per line. 
If this is invariable the case then you might want to consider writing 
an unpacking assignment instead, such as

cToken0, cToken1, newline = currentLine.strip().split(delimiter)

to save the indexing. Not a big deal, thugh, and it *will* break if you 
have more than one delimiter in a line, as the interpreter won;t then 
know what to do with the third and subsequent elements.

>  # prepare for next iteration
>  lastLine = currentLine
> 
Of course now you are going to try and strip the delimiter off the line 
and split it again when you loop around again. You should now just be 
able to say

 lastTokens = currentTokens

instead.

>  if lastGeno == currentGeno:
> table.markEquivalent(int(lastTokens[1]),int(currentTokens[1]))
> 
> So on every iteration I'm processing mutable strings -- this seems 
> wrong. What's the best way to speed this up? Can I switch to some fast 
> byte-oriented immutable string library? Are there optimizing compilers? 
> Are there better ways to prep the file handle?
> 
I'm sorry but I am not sure where the mutable strings come in. Python 
strings are immutable anyway. Well-known for it. It might be a slight 
problem that you are creating a second terminator-less copy of each 
line, but that's probably inevitable.

Of course you leave us in the dark about the nature of 
table.markEquivalent as well. Depending on the efficiency of the 
extract() operation you might want to consider short-circuiting the loop 
if the two tokens have already been marked as equivalent. That might be 
a big win or not depending on relative efficiency.

> Perhaps this is a job for C, but I am of that soft generation which 
> fears memory management. I'd need to learn how to do buffered reading in 
> C, how to wrap the C in python, and how to let the C call back into 
> python to call markEquivalent(). It sounds painful. I _have_ done some 
> benchmark comparisons of only the underlying line-based file reading 
> against a Common Lisp version, but I doubt I'm using the optimal 
> construct in either language so I hesitate to trust my results, and 
> anyway the interlanguage bridge will be even more obscure in that case.
> 
Probably the biggest gain will be in simply not reading the whole file 
into memory by calling its .readlines() method.

Summarising. consider something more like:

filehandle = open("data",'r',buffering=64*1024)
# You could also try just leaving the buffering spec out

lastTokens = filehandle.readline().strip().split(delim)
lastGeno = extract(lastTokens[0])

for currentLine in filehandle:

 currentTokens = currentLine.strip().split(delim)
 currentGeno = extract(currentTokens[0])
 if lastGeno == currentGeno:
 table.markEquivalent(int(lastTokens[1]), int(currentTokens[1]))

 lastGeno = currentGeno
 lastTokens =

RE: Print a PDF transparently

2006-02-21 Thread Tim Golden
[Daniel Crespo]
| 
| > Have you seen this?
| > http://tgolden.sc.sabren.com/python/win32_how_do_i/print.html
| > In particular, the section on using win32print directly.
| 
| Yes, I have. The problems is that an external program is launched for
| handling the file and print it. 

[sorry, bit long-winded, but trying to help]

I think the poster was referring to the situation you 
described where the problem reduced to sending a .ps
file directly to a postscript-capable printer.

To put things in as much of a nutshell as I can, I
think the following are true:

1) You have a document in some format which
you wish to print either via Postscript or PDF.

2) You do have postscript-enabled printers available.

3) As far as possible, you don't really want to install 
external apps across 70 offices.

What's not entirely clear is the extent to which you
can automate the generation of a postscript doc. Let's
assume that this can be done centrally. If that's the
case then you can take the file and use the technique
described in the link above, but specifically in this
section:

http://tgolden.sc.sabren.com/python/win32_how_do_i/print.html#win32print

which hasn't to do with running Acrobat or Ghostscript.

Basically, adapting the example, you'd do this:


import win32print
printer_name = win32print.GetDefaultPrinter ()
#
# raw_data could equally be raw PCL/PS read from 
#  some print-to-file operation
#
name_of_postscript_file = "document.ps"
raw_data = open (name_of_postscript_file, "rb").read ()

hPrinter = win32print.OpenPrinter (printer_name)
try:
  hJob = win32print.StartDocPrinter (hPrinter, 1, ("test of raw data",
None, "RAW"))
  try:
win32print.WritePrinter (hPrinter, raw_data)
  finally:
win32print.EndDocPrinter (hPrinter)
finally:
  win32print.ClosePrinter (hPrinter)



If you're not really sure how to get a postscript
doc, I usually just install an AppleLaserwriter of some
description as a driver, printing to file, and just
generate things that way. Exactly how you do that
depends on what the original document is.

Now, if the original is in PDF (ie PDF isn't simply
the staging post I've assumed it was), you'll need
to use *something* like ghostscript to print or
to generate postscript. Automating Acrobat Reader 
is a nightmare, because there's always something 
you don't want to happen -- it stays open, or you 
can't specify the printer to use, or something.
There are other PDF-display tools out there
(someone mentioned one recently on c.l.py and
there's xpdf as well, which might help).

TJG


This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk

-- 
http://mail.python.org/mailman/listinfo/python-list


Little tool - but very big size... :-(

2006-02-21 Thread Durumdara
Hi !

I have a problem.
I have a little tool that can get data about filesystems and wrote it in
python.

The main user asked me a GUI for this software.

This user is needed a portable program, so I create this kind of the
software with Py2Exe.

But it have very big size: 11 MB... :-(

The dist directory:
2006.02.21.  10:09  .
2006.02.21.  10:09  ..
2005.09.28.  12:4177 824 bz2.pyd
2006.02.21.  10:09 0 dirlist.txt
2006.02.20.  12:51   611 384 library.zip
2006.02.15.  16:2223 558 main.ico
2004.12.16.  17:22   348 160 MSVCR71.dll
2005.09.28.  12:41 1 867 776 python24.dll
2006.01.11.  12:19   102 400 pywintypes24.dll
2005.09.28.  12:41   405 504 unicodedata.pyd
2005.09.28.  12:41 4 608 w9xpopen.exe
2006.01.11.  12:1973 728 win32api.pyd
2006.01.11.  12:2081 920 win32file.pyd
2006.01.11.  12:26   106 496 win32security.pyd
2006.01.10.  19:09 4 943 872 wxmsw26uh_vc.dll
2006.02.20.  12:5140 960 wxPyHDDirList.exe
2005.09.28.  12:4169 632 zlib.pyd
2006.01.10.  19:13   626 688 _controls_.pyd
2006.01.10.  19:12   696 320 _core_.pyd
2006.01.10.  19:13   364 544 _gdi_.pyd
2006.01.10.  19:13   491 520 _misc_.pyd
2006.01.10.  19:13   548 864 _windows_.pyd
  20 file11 485 758 byte

I need to have more compressed result. Can I compress dll-s, pyd-s with
Py2Exe ?
Can I decrease the total size with something ?

If not, how to I create an self-unpackager and self-starter program that
use an temporary directory in the disk ? With WinRar ?

Thanx for help:
dd



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: In need of a virtual filesystem / archive

2006-02-21 Thread Rene Pijlman
Enigma Curry:
>I need to store a large number of files in an archive. From Python, I
>need to be able to create an archive, put files into it, modify files
>that are already in it, and delete files already in it.

Use the file system. That's what it's for.

-- 
René Pijlman
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python vs. Lisp -- please explain

2006-02-21 Thread Ben Sizer
Steven D'Aprano wrote:
> The "Python is both interpreted and compiled" camp, who
> believe that both steps are equally important, and to
> raise one over the other in importance is misleading.
> That's why Sun doesn't describe Java as interpreted,
> but as byte-code compiled. They did that before they
> had JIT compilers to compile to machine code.
> Consequently nobody thinks of Java source having to be
> parsed, and parsed, and parsed, and parsed again.

They also described it that way to help marketing, and I don't think
that should be overlooked. They would have known full well that calling
their language "interpreted" would have affected public perceptions.

It's interesting to see how culture affects things. You talked of 'IT
people' in your post and hold up Java of an example of how byte-code
doesn't mean slow, the implication being that Python uses the same
mechanisms as Java and therefore is good enough. In the general IT
world, Java is quite popular (to make a bit of an understatement) and
it would often be used as some sort of benchmark.

On the other hand, the backgrounds I have familiarity with are computer
game development and embedded development. In these areas, we would
point to Java as evidence that 'interpreted' bytecode is too slow and
that anything using a similar technology is likely to be a problem.

I'm not saying you're wrong, just highlighting that comparisons
themselves always sit in some wider context which can make the
comparison unimportant.

I think it's also important to note that 'interpreted' doesn't
necessarily mean 'parsed repeatedly'. Many older machines which came
with BASIC installed would store their statements in a tokenised form -
arguably bytecode with a large instruction set, if you look at it a
certain way. This isn't necessarily so far from what Python does, yet
few people would argue that those old forms of BASIC weren't
interpreted.

-- 
Ben Sizer

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope 3?? How do you start it on Windows?

2006-02-21 Thread Max M
kbperry wrote:
> Call me crazy and stupid, but how do you start zope?  I downloaded the
> thing, installed it, and then nothing.  I can find a stupid README.txt
> that is useful.
> 
> Can someone help?Please!


Install Python 2.4

Install Zope 3 in that python by using setup.py

Make a Zope instance with Python24\Scripts\mkzopeinstance.bat

Start that instance by running

/bin/runzope.bat


-- 

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: getting started, .py file

2006-02-21 Thread Steven D'Aprano
Ingrid wrote:

> Thanks everyone. That's exactly what I was looking for, but I still
> can't seem to make it work. I've got the interpreter starting in
> "C:\Program Files\Python2.4", and my code is in "C:\Documents and
> Settings\Ingrid\My Documents". So, I did:
>   import os
>   os.chdir("C:\\Documents and Settings\\Ingrid\\My Documents")
>   from mycode import *

I see that Ingrid found the solution to this problem. 
But I wonder, should Python support importing absolute 
pathnames?

I've trying googling, but either my google skills are 
lacking or nobody has ever thought of doing import 
"C:\directory\module" before.


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread bonono
11MB is seldom a concern for today's machine.

Durumdara wrote:
> Hi !
>
> I have a problem.
> I have a little tool that can get data about filesystems and wrote it in
> python.
>
> The main user asked me a GUI for this software.
>
> This user is needed a portable program, so I create this kind of the
> software with Py2Exe.
>
> But it have very big size: 11 MB... :-(
>
> The dist directory:
> 2006.02.21.  10:09  .
> 2006.02.21.  10:09  ..
> 2005.09.28.  12:4177 824 bz2.pyd
> 2006.02.21.  10:09 0 dirlist.txt
> 2006.02.20.  12:51   611 384 library.zip
> 2006.02.15.  16:2223 558 main.ico
> 2004.12.16.  17:22   348 160 MSVCR71.dll
> 2005.09.28.  12:41 1 867 776 python24.dll
> 2006.01.11.  12:19   102 400 pywintypes24.dll
> 2005.09.28.  12:41   405 504 unicodedata.pyd
> 2005.09.28.  12:41 4 608 w9xpopen.exe
> 2006.01.11.  12:1973 728 win32api.pyd
> 2006.01.11.  12:2081 920 win32file.pyd
> 2006.01.11.  12:26   106 496 win32security.pyd
> 2006.01.10.  19:09 4 943 872 wxmsw26uh_vc.dll
> 2006.02.20.  12:5140 960 wxPyHDDirList.exe
> 2005.09.28.  12:4169 632 zlib.pyd
> 2006.01.10.  19:13   626 688 _controls_.pyd
> 2006.01.10.  19:12   696 320 _core_.pyd
> 2006.01.10.  19:13   364 544 _gdi_.pyd
> 2006.01.10.  19:13   491 520 _misc_.pyd
> 2006.01.10.  19:13   548 864 _windows_.pyd
>   20 file11 485 758 byte
>
> I need to have more compressed result. Can I compress dll-s, pyd-s with
> Py2Exe ?
> Can I decrease the total size with something ?
>
> If not, how to I create an self-unpackager and self-starter program that
> use an temporary directory in the disk ? With WinRar ?
> 
> Thanx for help:
> dd

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Martin Franklin

Durumdara wrote:
> Hi !
> 
> I have a problem.
> I have a little tool that can get data about filesystems and wrote it in
> python.
> 
> The main user asked me a GUI for this software.
> 
> This user is needed a portable program, so I create this kind of the
> software with Py2Exe.
> 
> But it have very big size: 11 MB... :-(
> 
> I need to have more compressed result. Can I compress dll-s, pyd-s with
> Py2Exe ?
> Can I decrease the total size with something ?



you could try UPX http://upx.sourceforge.net/ I think it will handle
the .pyd and dll files (it did the last time I tested it)


> 
> If not, how to I create an self-unpackager and self-starter program that
> use an temporary directory in the disk ? With WinRar ?

I may be wrong (havn't used py2exe in a while) but I think it can now
(again) create a single exe file?  Otherwise something like Inno Setup
http://www.jrsoftware.org/isinfo.php or a simple self extracting zip
file



> 
> Thanx for help:
> dd
> 
> 
> 

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: fast text processing

2006-02-21 Thread Ben Sizer
Maybe this code will be faster? (If it even does the same thing:
largely untested)


filehandle = open("data",'r',buffering=1000)
fileIter = iter(filehandle)

lastLine = fileIter.next()
lastTokens = lastLine.strip().split(delimiter)
lastGeno = extract(lastTokens[0])

for currentLine in fileIter:
currentTokens = currentLine.strip().split(delimiter)
currentGeno = extract(currentTokens[0])

if lastGeno == currentGeno:
table.markEquivalent(int(lastTokens[1]),int(currentTokens[1]))

# prepare for next iteration
lastLine = currentLine
lastTokens = currentTokens
lastGeno = currentGeno


I'd be tempted to try a bigger file buffer too, personally.

--
Ben Sizer

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Durumdara
Hi !

Yes, it is. But that tool is designed for USB PenDrive usage.
The assessor is collect all tools it needed to see a machine(s) in the 
checked corporation.
He/she needs little programs, because he need to store the results of 
the checkings too, not the tools only.

So I need to minimalize the code size.

Thanx for help:
dd

[EMAIL PROTECTED] wrote:

>11MB is seldom a concern for today's machine.
>
>Durumdara wrote:
>  
>
>>Hi !
>>
>>I have a problem.
>>I have a little tool that can get data about filesystems and wrote it in
>>python.
>>
>>The main user asked me a GUI for this software.
>>
>>This user is needed a portable program, so I create this kind of the
>>software with Py2Exe.
>>
>>But it have very big size: 11 MB... :-(
>>
>>The dist directory:
>>2006.02.21.  10:09  .
>>2006.02.21.  10:09  ..
>>2005.09.28.  12:4177 824 bz2.pyd
>>2006.02.21.  10:09 0 dirlist.txt
>>2006.02.20.  12:51   611 384 library.zip
>>2006.02.15.  16:2223 558 main.ico
>>2004.12.16.  17:22   348 160 MSVCR71.dll
>>2005.09.28.  12:41 1 867 776 python24.dll
>>2006.01.11.  12:19   102 400 pywintypes24.dll
>>2005.09.28.  12:41   405 504 unicodedata.pyd
>>2005.09.28.  12:41 4 608 w9xpopen.exe
>>2006.01.11.  12:1973 728 win32api.pyd
>>2006.01.11.  12:2081 920 win32file.pyd
>>2006.01.11.  12:26   106 496 win32security.pyd
>>2006.01.10.  19:09 4 943 872 wxmsw26uh_vc.dll
>>2006.02.20.  12:5140 960 wxPyHDDirList.exe
>>2005.09.28.  12:4169 632 zlib.pyd
>>2006.01.10.  19:13   626 688 _controls_.pyd
>>2006.01.10.  19:12   696 320 _core_.pyd
>>2006.01.10.  19:13   364 544 _gdi_.pyd
>>2006.01.10.  19:13   491 520 _misc_.pyd
>>2006.01.10.  19:13   548 864 _windows_.pyd
>>  20 file11 485 758 byte
>>
>>I need to have more compressed result. Can I compress dll-s, pyd-s with
>>Py2Exe ?
>>Can I decrease the total size with something ?
>>
>>If not, how to I create an self-unpackager and self-starter program that
>>use an temporary directory in the disk ? With WinRar ?
>>
>>Thanx for help:
>>dd
>>
>>
>
>  
>

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Ravi Teja
Dependency tracking is usually not perfect in exe generators. I don't
recall how good py2exe is at this. If you understand which of those
files will not be used by your program, you can safely delete them. A
bit of trial and error here if you are not sure. Unless you are using
floppies or dialup to transfer this tool, you should simply not worry
about the file size.

As for compression, just use an installer builder like Inno Setup
http://www.jrsoftware.org/isinfo.php
http://www.istool.org/default.aspx/

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to break a for loop?

2006-02-21 Thread Gregory Petrosyan
Thanks to everybody for help. Python community is very friendly and
helpfull indeed!

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Writing my own typing monitor program for RSI sufferers...

2006-02-21 Thread Diez B. Roggisch
[EMAIL PROTECTED] schrieb:
> So, so close  The problem with this implementation is that it
> doesn't monitor usb keyboards under linux at all as far as I can
> tellsince no keyboard entry will show up in /proc/interrupts with a
> usb keyboard.  I absolutely need the keyboard monitoring as well.
> 
> Otherwise, your project would be the perfect starting point for me!
> 
> Anyone else have an idea as to what I can use with Python to know when
> the keyboard is being used even when I don't have focus?

If you can wait a few month, I'll be doing that in python based on this:

http://www.frogmouth.net/hid-doco/linux-hid.html

I'll be doing it for a art-project that is scheduled some months away. But I 
think the above information even should 
help you get started on your own.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: converting sqlite return values

2006-02-21 Thread Gerhard Häring
bolly wrote:
> Hi,
> I've been putting Python data into a sqlite3 database as tuples but
> when I retrieve them they come back as unicode data e.g
> 'u(1,2,3,4)'. 

Looks like you're using pysqlite 2.x.

> How can I change it back to a tuple so I can use it as a
> Python native datatype?

You cannot store tuples using pysqlite directly:

 >>> from pysqlite2 import dbapi2 as sqlite
 >>> con = sqlite.connect(":memory:")
 >>> cur = con.cursor()
 >>> cur.execute("create table test(foo)")

 >>> t = (3, 4, 5)
 >>> cur.execute("insert into test(foo) values (?)", (t,))
Traceback (most recent call last):
   File "", line 1, in ?
pysqlite2.dbapi2.InterfaceError: Error binding parameter 0 - probably 
unsupported type.
 >>>

That's because only a limited set of types that have a sensible mapping 
to SQLite's supported data types is supported.

So probably you did something like:

 >>> cur.execute("insert into test(foo) values (?)", (str(t),))

 >>> cur.execute("select foo from test")

 >>> res = cur.fetchone()[0]
 >>> res
u'(3, 4, 5)'
 >>>

Aha. You stored a string and got back a Unicode string. That's all ok 
because SQLite strings are by definition all UTF-8 encoded that's why 
the pysqlite developer decided that what you get back in Python are 
Unicode strings.

Now there are different possibilites to attack this problem.

a) Use SQLite as a relational database and don't throw arbitrary objects 
at it
b) Write a custom converter and adapter for your tuple type. See 
http://initd.org/pub/software/pysqlite/doc/usage-guide.html#sqlite-and-python-types

This way it will all work transparently from you once you've done the 
preparations.

c) Store and retrieve the whole thing as a BLOB and convert manually:

 >>> cur.execute("delete from test")

 >>> cur.execute("insert into test(foo) values (?)", (buffer(str(t)),))

 >>> cur.execute("select foo from test")

 >>> res = cur.fetchone()[0]
 >>> res

 >>> eval(str(res))
(3, 4, 5)

That's the simple apprach, but it sucks because eval() is sloppy 
programming IMO.

So I'd rather marshal and demarshal the tuple:

 >>> import marshal
 >>> cur.execute("delete from test")

 >>> cur.execute("insert into test(foo) values (?)", 
(buffer(marshal.dumps(t)),))

 >>> cur.execute("select foo from test")

 >>> res = cur.fetchone()[0]
 >>> marshal.loads(res)
(3, 4, 5)

> I have looked in the docs and seen there is a decode/encode method but
> how do I do this?

You don't. This was for only there in pysqlite 1.x and pysqlite 2.x. In 
pysqlite 2.x, you use the Python builtin buffer() callable to convert 
strings to buffers to mark them as BLOB values for pysqlite and you 
willg et back buffer objects from pysqlite for BLOB values, too.

HTH,

-- Gerhard
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread bonono
I doubt you can do much then as you mentioned you need GUI which is why
there is the GUI related dll in the list(takes a large chunk of it),
then the necessary python runtime.

However, even for USB pen, I don't think 11M is that much a big deal.
We have digicam that can produce file size like that so finding a large
pen drive is not much of an issue. Worse come to worse, convince him to
use a 2.5" USB drive container. It is a bit bigger than USB pen drive
but can give you 20G+ without external power, just as convenient as a
pen drive.

Durumdara wrote:
> Hi !
>
> Yes, it is. But that tool is designed for USB PenDrive usage.
> The assessor is collect all tools it needed to see a machine(s) in the
> checked corporation.
> He/she needs little programs, because he need to store the results of
> the checkings too, not the tools only.
>
> So I need to minimalize the code size.
>
> Thanx for help:
> dd
>
> [EMAIL PROTECTED] wrote:
>
> >11MB is seldom a concern for today's machine.
> >
> >Durumdara wrote:
> >
> >
> >>Hi !
> >>
> >>I have a problem.
> >>I have a little tool that can get data about filesystems and wrote it in
> >>python.
> >>
> >>The main user asked me a GUI for this software.
> >>
> >>This user is needed a portable program, so I create this kind of the
> >>software with Py2Exe.
> >>
> >>But it have very big size: 11 MB... :-(
> >>
> >>The dist directory:
> >>2006.02.21.  10:09  .
> >>2006.02.21.  10:09  ..
> >>2005.09.28.  12:4177 824 bz2.pyd
> >>2006.02.21.  10:09 0 dirlist.txt
> >>2006.02.20.  12:51   611 384 library.zip
> >>2006.02.15.  16:2223 558 main.ico
> >>2004.12.16.  17:22   348 160 MSVCR71.dll
> >>2005.09.28.  12:41 1 867 776 python24.dll
> >>2006.01.11.  12:19   102 400 pywintypes24.dll
> >>2005.09.28.  12:41   405 504 unicodedata.pyd
> >>2005.09.28.  12:41 4 608 w9xpopen.exe
> >>2006.01.11.  12:1973 728 win32api.pyd
> >>2006.01.11.  12:2081 920 win32file.pyd
> >>2006.01.11.  12:26   106 496 win32security.pyd
> >>2006.01.10.  19:09 4 943 872 wxmsw26uh_vc.dll
> >>2006.02.20.  12:5140 960 wxPyHDDirList.exe
> >>2005.09.28.  12:4169 632 zlib.pyd
> >>2006.01.10.  19:13   626 688 _controls_.pyd
> >>2006.01.10.  19:12   696 320 _core_.pyd
> >>2006.01.10.  19:13   364 544 _gdi_.pyd
> >>2006.01.10.  19:13   491 520 _misc_.pyd
> >>2006.01.10.  19:13   548 864 _windows_.pyd
> >>  20 file11 485 758 byte
> >>
> >>I need to have more compressed result. Can I compress dll-s, pyd-s with
> >>Py2Exe ?
> >>Can I decrease the total size with something ?
> >>
> >>If not, how to I create an self-unpackager and self-starter program that
> >>use an temporary directory in the disk ? With WinRar ?
> >>
> >>Thanx for help:
> >>dd
> >>
> >>
> >
> >  
> >

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Giovanni Bajo
Durumdara wrote:

> I need to have more compressed result. Can I compress dll-s, pyd-s
> with Py2Exe ?
> Can I decrease the total size with something ?
>
> If not, how to I create an self-unpackager and self-starter program
> that use an temporary directory in the disk ? With WinRar ?


You can use PyInstaller (http://pyinstaller.hpcf.upr.edu). PyInstaller easily
builds a single executable with everything inside it. It also optionally
compresses everything with UPX (http://upx.sourceforge.net/) so to reduce the
size at the maximum.

There are also other choices that can be made. For instance, wxWidgets is
*HUGE*.
-- 
Giovanni Bajo


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Mikael Olofsson
Durumdara wrote:
> But it have very big size: 11 MB... :-(
> 
> The dist directory:
> [snip relatively small files]
> 2005.09.28.  12:41 1 867 776 python24.dll
> [snip relatively small files]
> 2006.01.10.  19:09 4 943 872 wxmsw26uh_vc.dll
> [snip relatively small files]
>  20 file11 485 758 byte
> 
> I need to have more compressed result. Can I compress dll-s, pyd-s with
> Py2Exe ?
> Can I decrease the total size with something ?

I've snipped out the relatively small files above. Yes, true, some of 
them consume about 0.5MB each.

It seems to me that your choice of GUI framework is a major cost here. I 
have never used wxpython. Instead my GUIs are based on tkinter. What I 
typically end up with is roughly 7MB. My last example ended up in 7.5MB. 
Zipping the whole thing reduces that to 2.6MB. Is it completly out of 
the question to have a compressed version of the tool on your memory 
stick, and to decompress it on the examined computer before actually 
running the tool?

/MiO
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Durumdara
Dear Martin !

Thanx for it:

setup(
options = {"py2exe": {"bundle_files": 1,   # < this 
help me 
  "compressed": 1,
  "optimize": 2}},
# The lib directory contains everything except the executables and 
the python dll.
# Can include a subdirectory name.
windows = [wxPyHDDirList],
)

Can I increase level of the optimization ?
Can I increase level of the zip packing ?

Please help me: dd


Martin Franklin wrote:

>Durumdara wrote:
>  
>
>>Hi !
>>
>>I have a problem.
>>I have a little tool that can get data about filesystems and wrote it in
>>python.
>>
>>The main user asked me a GUI for this software.
>>
>>This user is needed a portable program, so I create this kind of the
>>software with Py2Exe.
>>
>>But it have very big size: 11 MB... :-(
>>
>>I need to have more compressed result. Can I compress dll-s, pyd-s with
>>Py2Exe ?
>>Can I decrease the total size with something ?
>>
>>
>
>
>
>you could try UPX http://upx.sourceforge.net/ I think it will handle
>the .pyd and dll files (it did the last time I tested it)
>
>
>  
>
>>If not, how to I create an self-unpackager and self-starter program that
>>use an temporary directory in the disk ? With WinRar ?
>>
>>
>
>I may be wrong (havn't used py2exe in a while) but I think it can now
>(again) create a single exe file?  Otherwise something like Inno Setup
>http://www.jrsoftware.org/isinfo.php or a simple self extracting zip
>file
>
>
>
>  
>
>>Thanx for help:
>>dd
>>
>>
>>
>>
>>
>
>  
>

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Gerhard Häring
Durumdara wrote:
> Hi !
> 
> I have a problem.
> I have a little tool that can get data about filesystems and wrote it in
> python.
> 
> The main user asked me a GUI for this software.
> 
> This user is needed a portable program, so I create this kind of the
> software with Py2Exe.
> 
> But it have very big size: 11 MB... :-( [...]

You can create an installer using NSIS and choose LZMA compression. 
That's the slickest way I know to distribute your application.

If that's too much trouble, you can also distribute your application as 
a self-extracting 7ZIP-Archive using LZMA compression 
(http://www.7-zip.org/).

Quick size test here with a small wxPython helper app of mine:

py2exe dist directory size: 12.469.572 Bytes

7ZIP self-extracting archive (LZMA, default settings): 2,80 MB 
(2.942.543 Bytes)

A NSIS installer will not add much overhead (a few hundred KB).

HTH,

-- Gerhard
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Gerhard Häring
Durumdara wrote:
> Hi !
> 
> Yes, it is. But that tool is designed for USB PenDrive usage.
> The assessor is collect all tools it needed to see a machine(s) in the 
> checked corporation.
> He/she needs little programs, because he need to store the results of 
> the checkings too, not the tools only. [...]

Additional Python or wxPython tools packed using py2exe will only need a 
few kilobytes if you all copy them to the same directory on the pen 
drive. All these tools can share the same .pyd and .dll files.

-- Gerhard
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Durumdara
Hi !

>I've snipped out the relatively small files above. Yes, true, some of 
>them consume about 0.5MB each.
>
>It seems to me that your choice of GUI framework is a major cost here. I 
>have never used wxpython. Instead my GUIs are based on tkinter. What I 
>typically end up with is roughly 7MB. My last example ended up in 7.5MB. 
>Zipping the whole thing reduces that to 2.6MB. Is it completly out of 
>the question to have a compressed version of the tool on your memory 
>stick, and to decompress it on the examined computer before actually 
>running the tool?
>
>/MiO
>  
>
Yes, the wxPython use the big files, and win32api do it too...
I need them, but wxPython is changeable to tkinter, because it is use 
only one special thing: a wx.GenericDirCtrl (the user can choose file or 
directory with this control in same way).

The last (compressed) version is 5 MB. That is better !

Thanx: dd



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: fast text processing

2006-02-21 Thread Alexis Gallagher
Steve,

First, many thanks!

Steve Holden wrote:
> Alexis Gallagher wrote:
>>
>> filehandle = open("data",'r',buffering=1000)
> 
> This buffer size seems, shall we say, unadventurous? It's likely to slow 
> things down considerably, since the filesystem is probably going to 
> naturally wnt to use a rather larger value. I'd suggest a 64k minumum.

Good to know. I should have dug into the docs deeper. Somehow I thought 
it listed lines not bytes.

>> for currentLine in filehandle.readlines():
>>
> Note that this is going to read the whole file in to (virtual) memory 
> before entering the loop. I somehow suspect you'd rather avoid this if 
> you could. I further suspect your testing has been with smaller files 
> than 80GB ;-). You might want to consider
> 

Oops! Thanks again. I thought that readlines() was the generator form, 
based on the docstring comments about the deprecation of xreadlines().

>> So on every iteration I'm processing mutable strings -- this seems 
>> wrong. What's the best way to speed this up? Can I switch to some fast 
>> byte-oriented immutable string library? Are there optimizing 
>> compilers? Are there better ways to prep the file handle?
>>
> I'm sorry but I am not sure where the mutable strings come in. Python 
> strings are immutable anyway. Well-known for it.

I misspoke. I think was mixing this up with the issue of object-creation 
overhead for all of the string handling in general. Is this a bottleneck 
to string processing in python, or is this a hangover from my Java days? 
I would have thought that dumping the standard string processing 
libraries in favor of byte manipulation would have been one of the 
biggest wins.

> Of course you leave us in the dark about the nature of 
> table.markEquivalent as well.

markEquivalent() implements union-join (aka, uptrees) to generate 
equivalence classes. Optimising that was going to be my next task

I feel a bit silly for missing the double-processing of everything. 
Thanks for pointing that out. And I will check out the biopython package.

I'm still curious if optimizing compilers are worth examining. For 
instance, I saw Pyrex and Pysco mentioned on earlier threads. I'm 
guessing that both this tokenizing and the uptree implementations sound 
like good candidates for one of those tools, once I shake out these 
algorithmic problems.


alexis
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Little tool - but very big size... :-(

2006-02-21 Thread Martin Franklin
Durumdara wrote:
> Dear Martin !
> 
> Thanx for it:
> 
> setup(
> options = {"py2exe": {"bundle_files": 1,   # < this 
> help me 
>   "compressed": 1,
>   "optimize": 2}},
> # The lib directory contains everything except the executables and 
> the python dll.
> # Can include a subdirectory name.
> windows = [wxPyHDDirList],
> )
> 
> Can I increase level of the optimization ?
> Can I increase level of the zip packing ?

Sorry, I have not used py2exe for some time, you would be better to ask 
this question on the py2exe list:

https://lists.sourceforge.net/lists/listinfo/py2exe-users

or read their documentation

http://www.py2exe.org/ or Wiki 
http://starship.python.net/crew/theller/moin.cgi/Py2Exe


in particular...

http://starship.python.net/crew/theller/moin.cgi/BetterCompression


HTH


> 
> Please help me: dd
> 
> 
> Martin Franklin wrote:
> 
>> Durumdara wrote:
>>  
>>
>>> Hi !
>>>
>>> I have a problem.
>>> I have a little tool that can get data about filesystems and wrote it in
>>> python.
>>>
>>> The main user asked me a GUI for this software.
>>>
>>> This user is needed a portable program, so I create this kind of the
>>> software with Py2Exe.
>>>
>>> But it have very big size: 11 MB... :-(
>>>
>>> I need to have more compressed result. Can I compress dll-s, pyd-s with
>>> Py2Exe ?
>>> Can I decrease the total size with something ?
>>>
>>>
>>
>>
>> you could try UPX http://upx.sourceforge.net/ I think it will handle
>> the .pyd and dll files (it did the last time I tested it)
>>
>>
>>  
>>
>>> If not, how to I create an self-unpackager and self-starter program that
>>> use an temporary directory in the disk ? With WinRar ?
>>>
>>>
>> I may be wrong (havn't used py2exe in a while) but I think it can now
>> (again) create a single exe file?  Otherwise something like Inno Setup
>> http://www.jrsoftware.org/isinfo.php or a simple self extracting zip
>> file
>>
>>
>>
>>  
>>
>>> Thanx for help:
>>> dd
>>>
>>>
>>>
>>>
>>>
>>  
>>
> 

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python vs. Lisp -- please explain

2006-02-21 Thread Kay Schluehr

Alexander Schmolck wrote:
> "Kay Schluehr" <[EMAIL PROTECTED]> writes:
>
> > Alexanders hypothesis is completely absurd.
>
> You're currently not in the best position to make this claim, since you
> evidently misunderstood what I wrote (I certainly did not mean to suggest that
> Guido *deliberately* chose to make python slow; quite the opposite in fact).

Like everyone else. It's sometimes hard extract the intended meaning in
particular if it's opposed to the published one. I apologize if I
overreacted.

>
> Maybe I wasn't sufficiently clear, so if rereading my original post doesn't
> bring about enlightenment, I'll try a restatement.
>
> > It turned out over the years that capabilities of Python optimization are
> > lower than those of Lisp and Smalltalk. But its a system effect and
> > epiphenomenon of certain design decisions.
>
> The point is that the design decisions, certainly for Common Lisp, scheme and
> particularly for dylan where also informed by what could be done
> *efficiently*, because the people who designed these languages knew a lot
> about advanced compiler implementation strategies for dynamic languages and
> thought that they could achieve high levels of expressiveness whilst retaining
> the possibility of very fast implementations (IIRC dylan specifically was
> meant to be something like within 90% of C performance). CL and dylan were
> also specifically designed for building very large and sophisticated systems,
> whereas it seems Guido originally thought that python would scale to about 500
> LOC.

O.K. To repeat it in an accurate manner. Python was originally designed
by Guido to be a scripting language for a new OS as a more complete
version of a shell scripting language. Unlike those its design was
strongly influenced by the usability ideas of the ABC development team.
Therefore speed considerations were not the primary concern but an open
model that was easily extendable both on the C-API level and the
language level. So a VM architecture was chosen to achieve this. Adding
new opcodes should have been as simple as interfacing with the C-API.
After growing strongly in the late 90s large scale projects emerged
such as Zope and many users started to request more Python performance
since they wanted to escape from the dual-language model. Writing
C-code was not self evident for a new programmers generation grewn up
with Java and the ffi turned out to be a hurdle. After remodeling the
object core ( "new style classes" ) progressive optimizations came to
hold. In 2002 a new genius programmer entered the scene, namely Armin
Rigo, who came up with Psyco and launched the PyPy project together
with a few other Python hackers in order to aggressively optimize
Python using Pythons introspective capabilities. That's were we still
are.

Remembering the historical context we might draw some parallels to
other contexts and language design intentions. We might also figure out
parallels and differences between motives of language designers and
leading persons who drive language evolution. Python is not just Guido
although his signature is quite pervasive. In his latest musings he
comes back to his central idea of language design as a kind of user
interface design. It's probably this shift in perspective that can be
attributed as original to him and which goes beyond making things just
"simple" or "powerfull" or "efficient" ( at least he made this shift
public and visible ). It is also the most controversial aspect of the
language because it is still inseparable from technical decisions (
non-true closures, explicit self, statement-expression distinction,
anonymous function as an expression with limited abilities etc. )

Kay

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: editor for Python on Linux

2006-02-21 Thread Fabio Zadrozny
Well... pydev has it: http://pydev.sf.net and pydev extensions goes much 
further: http://www.fabioz.com/pydev

Cheers,

Fabio

jean-michel bain-cornu wrote:

>>Boa-Constructor is an IDE rather than an editor.  Although it focuses
>>on wxPython, it has a good editor.
>>
>>
>Yes, some possibilities are pretty good, like for instance the 
>findAll/findInFiles keeping the results in a new tab (I never saw this 
>elsewhere). Another useful is having the debugger launching the 
>application in a separate process, an excellent way to avoid bugs 
>pollution coming into the IDE.
>rgds
>jm
>  
>


-- 
Fabio Zadrozny
--
Software Developer

ESSS - Engineering Simulation and Scientific Software
www.esss.com.br

Pydev Extensions
http://www.fabioz.com/pydev

PyDev - Python Development Enviroment for Eclipse
http://pydev.sf.net
http://pydev.blogspot.com


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: general coding issues - coding style...

2006-02-21 Thread calmar
On 2006-02-20, JW <[EMAIL PROTECTED]> wrote:

Hi JW,

> About this line:
> 1585 if sys.path[0][-12:] == "\library.zip":  #for py2exe
>
> pl... suggested:
> if sys.path[0].endswith( "\\library.zip" ):
>
>
> print "\\library.zip"
>

Yeah, I have two backslashes, but thaks for pointing out.

> If you don't use the double backslash, you'll eventually have a
> problem, especially in Windows, which unfortunately uses the backslash
> as a directory seperator.  You might also want to look at os.sep and
> the os.path.* functions, if you are interested in making your code work
> on different platforms.  

Yeah, I will use the os.sep variable, that's a good idea

thanks a lot,
calmar


-- 
  calmar

  (o_  It rocks: LINUX + Command-Line-Interface
  //\
  V_/_ http://www.calmar.ws
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: editor for Python on Linux

2006-02-21 Thread Sbaush
My favourite is Kate, available on KDE but working great also on Gnome!2006/2/21, Fabio Zadrozny <[EMAIL PROTECTED]>:
Well... pydev has it: http://pydev.sf.net and pydev extensions goes muchfurther: http://www.fabioz.com/pydevCheers,Fabio
jean-michel bain-cornu wrote:>>Boa-Constructor is an IDE rather than an editor.  Although it focuses>>on wxPython, it has a good editor.>Yes, some possibilities are pretty good, like for instance the
>findAll/findInFiles keeping the results in a new tab (I never saw this>elsewhere). Another useful is having the debugger launching the>application in a separate process, an excellent way to avoid bugs
>pollution coming into the IDE.>rgds>jm>>--Fabio Zadrozny--Software DeveloperESSS - Engineering Simulation and Scientific Software
www.esss.com.brPydev Extensionshttp://www.fabioz.com/pydevPyDev - Python Development Enviroment for Eclipse
http://pydev.sf.nethttp://pydev.blogspot.com--http://mail.python.org/mailman/listinfo/python-list
-- Sbaush
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie question: Multiple installations of Python on Windows machines

2006-02-21 Thread Fuzzyman

Don Taylor wrote:
> I have Python 2.4.2 installed on a Windows XP machine.
>
> There is an application that I want to use that refuses to install
> unless I have Python 2.3.x installed.  (The only way that I can install
> this is to use it's .exe installer)
>
> Can I install two versions of Python on Windows, and if so is there
> something I should do to make sure that the right version is used at the
> right time?  (2.3.x with this one package, and 2.4.2 with everything else).
>

A lot of 'exe' installers are special types of zip archvies. You might
be able to open it using winzip or winrar and do a manual install.

Another alternative is to use the free VMWare player to create a fresh
windows install that will run in a window. You can then install Python
2.3 and your application and extract the files you need to see if you
can make it work under Python 2.4.

If the application contains compiled extensions then they won't be
portable from Python 2.3 to Python 2.4 - however the application will
almost certainly run with Movable Python 2.3 (which won't interfere at
all with your Python 2.4 install).

   http://www.voidspace.org.uk/python/movpy/

You'll still have to get at the application files though. I have a
VMWare install with Python 2.3 on it, so I may be able to help.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Thanks,
> 
> Don.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: changing value of 'self' when subclassing int

2006-02-21 Thread Fuzzyman

David Coffin wrote:
> I'd like to subclass int to support list access, treating the integer
> as if it were a list of bits.
> Assigning bits to particular indices involves changing the value of
> the integer itself, but changing 'self' obviously just alters the
> value of that local variable.
> Is there some way for me to change the value of the BitSequence
> object itself? I've also tried wrapping and delegating using
> __getattr__, but I couldn't figure out how to handle in-place methods.
>

Probably your best bet is to have the assignment done as a method call
which returns the new value.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Thanks for your help,
> David Coffin
> 
>

-- 
http://mail.python.org/mailman/listinfo/python-list


odt -> pdf

2006-02-21 Thread Katja Suess
Hi
Cause Google didn't find the Info I was trying to find her my post.
Is PyUNO still _the_ Tool to create PDFs out of OpenOffice docs (odt, not swx) ?
Do other tools exist?
Do you prefer generatingp PDFs using the XML-Strukture of odt files?
Regards,
Katja

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: DCOracle2: OCI error

2006-02-21 Thread Chris
Update: Instead of disconnect / reconnect, I made it so that in case of
1084 errors I rollback and retry the same INSERT again.
It worked... It's not beautiful code but it's working.
So really I think something's fishy in dco2.c
If someone tell me what should be investigated I could look into the
issue.

Chris

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: [DICTIONARY] - Copy dictionary entries to attributes

2006-02-21 Thread Ilias Lazaridis
Diez B. Roggisch wrote:
> Ilias Lazaridis schrieb:
>> remark: not sure if the term "dictionary" is correct here.
>>
>> I have the following situation:
>>
>> within a setup.cfg, settings are passed this way:
>>
>> settings=project_page=theProjectPage.com
>> myVar=myValue
>>
>> those are accessible later like this:
>>
>> settings['project_page'] / settings['myValue']
>>
>> -
>>
>> Now my question: is there any standard function to map the settings 
>> directly to attributes?
>>
>> something like:
>>
>> dictionary_make_attributes(settings)
>>
>> thus they can be accessed via:
>>
>> settings.project_page / settings.myVar
>>
>> or
>>
>> copy_dictionary_entries_to_attributes(vars, settings)
>>
>> vars.project_page / vars.myVar
> 
> Either you use __getitem__, or the "bunch"-recipe:
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52308
> 
> Diez

Looks intresting, but I am a newcomer to python.

How would the __getitem__ implementation look like?

.

-- 
http://lazaridis.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: - Copy dictionary entries to attributes

2006-02-21 Thread Ilias Lazaridis
Ben Wilson wrote:
> Perhaps:
> 
> def dictionary_make_attributes(self, settings):
>  for k,v in settings:
>  setattr(self, k, v)

this one resulted in an error:

"ValueError: too many values to unpack"

it works with this correction:

for k,v in settings.items()

> http://ftp.python.org/doc/lib/built-in-funcs.html#l2h-64



-- 
http://lazaridis.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: - Copy dictionary entries to attributes

2006-02-21 Thread Ilias Lazaridis
Alex Martelli wrote:
> Ben Wilson <[EMAIL PROTECTED]> wrote:
> 
>> Perhaps:
>>
>> def dictionary_make_attributes(self, settings):
>>  for k,v in settings:
>>  setattr(self, k, v)

for k,v in settings.items()

> This is a very general solution and will work for all kinds of objects
> with settable attributes, even if some of the attributes are properties,
> slots or weirder descriptors yet.
> 
> For plain vanilla class instances, though,
>   self.__dict__.update(settings)
> may be sufficient.
> 
> Alex

this is the construct I was looking for.

see it active:

http://pudge.lesscode.org/trac/changeset/118

http://pudge.lesscode.org/trac/ticket/16

.

-- 
http://lazaridis.com
-- 
http://mail.python.org/mailman/listinfo/python-list


CGAL bindings

2006-02-21 Thread jelle
Hi,

I'm wondering whether anyone is aware of efforts being made wrapping
the CGAL library?
It's a highly advanced computational geometry library, see cgal.org

-j

//any other computational geometry links appreciated...

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: DCOracle2: OCI error

2006-02-21 Thread Diez B. Roggisch
Chris wrote:

> Update: Instead of disconnect / reconnect, I made it so that in case of
> 1084 errors I rollback and retry the same INSERT again.
> It worked... It's not beautiful code but it's working.
> So really I think something's fishy in dco2.c
> If someone tell me what should be investigated I could look into the
> issue.

No idea, but could as well be OCI. We're working here with ORACLE 10g with
the java JDBC-driver - and already found an oracle-bug and a driver-bug. So
what I would recommend is looking into what dco2 actually does at that
point - if it only puts a thin layer above OCI, and the problems come from
there.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: deriving from float or int

2006-02-21 Thread bearophileHUGS
Probably this is interesting for you:
http://home.tiscali.be/be052320/Unum.html

I think its API can be improved, but it can be used.

Bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Tkinter Checkboxes

2006-02-21 Thread D
Thank, Mikael..after messing with it for quite awhile, I finally found
out I need the .get()!  Appreciate the help..

Doug

-- 
http://mail.python.org/mailman/listinfo/python-list


algorithm, optimization, or other problem?

2006-02-21 Thread Brian Blais
Hello,

I am trying to translate some Matlab/mex code to Python, for doing neural 
simulations.  This application is definitely computing-time limited, and I need 
to 
optimize at least one inner loop of the code, or perhaps even rethink the 
algorithm. 
   The procedure is very simple, after initializing any variables:

1) select a random input vector, which I will call "x".  right now I have it as 
an 
array, and I choose columns from that array randomly.  in other cases, I may 
need to 
take an image, select a patch, and then make that a column vector.

2) calculate an output value, which is the dot product of the "x" and a weight 
vector, "w", so

y=dot(x,w)

3) modify the weight vector based on a matrix equation, like:

w=w+ eta * (y*x - y**2*w)
   ^
   |
   + learning rate constant

4) repeat steps 1-3 many times

I've organized it like:

for e in 100:   # outer loop
 for i in 1000:  # inner loop
 (steps 1-3)

 display things.

so that the bulk of the computation is in the inner loop, and is amenable to 
converting to a faster language.  This is my issue:

straight python, in the example posted below for 25 inner-loop steps, takes 
20 
seconds for each outer-loop step.  I tried Pyrex, which should work very fast 
on such 
a problem, takes about 8.5 seconds per outer-loop step.  The same code as a 
C-mex 
file in matlab takes 1.5 seconds per outer-loop step.

Given the huge difference between the Pyrex and the Mex, I feel that there is 
something I am doing wrong, because the C-code for both should run comparably. 
Perhaps the approach is wrong?  I'm willing to take any suggestions!  I don't 
mind 
coding some in C, but the Python API seemed a bit challenging to me.

One note: I am using the Numeric package, not numpy, only because I want to be 
able 
to use the Enthought version for Windows.  I develop on Linux, and haven't had 
a 
chance to see if I can compile numpy using the Enthought Python for Windows.

If there is anything else anyone needs to know, I'll post it.  I put the main 
script, 
and a dohebb.pyx code below.


thanks!

Brian Blais

-- 
-

 [EMAIL PROTECTED]
 http://web.bryant.edu/~bblais




# Main script:

from dohebb import *
import pylab as p
from Numeric import *
from RandomArray import *
import time

x=random((100,1000))# 1000 input vectors

numpats=x.shape[0]
w=random((numpats,1));

th=random((1,1))

params={}
params['eta']=0.001;
params['tau']=100.0;
old_mx=0;
for e in range(100):

 rnd=randint(0,numpats,25)
 t1=time.time()
 if 0:  # straight python
 for i in range(len(rnd)):
 pat=rnd[i]
 xx=reshape(x[:,pat],(1,-1))
 y=matrixmultiply(xx,w)
 w=w+params['eta']*(y*transpose(xx)-y**2*w);
 th=th+(1.0/params['tau'])*(y**2-th);
 else: # pyrex
 dohebb(params,w,th,x,rnd)
 print time.time()-t1


p.plot(w,'o-')
p.xlabel('weights')
p.show()


#=

# dohebb.pyx

cdef extern from "Numeric/arrayobject.h":

   struct PyArray_Descr:
 int type_num, elsize
 char type

   ctypedef class Numeric.ArrayType [object PyArrayObject]:
 cdef char *data
 cdef int nd
 cdef int *dimensions, *strides
 cdef object base
 cdef PyArray_Descr *descr
 cdef int flags


def dohebb(params,ArrayType w,ArrayType th,ArrayType X,ArrayType rnd):


 cdef int num_iterations
 cdef int num_inputs
 cdef int offset
 cdef double *wp,*xp,*thp
 cdef int *rndp
 cdef double eta,tau

 eta=params['eta']  # learning rate
 tau=params['tau']  # used for variance estimate

 cdef double y
 num_iterations=rnd.dimensions[0]
 num_inputs=w.dimensions[0]

 # get the pointers
 wp=w.data
 xp=X.data
 rndp=rnd.data
 thp=th.data

 for it from 0 <= it < num_iterations:

 offset=rndp[it]*num_inputs

 # calculate the output
 y=0.0
 for i from 0 <= i < num_inputs:
 y=y+wp[i]*xp[i+offset]

 # change in the weights
 for i from 0 <= i < num_inputs:
 wp[i]=wp[i]+eta*(y*xp[i+offset] - y*y*wp[i])

 # estimate the variance
 thp[0]=thp[0]+(1.0/tau)*(y**2-thp[0])





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: changing value of 'self' when subclassing int

2006-02-21 Thread Nick Craig-Wood
David Coffin <[EMAIL PROTECTED]> wrote:
>  I'd like to subclass int to support list access, treating the integer  
>  as if it were a list of bits.
>  Assigning bits to particular indices involves changing the value of  
>  the integer itself, but changing 'self' obviously just alters the  
>  value of that local variable.
>  Is there some way for me to change the value of the BitSequence  
>  object itself? I've also tried wrapping and delegating using  
>  __getattr__, but I couldn't figure out how to handle in-place
>  methods.

Here was my take on the problem when it came up for me.  Note the
FIXMEs ;-) My first attempt was to subclass long which was a dismal
failure.

class bits:
"""
Space efficient bit storage.  It returns something which acts
like an infinitely long array.  The array is initialised to all 0s

FIXME need to implement lots more methods!
FIXME check for -ve indices etc!
FIXME can do boolean methods by arithmetic on longs
FIXME can't subclass long as long is immutable :-(
"""
def __init__(self, initial = None):
"Initialise with the initial state of the array or nothing for it to be 
set to all 0s"
self.i = 0L
if initial:
for i in initial:
self.i <<= 1
self.i |= i
def mask(self, n):
return 1L << n
def __getitem__(self, n):
return (self.i & self.mask(n)) != 0L
def __setitem__(self, n, v):
m = self.mask(n)
if v:
self.i |= m
elif self.i & m:
self.i ^= m
# FIXME implement len() and use it in __repr__ to output directly!
def __repr__(self):
# FIXME is there a way to convert a long to binary?
content = ""
if self.i:
L = []
i = self.i
while i:
L.append(int(i & 1L))
i >>= 1
L.reverse()
content = str(L)
return "%s(%s)" % (self.__class__.__name__, content)

-- 
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Pyserial never read

2006-02-21 Thread Nick Craig-Wood
luca72 <[EMAIL PROTECTED]> wrote:
>  Thanks for your help, but it don't solve the problem.
>  I receive only the echo and full stop.

Try swapping pins 2 and 3 in the lead.

-- 
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to break a for loop?

2006-02-21 Thread bearophileHUGS
This time it seems that using itertools gives slower results, this is
the test code:

import itertools as it
from operator import __not__

def trim_leading_zeros(seq):
if not seq:
return seq
for pos, el in enumerate(seq):
if el != 0:
break
if seq[pos] == 0:
del seq[:]
return seq
return seq[pos:]

def trim_leading_zeros2(seq):
seq[:] = it.dropwhile(__not__, seq)
return seq

data = ([0,0,0,0,0,1,2,3,4],
[1,2,3,4],
[0,1],
[0,0,0],
[0,0,0,1],
[])

for l in data:
print l, trim_leading_zeros(l), trim_leading_zeros2(l)

from time import clock
n = 3 * 10**5

l = [0] * n + [1] * n
t = clock()
trim_leading_zeros(l)
print round(clock()-t, 2), "s"

l = [0] * n + [1] * n
t = clock()
trim_leading_zeros2(l)
print round(clock()-t, 2), "s"

Gives this output on my PC:

[0, 0, 0, 0, 0, 1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4]
[1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4]
[0, 1] [1] [1]
[0, 0, 0] [] []
[0, 0, 0, 1] [1] [1]
[] [] []
0.3 s
2.86 s

Bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: In need of a virtual filesystem / archive

2006-02-21 Thread Ivan Vilata i Balaguer
En/na Enigma Curry ha escrit::

> I need to store a large number of files in an archive. From Python, I
> need to be able to create an archive, put files into it, modify files
> that are already in it, and delete files already in it.
>[...]
> Is there any archive format that can allow Python to modify a file in
> the archive *in place*? That is to say if my archive is 2GB large and I
> have a small text file in the archive I want to be able to modify that
> small text file (or delete it) without having to rewrite the entire
> archive to disk.
>[...]

Although it is not its main usage, PyTables_ can be used to store
ordinary files in a single HDF5_ file.  HDF5 files have a hierarchical
structure of nodes and groups which maps quite well to files and
directories.  You can create, read, modify, copy, move and remove nodes
at will, freed space is reclaimed, and HDF5 is very efficient no matter
how large data is.

For working with the files, PyTables includes a FileNode_ module which
offers Python file semantics for nodes in an HDF5 file.  You can also
keep nodes transparently compressed, or you may repack the whole HDF5
file to defragment it or (de)compress its nodes, which may make a
reasonable option to a compressed archive.

I will be pleased to give more information.  Hope that helps.

.. _PyTables: http://www.pytables.org/
.. _HDF5: http://hdf.ncsa.uiuc.edu/HDF5/
.. _FileNode: http://pytables.sourceforge.net/html-doc/usersguide6.html

import disclaimer

::

Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
   Cárabos Coop. V.  V  V   Enjoy Data
  ""



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to break a for loop?

2006-02-21 Thread bonono

[EMAIL PROTECTED] wrote:
> This time it seems that using itertools gives slower results, this is
> the test code:
Understandable, as dropwhile still needs to go through the whole list
and the difference will be larger when the list get longer. Though I
still prefer that if the list is not horribly long as it is like a
spec. But then, one can always name the fast in place del slice version
like "dropwhileFast".

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: share function argument between subsequent calls but not between class instances!

2006-02-21 Thread Duncan Booth
Steven D'Aprano wrote:

>>> The most common idiom for such a marker is the None value.
>>> 
>> 
>> Can you provide any firm evidence that using None is more common?
> 
> 
> Yes, I wrote a quick and dirty script to roughly count the default
> values in the Python 2.3 standard library. Here are my results:
> 
> $ python default_counter.py
> 185 .py source files were opened.
> 4437 function or method definitions were found.
> These functions included at least 1228 arguments with default values.
> 529 or 4.307818e+01% used None as the default value.
> 
> So, roughly 40% of default values in the standard library are None. 

Fair enough, although I would point out that you haven't made any attempt 
to distinguish those cases where None is being used as a marker from the 
cases where it is being used as a value in its own right or a flag to 
control the function logic.

The marker cases do seem to be the most common but there are plenty of 
other cases:

e.g. base64.b64encode & base64.b64decode avoid part of the code if passed 
None, but don't actually substitute another value in place of the default.

cgi.FieldStorage has methods getvalue, getfirst where the default=None is 
simply that: the default to be returned. The make_file method has a 
defaulted argument which it doesn't use at all.

Also, most of the standard library predates a time when you could create a 
unique marker value just by calling 'object()'. When it was written None 
was by far the simplest option even in cases where a separate marker value 
might have been more appropriate.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Building python 2.4.2 on Cygwin

2006-02-21 Thread Jason Tishler
Steve,

On Mon, Feb 20, 2006 at 11:49:06PM -0500, Stephen Gross wrote:
> >On Mon, Feb 20, 2006 at 12:24:34PM -0800, mrstephengross wrote:
> >>Ok, I'm working on building python 2.4.2 on cygwin. I *think* it's
> >>version 3.0 or 3.1 (is there a quick way to find out what version of
> >>cygwin is running within a shell?)
> >
> >Use "uname -r".  What version are you running?
> 
> Uname tells me:
> 
> $ uname -r
> 1.5.18(0.132/4/2)

Just to be sure, what does the following indicate?

$ cygcheck -c cygwin

Sorry, but I cannot reproduce your problem under Cygwin 1.5.18.

BTW, can you use the pre-built Python 2.4.1 that is part of the standard
Cygwin distribution?

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6
-- 
http://mail.python.org/mailman/listinfo/python-list


CGI and Redirect

2006-02-21 Thread Grzegorz Ślusarek
Hi All. I need to redirect user in my CGI script,  i Try to use prin 
"Location: "+url but this is not working. Can anyone tell me what I'm 
doing wrong?
Any thanks will be apreciated
Gregor
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: CGI and Redirect

2006-02-21 Thread Gerhard Häring
Grzegorz Ślusarek wrote:
> Hi All. I need to redirect user in my CGI script,  i Try to use prin 
> "Location: "+url but this is not working. Can anyone tell me what I'm 
> doing wrong?
> Any thanks will be apreciated

I guess you forgot to set the HTTP-Status. Either:

print "Status: 301" # Moved Permanently

or

print "Status: 302" # Moved Temporarily

HTH,

-- Gerhard
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CGI and Redirect

2006-02-21 Thread Fredrik Lundh
Grzegorz Slusarek wrote:

> Hi All. I need to redirect user in my CGI script,  i Try to use prin
> "Location: "+url but this is not working. Can anyone tell me what I'm
> doing wrong?

something like

print "Status: 302"
print "Location: " + new_url
print
sys.exit(1)

should work (assuming that sys is imported, of course)

(if it doesn't, you have to define "not working")

hope this helps!

 



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Building python 2.4.2 on Cygwin

2006-02-21 Thread Stephen Gross
>Just to be sure, what does the following indicate?
$ cygcheck -c cygwin
Cygwin Package Information
Package  VersionStatus
cygwin   1.5.18-1   OK

>BTW, can you use the pre-built Python 2.4.1 that is part of the standard
Cygwin distribution?

Nope--I need to do a custom python build for a very specific situation...

Thanks,
--Steve
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread fraca7
Suresh Jeevanandam a écrit :
> # I am new to python.
> 
> In python all numbers are immutable. This means there is one object ( a 
> region in the memory ) created every time we do an numeric operation. I 
> hope there should have been some good reasons why it was designed this way.

The memory allocation for integers is optimized. 'Small' integers 
(between -5 and 100 IIRC) are allocated once and reused. The memory for 
larger integers is allocated once and reused whenever possible, so the 
malloc() overhead is negligible.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python vs. Lisp -- please explain

2006-02-21 Thread Chris Mellon
On 2/20/06, Donn Cave <[EMAIL PROTECTED]> wrote:
> Quoth Steven D'Aprano <[EMAIL PROTECTED]>:
> ...
> | Nobody denies that Python code running with no optimization tricks is
> | (currently) slower than compiled C code. That's a matter of objective
> | fact. Nobody denies that Python can be easily run in interactive mode.
> | Nobody denies that *at some level* Python code has to be interpreted.
> |
> | But ALL code is interpreted at some level or another. And it is equally
> | true that at another level Python code is compiled. Why should one take
> | precedence over the other?
>
> I have no idea, what precedence?  All I'm saying is that Python matches
> what people think of as an interpreted language.  You can deny it, but
> but it's going to look like you're playing games with words, and to no
> real end, since no one could possibly be deceived for very long.  If you
> give me a Python program, you have 3 choices:  cross your fingers and
> hope that I have the required Python interpreter version, slip in a
> 25Mb Python interpreter install and hope I won't notice, or come clean
> and tell me that your program needs an interpreter and I should check to
> see that I have it.


You're correct as far as it goes, but can you provide a reasonable
definition for "interpreted" that matches the common usage? Most
people can't.

When asked to name some interpreted (or scripting) languages, they'll
name some off - perl, python, ruby, javascript, basic...

They won't say Java. Ask them why Python is interpreted and Java isn't
and you'll have a hard time getting a decent technical answer, because
Python isn't all that different from Java in that regard, especially
pre-JIT versions of Java.

Probably the most accurate definition of "interpreted" as it is used
in the wild is "one of these languages: perl, python, perl, ruby,
etc". That is, you're essentially claiming that Python is interpreted
because everyone thinks of it that way, technical correctness be
damned.

There is an obvious difference between Python and C. Nobody would deny
that. But it's a fairly hard thing to *quantify*, which is why people
make sloppy categorizations. That's not a problem as long as there
isn't prejudice associated with the categorization, which there is.

I wonder how "interpreted" people would think Python is if the
automagic compilation to .pyc was removed and you had to call
"pythonc" first.

>
> Donn Cave, [EMAIL PROTECTED]
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list


message

2006-02-21 Thread Nina Almaguer



Hi, 

We are a web 
integration firm in Reston, VA and would like to post the 
following developer position (I have already joined the 
group).
 
 

When you begin your career 
at Siteworx, you’ll be part of a rapidly growing software and services company. 
In fact, you have probably visited some of our web applications and sites 
already – many are for companies with household names. It’s this kind of success 
that creates exceptional opportunities for ambitious people. If you’re looking 
for a place where you can perform your best, consider the following position in 
our Reston, Virginia headquarters:
 
Software 
Developer
 
You will develop Internet applications in teams from Siteworx 
specifications. You will work closely with many Senior Developers to improve 
your knowledge and technical prowess.  
Siteworx has very high standards and understands that very few people are 
qualified to do what we do upon hiring, however, the position requires a BA/BS 
in a technical field and 1+ years experience (internships applicable) in 
software development.  Fluency with 
three or more of following is necessary: XHTML, CSS, C/C++, _javascript_, Java, 
Python, XSLT.  A background in 
Object Oriented Programming is necessary and familiarity with Zope is a major 
plus.
 
Siteworx has outstanding 
benefits including 401(k), and health/dental insurance. If you are interested, 
please e-mail your resume to: [EMAIL PROTECTED]. 

 
 
 
 

.Nina 
Almaguer.SITEWORX1892 
 Preston White DriveThird FloorReston, VA 20191703 520 1550 
x224 direct 
571 
239 2841 mobile 
703 
935  0426  fax.[EMAIL PROTECTED]www.siteworx.com
 
 
-- 
http://mail.python.org/mailman/listinfo/python-list

Zope/Plone - Is it the right solution?

2006-02-21 Thread kbperry
Hi everyone,
I am currently a student, and for our HCI class project we are
redeveloping our CS website.  I attend a very large university (around
30,000 students), and the CS site will need to be updated by many
people that don't have technical skills (like clerical staff).

The biggest problem with the current site is that not enough people
have access to update it.  Since I love python, these seemed like
viable solutions.

1)  Is Zope/Plone overkill for this type of project?

2)  Why use Plone vs. straight up Zope?

3)  Is there a way to get over the steep learning curves (that I have
read about)?

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: algorithm, optimization, or other problem?

2006-02-21 Thread Gerard Flanagan
Brian Blais wrote:
> Hello,
>
> I am trying to translate some Matlab/mex code to Python, for doing neural
> simulations.  This application is definitely computing-time limited, and I 
> need to
> optimize at least one inner loop of the code, or perhaps even rethink the 
> algorithm.
>The procedure is very simple, after initializing any variables:
>
> 1) select a random input vector, which I will call "x".  right now I have it 
> as an
> array, and I choose columns from that array randomly.  in other cases, I may 
> need to
> take an image, select a patch, and then make that a column vector.
>
> 2) calculate an output value, which is the dot product of the "x" and a weight
> vector, "w", so
>
>   y=dot(x,w)
>
> 3) modify the weight vector based on a matrix equation, like:
>
>   w=w+ eta * (y*x - y**2*w)
>^
>|
>+ learning rate constant
>
> 4) repeat steps 1-3 many times
>


Brian

ETA = 0.001
INV_TAU = 1.0/100.0

rather than:

> params={}
> params['eta']=0.001;
> params['tau']=100.0;

ie. take the dictionary lookup (and the repeated division) out?

If you have a collection of random vectors do you gain anything by
*choosing* them randomly?

I'm not exactly sure if it's equivalent to your existing code, but how
about:

x=random((100,25))
w=random((100,1));
th=random((1,1))
for vector in x:
...
w=w+ ETA * (y*vector - y**2*w)
th = th + INV_TAU *(y**2-th)
...

?  (I'm not familar with Numeric)

Gerard

> old_mx=0;
> for e in range(100):
>
>  rnd=randint(0,numpats,25)
>  t1=time.time()
>  if 0:  # straight python
>  for i in range(len(rnd)):
>  pat=rnd[i]
>  xx=reshape(x[:,pat],(1,-1))
>  y=matrixmultiply(xx,w)
>  w=w+params['eta']*(y*transpose(xx)-y**2*w);
>  th=th+(1.0/params['tau'])*(y**2-th);
>  else: # pyrex
>  dohebb(params,w,th,x,rnd)
>  print time.time()-t1
> 
>

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: algorithm, optimization, or other problem?

2006-02-21 Thread Gerard Flanagan

Gerard Flanagan wrote:
> Brian Blais wrote:
> > Hello,
> >
> > I am trying to translate some Matlab/mex code to Python, for doing neural
> > simulations.  This application is definitely computing-time limited, and I 
> > need to
> > optimize at least one inner loop of the code, or perhaps even rethink the 
> > algorithm.
> >The procedure is very simple, after initializing any variables:
> >
> > 1) select a random input vector, which I will call "x".  right now I have 
> > it as an
> > array, and I choose columns from that array randomly.  in other cases, I 
> > may need to
> > take an image, select a patch, and then make that a column vector.
> >
> > 2) calculate an output value, which is the dot product of the "x" and a 
> > weight
> > vector, "w", so
> >
> > y=dot(x,w)
> >
> > 3) modify the weight vector based on a matrix equation, like:
> >
> > w=w+ eta * (y*x - y**2*w)
> >^
> >|
> >+ learning rate constant
> >
> > 4) repeat steps 1-3 many times
> >
>
>
> Brian
>
> ETA = 0.001
> INV_TAU = 1.0/100.0
>
> rather than:
>
> > params={}
> > params['eta']=0.001;
> > params['tau']=100.0;
>

or:

ETA = params['eta']
INV_TAU = 1.0/params['tau']


Gerard

> ie. take the dictionary lookup (and the repeated division) out?
>
> If you have a collection of random vectors do you gain anything by
> *choosing* them randomly?
>
> I'm not exactly sure if it's equivalent to your existing code, but how
> about:
>
>   x=random((100,25))
>   w=random((100,1));
>   th=random((1,1))
>   for vector in x:
>   ...
>   w=w+ ETA * (y*vector - y**2*w)
>   th = th + INV_TAU *(y**2-th)
>   ...
>
> ?  (I'm not familar with Numeric)
>
> Gerard
>
> > old_mx=0;
> > for e in range(100):
> >
> >  rnd=randint(0,numpats,25)
> >  t1=time.time()
> >  if 0:  # straight python
> >  for i in range(len(rnd)):
> >  pat=rnd[i]
> >  xx=reshape(x[:,pat],(1,-1))
> >  y=matrixmultiply(xx,w)
> >  w=w+params['eta']*(y*transpose(xx)-y**2*w);
> >  th=th+(1.0/params['tau'])*(y**2-th);
> >  else: # pyrex
> >  dohebb(params,w,th,x,rnd)
> >  print time.time()-t1
> > 
> >

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope/Plone - Is it the right solution?

2006-02-21 Thread Chris Lasher
You may want to look at either of the popular frameworks, TurboGears
http://www.turbogears.org/ or Django http://www.djangoproject.com/

I have very little experience with both, but I decided to try learning
the Django framework after watching the Snakes and Rubies videos. (See
http://www.djangoproject.com/snakesandrubies/ ) I am working my way
through the Django tutorials and am very impressed and looking forward
to deploying it on my own site. The admin interface that you get
basically "free" is a very slick touch, especially with all the
widgets. My recommendation is to check out Django's admin interface
(see Adrian Holovaty's presentation of Django in Snakes and Rubies and
http://www.djangoproject.com/documentation/tutorial2/ ) and see if that
won't do the trick for allowing easy content contribution without the
contributors having to know any of the Python, HTML, or CSS behind the
pages.

Chris

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Writing my own typing monitor program for RSI sufferers...

2006-02-21 Thread skip

> So,  so close  The  problem  with this  implementation is  that it
>  doesn't  monitor usb keyboards under   linux at all   as far as I can
> tellsince no keyboard entry  will show up in /proc/interrupts with
> a usb keyboard.  I absolutely need the keyboard monitoring as well.

Sam's> Otherwise, your project would be the perfect starting point for me!

Hmmm...  I'm confused.  It is a starting point, just not one that monitors
USB keyboards.  A "perfect" starting point would be one in which you didn't
need to make any changes.  Why throw the baby out with the bath water?  Why
not just rewrite check_linux_interrupts to somehow watch the usb keyboard as
well (however that might be done)?  It's not going to be any more difficult
to check for USB keyboard interrupts in this project than in some other and
it already has a more-or-less functional user interface and support for
Windows.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Building python 2.4.2 on Cygwin

2006-02-21 Thread Jason Tishler
Steve,

On Tue, Feb 21, 2006 at 09:31:48AM -0500, Stephen Gross wrote:
> >Just to be sure, what does the following indicate?
> $ cygcheck -c cygwin
> Cygwin Package Information
> Package  VersionStatus
> cygwin   1.5.18-1   OK

Try reinstalling Cygwin 1.5.18 or upgrading to 1.5.19.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope 3?? How do you start it on Windows?

2006-02-21 Thread kbperry
Cool thx Matt.  I did finally figure it out, but barely.  Why would you
want to download and install Zope without creating an instance?  It
seems kind of dumb to me.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Print a PDF transparently

2006-02-21 Thread Grant Edwards
On 2006-02-20, Tim Golden <[EMAIL PROTECTED]> wrote:
> [Daniel Crespo]
>| 
>| > Have you seen this?
>| > http://tgolden.sc.sabren.com/python/win32_how_do_i/print.html
>| > In particular, the section on using win32print directly.
>| 
>| Yes, I have. The problems is that an external program is launched for
>| handling the file and print it. 
>
> [sorry, bit long-winded, but trying to help]
>
> I think the poster was referring to the situation you 
> described where the problem reduced to sending a .ps
> file directly to a postscript-capable printer.


People are still using OS distributions that don't know how to
print Postscript or PDF "out of the box"? 

All of the decent Linux distros have had the ability to print
postscript/PDF files "transparently" for many, many years (even
to non-postscript printers).

Sheesh.


-- 
Grant Edwards   grante Yow!  Four thousand
  at   different MAGNATES, MOGULS
   visi.com& NABOBS are romping in my
   gothic solarium!!
-- 
http://mail.python.org/mailman/listinfo/python-list


No

2006-02-21 Thread Gaz
Check this piece of code:

#now add an image part
part = writer.nextpart()
part.addheader('Content-Transfer-Encoding', 'base64')
body = part.startbody('image/jpeg')
base64.encode(open('c:\check.jpg', 'rb'), body)

I get the following error:

IOError: [Errno 2] No such file or directory: 'c:\\check.jpg'
  args = (2, 'No such file or directory')
  errno = 2
  filename = r'c:\check.jpg'
  strerror = 'No such file or directory'

Dont know why im getting such error, the file is there. Perhaps wrong
filetype? Wrong read mode?

-- 
http://mail.python.org/mailman/listinfo/python-list


popen2() does not work correctly on windows!

2006-02-21 Thread Peter Cai
I want to catch the output of a program but I can not use redirect
because it's interactive and thus I need to see the output on the
screen.

So I use popen2() to execute it and then both print the output on the screen and in a log file.
But when I want to write something command to it, it seems that the program can not get it.

I'm working on windows XP-- 那不是缺陷 只是你不在梦中
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Zope/Plone - Is it the right solution?

2006-02-21 Thread Fredrik Lundh
"kbperry" wrote:

> I am currently a student, and for our HCI class project we are
> redeveloping our CS website.  I attend a very large university (around
> 30,000 students), and the CS site will need to be updated by many
> people that don't have technical skills (like clerical staff).
>
> The biggest problem with the current site is that not enough people
> have access to update it.  Since I love python, these seemed like
> viable solutions.
>
> 1)  Is Zope/Plone overkill for this type of project?
>
> 2)  Why use Plone vs. straight up Zope?
>
> 3)  Is there a way to get over the steep learning curves (that I have
> read about)?

do you want to build a web application, use a ready-made CMS, or is the goal to
easily get lots of information on to the (intra)web ?

if the latter, a modern wiki with good access control could be worth 
investigating:

http://moinmoin.wikiwikiweb.de/
http://moinmoin.wikiwikiweb.de/HelpOnAccessControlLists

(for performance, you may want to run the wiki behind mod_proxy)

if you want a ready-made content management system, pick Plone.

if you want to build your own web application, you know Python reasonably
well, you don't want much of a learning curve, and you want to start right now,
pick Django.

(I'm now leaving the microphone to the "pick me pick me!" crowd ;-)

 



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: [Numpy-discussion] algorithm, optimization, or other problem?

2006-02-21 Thread Bruce Southey
Hi,
In the current version, note that Y is scalar so replace the squaring
(Y**2) with Y*Y as you do in the dohebb function.  On my system
without blas etc removing the squaring removes a few seconds (16.28 to
12.4). It did not seem to help factorizing Y.

Also, eta and tau are constants so define them only once as scalars
outside the loops and do the division outside the loop. It only saves
about 0.2 seconds but these add up.

The inner loop probably can be vectorized because it is just vector
operations on a matrix. You are just computing over the ith dimension
of X.  I think that you could be able to find the matrix version on
the net.

Regards
Bruce



On 2/21/06, Brian Blais <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I am trying to translate some Matlab/mex code to Python, for doing neural
> simulations.  This application is definitely computing-time limited, and I 
> need to
> optimize at least one inner loop of the code, or perhaps even rethink the 
> algorithm.
>The procedure is very simple, after initializing any variables:
>
> 1) select a random input vector, which I will call "x".  right now I have it 
> as an
> array, and I choose columns from that array randomly.  in other cases, I may 
> need to
> take an image, select a patch, and then make that a column vector.
>
> 2) calculate an output value, which is the dot product of the "x" and a weight
> vector, "w", so
>
> y=dot(x,w)
>
> 3) modify the weight vector based on a matrix equation, like:
>
> w=w+ eta * (y*x - y**2*w)
>^
>|
>+ learning rate constant
>
> 4) repeat steps 1-3 many times
>
> I've organized it like:
>
> for e in 100:   # outer loop
>  for i in 1000:  # inner loop
>  (steps 1-3)
>
>  display things.
>
> so that the bulk of the computation is in the inner loop, and is amenable to
> converting to a faster language.  This is my issue:
>
> straight python, in the example posted below for 25 inner-loop steps, 
> takes 20
> seconds for each outer-loop step.  I tried Pyrex, which should work very fast 
> on such
> a problem, takes about 8.5 seconds per outer-loop step.  The same code as a 
> C-mex
> file in matlab takes 1.5 seconds per outer-loop step.
>
> Given the huge difference between the Pyrex and the Mex, I feel that there is
> something I am doing wrong, because the C-code for both should run comparably.
> Perhaps the approach is wrong?  I'm willing to take any suggestions!  I don't 
> mind
> coding some in C, but the Python API seemed a bit challenging to me.
>
> One note: I am using the Numeric package, not numpy, only because I want to 
> be able
> to use the Enthought version for Windows.  I develop on Linux, and haven't 
> had a
> chance to see if I can compile numpy using the Enthought Python for Windows.
>
> If there is anything else anyone needs to know, I'll post it.  I put the main 
> script,
> and a dohebb.pyx code below.
>
>
> thanks!
>
> Brian Blais
>
> --
> -
>
>  [EMAIL PROTECTED]
>  http://web.bryant.edu/~bblais
>
>
>
>
> # Main script:
>
> from dohebb import *
> import pylab as p
> from Numeric import *
> from RandomArray import *
> import time
>
> x=random((100,1000))# 1000 input vectors
>
> numpats=x.shape[0]
> w=random((numpats,1));
>
> th=random((1,1))
>
> params={}
> params['eta']=0.001;
> params['tau']=100.0;
> old_mx=0;
> for e in range(100):
>
>  rnd=randint(0,numpats,25)
>  t1=time.time()
>  if 0:  # straight python
>  for i in range(len(rnd)):
>  pat=rnd[i]
>  xx=reshape(x[:,pat],(1,-1))
>  y=matrixmultiply(xx,w)
>  w=w+params['eta']*(y*transpose(xx)-y**2*w);
>  th=th+(1.0/params['tau'])*(y**2-th);
>  else: # pyrex
>  dohebb(params,w,th,x,rnd)
>  print time.time()-t1
>
>
> p.plot(w,'o-')
> p.xlabel('weights')
> p.show()
>
>
> #=
>
> # dohebb.pyx
>
> cdef extern from "Numeric/arrayobject.h":
>
>struct PyArray_Descr:
>  int type_num, elsize
>  char type
>
>ctypedef class Numeric.ArrayType [object PyArrayObject]:
>  cdef char *data
>  cdef int nd
>  cdef int *dimensions, *strides
>  cdef object base
>  cdef PyArray_Descr *descr
>  cdef int flags
>
>
> def dohebb(params,ArrayType w,ArrayType th,ArrayType X,ArrayType rnd):
>
>
>  cdef int num_iterations
>  cdef int num_inputs
>  cdef int offset
>  cdef double *wp,*xp,*thp
>  cdef int *rndp
>  cdef double eta,tau
>
>  eta=params['eta']  # learning rate
>  tau=params['tau']  # used for variance estimate
>
>  cdef double y
>  num_iterations=rnd.dimensions[0]
>  num_inputs=w.dimensions[0]
>
>  # get the pointers
>  wp=w.data
>  xp=X.data
>  rndp=rnd.data
>  thp=th.data
>
>  for it from 0 <= it < num_iterations:
>

Re: Mutable numbers

2006-02-21 Thread Steve Holden
fraca7 wrote:
> Suresh Jeevanandam a écrit :
> 
>># I am new to python.
>>
>>In python all numbers are immutable. This means there is one object ( a 
>>region in the memory ) created every time we do an numeric operation. I 
>>hope there should have been some good reasons why it was designed this way.
> 
> 
> The memory allocation for integers is optimized. 'Small' integers 
> (between -5 and 100 IIRC) are allocated once and reused. The memory for 
> larger integers is allocated once and reused whenever possible, so the 
> malloc() overhead is negligible.

The first bit's right, the second bit isn't:

  >>> id(12100)
4604168
  >>> id(121*100)
4604204
  >>>

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Steve Holden
fraca7 wrote:
> Suresh Jeevanandam a écrit :
> 
>># I am new to python.
>>
>>In python all numbers are immutable. This means there is one object ( a 
>>region in the memory ) created every time we do an numeric operation. I 
>>hope there should have been some good reasons why it was designed this way.
> 
> 
> The memory allocation for integers is optimized. 'Small' integers 
> (between -5 and 100 IIRC) are allocated once and reused. The memory for 
> larger integers is allocated once and reused whenever possible, so the 
> malloc() overhead is negligible.

[Thinks: or maybe fraca7 just meant that integers will be garbage 
collected when there are no more references to them].

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Print a PDF transparently

2006-02-21 Thread Daniel Crespo
Thank you very much Roger Upole and Tim Golden for your posts...

I found an exe that can print a pdf file (using Acrobat Reader) on any
windows printer right from the command line. It opens Acrobat, order it
to print the pdf file at a certain printer and then closes Acrobat. The
exe is a c-compiled program from someone that supplied it in a forum.

That works almost perfect for me. The only thing now is that when I'm
printing a PDF in dot matrix printers, the output is very ugly, and I
don't know how to improve it.

I think I will go for printing in plain text. The only problem with it
is that one have to depend on the physical printer configuration.

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: editor for Python on Linux

2006-02-21 Thread Joel Hedlund
>>I really think that IDLE is one of the best around in Python source editing.
> 
> For me, I find that IDLE is about the worse for editing Python sources.

"Worse"? Now that's harsh. I'm with billie on this one. I usually spend a day 
or so every 3 months trying to find a free python editor that surpasses IDLE. 
I've been doing it for 3 years now and for me, IDLE is still king of the hill. 

What I like about IDLE is that it is configurable, intuitive and rock stable, 
and I've come to realize that combination is rare indeed in the world of free 
editors. 

Another pro for IDLE is that you probably already have it installed, since it 
comes included in the standard python releases. If you decide to give IDLE a go 
you might also want to check out the latest subversion version of IDLE, since 
it has a bunch of really useful syntax helper updates.

Cheers!
/Joel Hedlund
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Fredrik Lundh
Steve Holden wrote:

> > The memory allocation for integers is optimized. 'Small' integers
> > (between -5 and 100 IIRC) are allocated once and reused. The memory for
> > larger integers is allocated once and reused whenever possible, so the
> > malloc() overhead is negligible.
>
> The first bit's right, the second bit isn't:
>
>   >>> id(12100)
> 4604168
>   >>> id(121*100)
> 4604204
>   >>>

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

-- 
http://mail.python.org/mailman/listinfo/python-list



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope/Plone - Is it the right solution?

2006-02-21 Thread kbperry
Well,
I guess our main goal in this class is to improve usability and user
experiences at the site.

While we want to improve our site visually and make it more usable to
the prospective students, the site needs to be easily updated.  I don't
think that I am looking for a wiki.

I will definitely check out Django and turbogears.

Thanks a ton for the info!

-- 
http://mail.python.org/mailman/listinfo/python-list


Video Capture modules in linux

2006-02-21 Thread [EMAIL PROTECTED]
heya,

i need this video capture modules which i can use in linux, u know the
similar one used in windows, Now Windows is not my type and it would be
really helpful if someone helps me out...i tried libfg but its not
useful in my case (BTW its a great library)...

regards
Sudharshan S

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Pyserial never read

2006-02-21 Thread Peter Hansen
Nick Craig-Wood wrote:
> luca72 <[EMAIL PROTECTED]> wrote:
> 
>> Thanks for your help, but it don't solve the problem.
>> I receive only the echo and full stop.
> 
> Try swapping pins 2 and 3 in the lead.

Anything's possible, but given that in his original post he says it 
works when he uses Delphi, it seems unlikely making a change to the 
hardware is necessary.

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


PDF on dot matrix printers

2006-02-21 Thread Daniel Crespo
Hi to all,

Does anyone know about how to make a dot matrix printer print a PDF
file with the quality of a Microsoft Word printing?

When I print from Microsoft Word, the output is perfect for the kind of
printer. But, from a PDF file, the characters are distortionated in an
uniform way. With "uniform" I mean that no matter where I put the
characters in the page, the character "e" always will be smaller than
an "a". So, when seeing the output page, and trying to read what it has
written, you realize that the characters have different heights... It
could be a matter of resolution, obviously not in the printer, but in
the PDF. So I'm wondering if there are some tasks that I could do for
accomplish it.

Thanks for reading

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Fredrik Lundh
Steve Holden wrote:

> > The memory allocation for integers is optimized. 'Small' integers
> > (between -5 and 100 IIRC) are allocated once and reused. The memory for
> > larger integers is allocated once and reused whenever possible, so the
> > malloc() overhead is negligible.
>
> The first bit's right, the second bit isn't:
>
>   >>> id(12100)
> 4604168
>   >>> id(121*100)
> 4604204
>   >>>

quoting from Objects/intobject.c :

   Integers are quite normal objects, to make object handling uniform.
   (Using odd pointers to represent integers would save much space
   but require extra checks for this special case throughout the code.)
   Since a typical Python program spends much of its time allocating
   and deallocating integers, these operations should be very fast.
   Therefore we use a dedicated allocation scheme with a much lower
   overhead (in space and time) than straight malloc(): a simple
   dedicated free list, filled when necessary with memory from malloc().

http://svn.python.org/projects/python/trunk/Objects/intobject.c

 



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Dave Hansen
On Tue, 21 Feb 2006 12:44:52 +0530 in comp.lang.python, Suresh
Jeevanandam <[EMAIL PROTECTED]> wrote:

># I am new to python.
>
>In python all numbers are immutable. This means there is one object ( a 
>region in the memory ) created every time we do an numeric operation. I 
>hope there should have been some good reasons why it was designed this way.
>
>But why not have mutable numbers also in the language. A type which 
>would behave as follows:
>
>a = MutableInt(12)

a = [12]

>b = a
>
>Now both a and b should refer to the same memory location. Any change in 
>the object should get reflected in all the references.
>
>a.assign(13) # Same memory location should be updated with value 13, b 

a[0] = 13

>is also 13 now.
>
>Now we can further optimize:
>
>a.incrementOne() # equivalent to a++ in C++
>a.decrementOne()

a[0] += 1
a[0] -= 1

>
>and the augmented assignment operation also could be made optimized.
>
>In any application most of the operation is numerical. So, i think, we 
>should get a good speed advantage with the availability of mutable 
>numbers. What do you think ?

I don't see how.

HTH,
-=Dave

-- 
Change is inevitable, progress is not.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Pyserial never read

2006-02-21 Thread Peter Hansen
luca72 wrote:
> Thanks for your help, but it don't solve the problem.
> I receive only the echo and full stop.

If you want help, you'll do better to post small pieces of code that you 
are actually using, rather than making us guess or imagine what you are 
doing.  There are perhaps a dozen things that can go wrong with serial 
communications, and it's not efficient for us to start suggesting them 
one at a time...

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


streaming Popen.stdout

2006-02-21 Thread Michele Simionato
I was experimenting with subprocess.Popen with the following script:

$ cat Popen_ex.py
import sys, subprocess
po = subprocess.Popen([sys.executable, 'hello.py'],
stdout=subprocess.PIPE,
  bufsize=0)
for line in po.stdout:
print line,

where hello.py is the following script:

$ cat hello.py
import time
time.sleep(1)
print 'hello'
time.sleep(1)
print 'world'
time.sleep(1)
print '*END*'
time.sleep(1)

It turns out that Popen collects all the output and write everything
together
after 4 seconds, where I would like to print a line every seconds.
How can I get that in a simple way? A Unix-only solution would be fine,
too.


Michele Simionato

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: No

2006-02-21 Thread Peter Hansen
Gaz wrote:
> Check this piece of code:
> 
> #now add an image part
> part = writer.nextpart()
> part.addheader('Content-Transfer-Encoding', 'base64')
> body = part.startbody('image/jpeg')
> base64.encode(open('c:\check.jpg', 'rb'), body)
> 
> I get the following error:
> 
> IOError: [Errno 2] No such file or directory: 'c:\\check.jpg'
>   args = (2, 'No such file or directory')
>   errno = 2
>   filename = r'c:\check.jpg'
>   strerror = 'No such file or directory'
> 
> Dont know why im getting such error, the file is there. Perhaps wrong
> filetype? Wrong read mode?

What does it show if you put "print os.listdir('c:/')" on the line 
before the open?  (You'll obviously need to import os first.)

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread fraca7
Steve Holden a écrit :

> [Thinks: or maybe fraca7 just meant that integers will be garbage 
> collected when there are no more references to them].

Actually I meant that the memory is reused, but the same integer won't 
always have the same address.

I guess that in your example, the '121' is assigned the 'old' address of 
12100, and the result of the operation is assigned the next chunk; the 
following seems to confirm that:

  >>> id(12100)
  8628480
  >>> id(102)
  8628480
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread fraca7
fraca7 a écrit :
> Steve Holden a écrit :
> 
>> [Thinks: or maybe fraca7 just meant that integers will be garbage 
>> collected when there are no more references to them].
> 
> Actually I meant that the memory is reused, but the same integer won't 
> always have the same address.

And of course this means they're garbaged collected, but free() won't be 
called. I must read before answering :)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Video Capture modules in linux

2006-02-21 Thread [EMAIL PROTECTED]
well wht can i say it simply refuses to compile...i m getting make
errors. i think the culprit is the gcc-4.0..secondly i havent had a
chance to really try it out..aneways does libfg manipulate inputs from
webcams..like VideoCapture modules ..also i think libfg is mainly for C
although a module for python exists..forgive me i m a real newbie..

regards
Sudharshan S

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope/Plone - Is it the right solution?

2006-02-21 Thread Rene Pijlman
Michele Simionato:
>I usually do not recommend Zope, unless you want to make a career as a
>Zope consultant. 

You have a point there.

>If you have to interact with a relational database, your life with Zope 
>may be hard: 

And if you have to interact with flat files, your life with an RDBMS may
be hard. Zope contains a full-fledged post-relational object database
called ZODB.

The time it took me to learn to program with it (outside of Zope) is less
than 1% of the time I've spent learning the relational model and SQL.

Get your data out of those silly rows and columns, and put it in
first-class objects!

-- 
René Pijlman
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Multiple assignment and the expression on the right side

2006-02-21 Thread Alex Martelli
Suresh Jeevanandam <[EMAIL PROTECTED]> wrote:
   ...
> I think I got confused because of "Each time" in the sentence which 
> gives a feeling that it gets executed several times. Maybe, It could 
> have been just written, "When the statement gets executed, the right 
> hand side is evaluated once, and the result is assigned to each of the
> target".

Ah, but that suggests to *ME* that if you have the statement in a loop,
the RHS executes only once (the first time).  Let's try:

"""
Each time the statement executes, the RHS expression is evaluated just
once, no matter how many targets are part of the statement. Each target
then gets bound to the single object returned by the expression, just as
if several simple assignments executed one after the other. 
"""

[[I'm using LHS and RHS in the 2nd edition, for extra conciseness, after
introducing the acronyms just once where I first use them]].

This seems to cover the bases:

a. RHS is evaluated EACH TIME the statement executes,
b. but for each such time, RHS is evaluated JUST ONCE.

and the addition of the "no matter" clause should pound the nail into
the coffin.  What do y'all think?


Thanks for the comments, by the way, and my apologies for reacting to
them rather snappily at first blush -- I had just spent a long weekend
mostly slaving on the 2nd edition, so I guess I was irritable, but
that's no excuse for my discourtesy: sorry. Comments and criticisms on
my work are in fact always welcome (submitting them as errata on
O'Reilly's site ensures I will see them, which isn't certain here),
since they give me the best chance to enhance the work's quality (it's
quite common for the author of some text to fail to see potential issues
with the text, which may trip some readers but appears to be perfect to
the author -- that's why two heads are better than one, and N>2 even
better than two!-).


So, again, thanks.

Alex
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Zope/Plone - Is it the right solution?

2006-02-21 Thread N.Davis
kbperry wrote:
   I attend a very large university (around
> 30,000 students), and the CS site will need to be updated by many
> people that don't have technical skills (like clerical staff).
That is exactly the kind of thing Plone is good at. A good match I would 
say.

Check out  http://plone.org/documentation/faq

The answers to your other questions can be found there.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Mutable numbers

2006-02-21 Thread Rocco Moretti
Steve Holden wrote:
> fraca7 wrote:
> 
>> The memory allocation for integers is optimized. 'Small' integers 
>> (between -5 and 100 IIRC) are allocated once and reused. The memory 
>> for larger integers is allocated once and reused whenever possible, so 
>> the malloc() overhead is negligible.
> 
> 
> The first bit's right, the second bit isn't:
> 
>  >>> id(12100)
> 4604168
>  >>> id(121*100)
> 4604204
>  >>>
> 

FWIW, I read "whenever possible" not as "whenever theoretically 
possible" but as "whenever the (comparatively simple) interpreter 
recognizes that reuse is possible".

 >>> a = 12100
 >>> b = 12100
 >>> a is b
False
 >>> def f():
a = 12100
b = 12100
c = 121*100
print a is b
print a is c


 >>> f()
True
False
 >>>

The interpreter, when compiling the function, can recognize that the two 
constants are identical, and makes them the same object. On the 
interactive interpreter, or after a computation, the interpreter doesn't 
bother to check to see if it already has the same value integer object. 
(It could, but the overhead would likely swamp any savings.)
-- 
http://mail.python.org/mailman/listinfo/python-list


  1   2   3   >