Re: Strange behavior in string interpolation of constants

[Ned Batchelder]

On 10/16/17 7:39 PM, מיקי מונין wrote:

Hello, I am working on an article on python string formatting. As a part of
the article I am researching the different forms of python string
formatting.

While researching string interpolation(i.e. the % operator) I noticed
something weird with string lengths.

Given two following two functions:

def simple_interpolation_constant_short_string():
 return "Hello %s" % "World!"

def simple_interpolation_constant_long_string():
 return "Hello %s. I am a very long string used for research" % "World!"


Lets look at the bytecode generated by them using the dis module

The first example produces the following bytecode:
   9   0 LOAD_CONST   3 ('Hello World!')
   2 RETURN_VALUE

It seems very normal, it appears that the python compiler optimizes the
constant and removes the need for the string interpolation

However the output of the second function caught my eye:

  12  0 LOAD_CONST   1 ('Hello %s. I am a very long
string used for research')
   2 LOAD_CONST2 ('World!')
   4 BINARY_MODULO
   6 RETURN_VALUE

This was not optimized by the compiler! Normal string interpolation was
used!

Based on some more testing it appears that for strings that would result in
more than 20 characters no optimization is done, as evident by these
examples:

def expected_result():
 return "abcdefghijklmnopqrs%s" % "t"

Bytecode:
  15  0 LOAD_CONST   3 ('abcdefghijklmnopqrst')
   2 RETURN_VALUE

def abnormal_result():
 return "abcdefghijklmnopqrst%s" % "u"

Bytecode:

  18  0 LOAD_CONST   1 ('abcdefghijklmnopqrst%s')
   2 LOAD_CONST 2 ('u')
   4 BINARY_MODULO
   6 RETURN_VALUE

I am using Python 3.6.3
I am curios as to why this happens. Can anyone shed further light on this
behaviour?


Optimizers have plenty of heuristics.  This one seems to avoid the 
constant folding if the string is larger than 20.  The code seems to 
bear this out 
(https://github.com/python/cpython/blob/master/Python/peephole.c#L305):


    } else if (size > 20) {
    Py_DECREF(newconst);
    return -1;
    }

As to why they chose 20?  There's no clue in the code, and I don't know.

--Ned.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Chris Angelico]

On Thu, Jan 14, 2016 at 3:03 PM, Michal Nalevanko
 wrote:
> I've just installed Python 3.5.1 on my computer and came across somewhat
> strange behavior.
>
> This is a snapshot of a short code that I wrote: https://goo.gl/izYbD0
>
> Quite simple, you might say. Obviously, the program should create a new
> text file, nothing else. But do you see the numbers it had added to the
> output?
>
> Is this behavior normal or am I overlooking something? Thank you very much
> for your help.

Hi Michal!

When you work with Python interactively, the interpreter prints out
the results of expressions automatically. This doesn't happen when you
write a script, but it's often helpful with simple tinkering. The
numbers getting printed out are the returned values from the .write()
method; they're a simple check saying how much stuff got written. You
can ignore them - your file has been created correctly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Peter Otten]

Steven D'Aprano wrote:

 On Mon, 02 Jun 2014 20:05:29 +0200, robertw89 wrote:
 
 I invoked the wrong bug.py :/ , works fine now (this happens to me when
 im a bit tired sometimes...).
 
 Clarity in naming is an excellent thing. If you have two files called
 bug.py, that's two too many.

In the case of the OP the code is likely to be thrown away once the bug is 
found. Putting all experiments into a single folder even with the overly 
generic name bug would have been good enough to avoid the problem.
 
 Imagine having fifty files called program.py. Which one is which? How
 do you know? Programs should be named by what they do (think of Word,
 which does word processing, or Photoshop, which does photo editing), or
 when that isn't practical, at least give them a unique and memorable name
 (Outlook, Excel). The same applies to files demonstrating bugs.
 
Outlook and Excel are only good names because these are popular 
applications. If I were to name some private scripts in that style and not 
use them for a few months -- I don't think I'd have a clue what excel.py is 
meant to do. 

I have a few find_dupes dedupe_xxx compare_xxx scripts lying around and no 
idea which is which. So a reasonably clear name is not sufficient if there 
are other scripts that perform similar tasks.

One approach that seems to be working so far is to combine several scripts 
into one using argparse subparsers. This results in more frequent usage 
which means I can get away with short meaningless names, and infrequent 
actions are just one

$ xx -h

away.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Steven D'Aprano]

On Tue, 03 Jun 2014 10:01:26 +0200, Peter Otten wrote:

 Steven D'Aprano wrote:
 
 On Mon, 02 Jun 2014 20:05:29 +0200, robertw89 wrote:
 
 I invoked the wrong bug.py :/ , works fine now (this happens to me
 when im a bit tired sometimes...).
 
 Clarity in naming is an excellent thing. If you have two files called
 bug.py, that's two too many.
 
 In the case of the OP the code is likely to be thrown away once the bug
 is found. Putting all experiments into a single folder even with the
 overly generic name bug would have been good enough to avoid the
 problem.

Depends on how many bugs the OP thinks he has found. (Hint: check on 
the python bug tracker.) And of course you can't have multiple files in 
the same directory unless they have different names, so a good naming 
system is still needed.

But as you point out later:

 Imagine having fifty files called program.py. Which one is which? How
 do you know? Programs should be named by what they do (think of Word,
 which does word processing, or Photoshop, which does photo editing), or
 when that isn't practical, at least give them a unique and memorable
 name (Outlook, Excel). The same applies to files demonstrating bugs.
  
 Outlook and Excel are only good names because these are popular
 applications. If I were to name some private scripts in that style and
 not use them for a few months -- I don't think I'd have a clue what
 excel.py is meant to do.

... a good naming scheme has to take into account how often you use it. 
Scripts that you *literally* throw away after use don't need to be named 
with a lot of care, just enough to keep the different versions distinct 
while you use them. More generic scripts that you keep around need a bit 
more care -- I must admit I have far too many scripts called make_avi 
for slightly different video-to-AVI conversion scripts. But at least 
they're not all called script.py.

Outlook and Excel are memorable names, but if you don't use them 
frequently, you may not associate the name with the application. In the 
Python world, we have memorable names like Psyco and Pyrex, but it took 
me *ages* to remember which one was which, because I didn't use them 
often enough to remember.

(Psycho is a JIT specialising compiler, now unmaintained and obsoleted by 
PyPy; Pyrex enables you to write C extensions using Python, also 
unmaintained, and obsoleted by Cython.)

[...]
 One approach that seems to be working so far is to combine several
 scripts into one using argparse subparsers. This results in more
 frequent usage which means I can get away with short meaningless names,
[...]


Very true, but the cost is added complexity in the script.


-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Mark Lawrence]

On 02/06/2014 17:35, robert...@googlemail.com wrote:

Hello folks,

I am not sure if it is only on my system the case that the code in
http://pastebin.com/WETvqMJN misbehaves in the stated way.
Can anybody reproduce it?

I thought it could be that the tabs/spaces do influence it, but it
doesn't care.

Thank you very much for your time.

Robert




Why couldn't you put your code inline, the same as you showed it here 
http://bugs.python.org/issue21631 ?


Here's the output I get.

Traceback (most recent call last):
  File C:\Users\Mark\MyPython\mytest.py, line 30, in module
print(_getStringOfElements(None))
  File C:\Users\Mark\MyPython\mytest.py, line 18, in _getStringOfElements
if iterationElement[type] == EnumSectionContentType.LABEL:
AttributeError: type object 'EnumSectionContentType' has no attribute 
'LABEL'


As R. David Murray has already said on the bug tracker mailing list, I 
think you need to be more careful with your testing.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Peter Otten]

robert...@googlemail.com wrote:

 Hello folks,
 
 I am not sure if it is only on my system the case that the code in
 http://pastebin.com/WETvqMJN misbehaves in the stated way.
 Can anybody reproduce it?
 
 I thought it could be that the tabs/spaces do influence it, but it doesn't
 care.
 
 Thank you very much for your time.
 
 Robert

I get what you expect:

$ cat bug.py
class EnumSectionContentType(object):
DATABYTE = 2
DATADOUBLEWORD = 3
DATAWORD = 4
#LABEL = 0

def _getStringOfElements(elements):
objectFileString = 

elements = [{'type': 2, 'data': {'elements': ['83H', '0FAH', '9AH', '27H', 
'81H', '49H', '0CEH', '11H']}}]

for iterationElement in elements:
objectFileString += INSIDE1 

if iterationElement[type] == EnumSectionContentType.LABEL:
objectFileString +=  iterationElement[data][labelname] + : + 
\n
elif iterationElement[type] == EnumSectionContentType.DATABYTE:
objectFileString += INSIDE + \n

if   iterationElement[type] == 
EnumSectionContentType.DATADOUBLEWORD:
objectFileString += objectFileString + dd 
elif iterationElement[type] == EnumSectionContentType.DATABYTE:
objectFileString += objectFileString + db 

return objectFileString

print(_getStringOfElements(None))
$ python3.4 bug.py
Traceback (most recent call last):
  File bug.py, line 27, in module
print(_getStringOfElements(None))
  File bug.py, line 15, in _getStringOfElements
if iterationElement[type] == EnumSectionContentType.LABEL:
AttributeError: type object 'EnumSectionContentType' has no attribute 'LABEL'

Are you absolutely sure you are invoking the right bug.py?
(I think on windows you can use type instead of cat to see 
the file contents).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[robertw89]

I invoked the wrong bug.py :/ , works fine now (this happens to me when im
a bit tired sometimes...).
Im unsure about the real bugreport, will investigate if I find some time
and motivation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Steven D'Aprano]

On Mon, 02 Jun 2014 20:05:29 +0200, robertw89 wrote:

 I invoked the wrong bug.py :/ , works fine now (this happens to me when
 im a bit tired sometimes...).

Clarity in naming is an excellent thing. If you have two files called 
bug.py, that's two too many.

Imagine having fifty files called program.py. Which one is which? How 
do you know? Programs should be named by what they do (think of Word, 
which does word processing, or Photoshop, which does photo editing), or 
when that isn't practical, at least give them a unique and memorable name 
(Outlook, Excel). The same applies to files demonstrating bugs.


-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Chris Angelico]

On Tue, Jun 3, 2014 at 10:22 AM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 On Mon, 02 Jun 2014 20:05:29 +0200, robertw89 wrote:

 I invoked the wrong bug.py :/ , works fine now (this happens to me when
 im a bit tired sometimes...).

 Clarity in naming is an excellent thing. If you have two files called
 bug.py, that's two too many.

 Imagine having fifty files called program.py. Which one is which? How
 do you know? Programs should be named by what they do (think of Word,
 which does word processing, or Photoshop, which does photo editing), or
 when that isn't practical, at least give them a unique and memorable name
 (Outlook, Excel). The same applies to files demonstrating bugs.

Heh. I agree, but I've been guilty of this exact problem myself.
Suppose you have a program that segfaults when given certain input, so
you take the input that crashes it, and progressively simplify it
until you have a minimal test-case. What do you call the file that
you're editing? What if you have a few different variants? I've had a
few called boom and boom2 and booom and so on, because
there's really nothing else to call it - if I knew what the cause of
the crash was, I wouldn't be naming the testcase files, I'd be fixing
the problem.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Igor Korot]

On Mon, Jun 2, 2014 at 5:30 PM, Chris Angelico ros...@gmail.com wrote:
 On Tue, Jun 3, 2014 at 10:22 AM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 On Mon, 02 Jun 2014 20:05:29 +0200, robertw89 wrote:

 I invoked the wrong bug.py :/ , works fine now (this happens to me when
 im a bit tired sometimes...).

 Clarity in naming is an excellent thing. If you have two files called
 bug.py, that's two too many.

 Imagine having fifty files called program.py. Which one is which? How
 do you know? Programs should be named by what they do (think of Word,
 which does word processing, or Photoshop, which does photo editing), or
 when that isn't practical, at least give them a unique and memorable name
 (Outlook, Excel). The same applies to files demonstrating bugs.

 Heh. I agree, but I've been guilty of this exact problem myself.
 Suppose you have a program that segfaults when given certain input, so
 you take the input that crashes it, and progressively simplify it
 until you have a minimal test-case. What do you call the file that
 you're editing? What if you have a few different variants? I've had a
 few called boom and boom2 and booom and so on, because
 there's really nothing else to call it - if I knew what the cause of
 the crash was, I wouldn't be naming the testcase files, I'd be fixing
 the problem.


test1, test2, test3,  testn.
At least it will be unique. ;-)

Thank you.


 ChrisA
 --
 https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Frank Millman]

ast nom...@invalid.com wrote in message 
news:530eda1d$0$2061$426a7...@news.free.fr...
 Hello

 box is a list of 3 integer items

 If I write:

box.sort()
if box == [1, 2, 3]:


 the program works as expected. But if I write:

if box.sort() == [1, 2, 3]:

 it doesn't work, the test always fails. Why ?


Try the following in the interpreter -

 box = [3, 2, 1]
 box.sort()
 box
[1, 2, 3]

 box = [3, 2, 1]
print(box.sort())
None
 box
[1, 2, 3]

box.sort() sorts box 'in situ', but does not return anything. That is why 
the second example prints None.

In your second example, you are comparing the return value of box.sort() 
with [1, 2, 3]. As the return value is None, they are unequal.

HTH

Frank Millman



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Eduardo A . Bustamante López]

On Thu, Feb 27, 2014 at 07:24:24AM +0100, ast wrote:
 Hello
 
 box is a list of 3 integer items
 
 If I write:
 
box.sort()
if box == [1, 2, 3]:
 
 
 the program works as expected. But if I write:
 
if box.sort() == [1, 2, 3]:
 
 it doesn't work, the test always fails. Why ?
 
 Thx
 -- 
 https://mail.python.org/mailman/listinfo/python-list

Because when you call the .sort() method on a list, it does the sort
in-place, instead of returning a sorted copy of the list. Check this:

 [2,1,3].sort()
 

The method does not return a value, that's why the direct comparison
fails.

What you might want is to use the sorted() method on the list, like
this:

 sorted([2,1,3])
[1, 2, 3]
 sorted([2,1,3]) == [1,2,3]
True

-- 
Eduardo Alan Bustamante López
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Lele Gaifax]

ast nom...@invalid.com writes:

 If I write:

box.sort()
if box == [1, 2, 3]:

 the program works as expected. But if I write:

if box.sort() == [1, 2, 3]:

 it doesn't work, the test always fails. Why ?

Because very often methods **dont't** return the object they are applied
(self that is).

This works though:

 box = [1,3,2]
 sorted(box) == [1,2,3]
True

hth, ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Marko Rauhamaa]

ast nom...@invalid.com:

 if I write:

if box.sort() == [1, 2, 3]:

 it doesn't work, the test always fails. Why ?

The list.sort() method returns None.

The builtin sorted() function returns a list:

   if sorted(box) == [1, 2, 3]:

would work.

Note that the list.sort() method is often preferred because it sorts the
list in place while the sorted() function must generate a fresh, sorted
list.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[ast]

Thanks for the very clear explanation
--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Gary Herron]

On 02/26/2014 10:24 PM, ast wrote:

Hello

box is a list of 3 integer items

If I write:

   box.sort()
   if box == [1, 2, 3]:


the program works as expected. But if I write:

   if box.sort() == [1, 2, 3]:


Most such questions can be answered by printing out the values in 
question and observing first hand what the value is.


So, print out box.sort() to see what it is.  You might be surprised.

Hint: box.sort() does indeed cause box to be sorted, and the sorted list 
is left in box, but the sorted list is not returned as a function value.




it doesn't work, the test always fails. Why ?

Thx


--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with sort()

[Larry Hudson]

On 02/26/2014 10:24 PM, ast wrote:

Hello

box is a list of 3 integer items

If I write:

box.sort()
if box == [1, 2, 3]:


the program works as expected. But if I write:

if box.sort() == [1, 2, 3]:

it doesn't work, the test always fails. Why ?

Thx


sort() sorts the sequence in place, but it _returns_ None.
Your second example becomes the equivalent of:

box.sort()
if None == [1, 2, 3]:

So although your box does become sorted, it is NOT what is compared in your if 
statement.

BTW, the sorted() function won't work here either.  It will return the sorted sequence, but it 
leaves the original unchanged.


 -=- Larry -=-

--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior for a 2D list

[Peter Otten]

Robrecht W. Uyttenhove wrote:

 Hello,
 
 I tried out the following code:
 y=[range(0,7),range(7,14),range(14,21),range(21,28),range(28,35)]
 y
 [[0, 1, 2, 3, 4, 5, 6],
  [7, 8, 9, 10, 11, 12, 13],
  [14, 15, 16, 17, 18, 19, 20],
  [21, 22, 23, 24, 25, 26, 27],
  [28, 29, 30, 31, 32, 33, 34]]
 
 y[1:5:2][::3]
 [[7, 8, 9, 10, 11, 12, 13]]
 
 I expected the 2D list:
 [[ 7, 10, 13],
  [21, 24, 27]]
 
 Any ideas?

It is not really a 2D list; rather a list of lists. You cannot see the two 
slices together, the slicing happens in two separate steps.

y[1:5:2] is a list containing two items (that happen to be lists)

 x = y[1:5:2]
 x
[[7, 8, 9, 10, 11, 12, 13], [21, 22, 23, 24, 25, 26, 27]]

x[::3] then operates on the outer list

 x[::3]
[[7, 8, 9, 10, 11, 12, 13]]

By the way, numpy offers the behaviour you expected:

 import numpy
 a = numpy.array(y)
 a
array([[ 0,  1,  2,  3,  4,  5,  6],
   [ 7,  8,  9, 10, 11, 12, 13],
   [14, 15, 16, 17, 18, 19, 20],
   [21, 22, 23, 24, 25, 26, 27],
   [28, 29, 30, 31, 32, 33, 34]])
 a[1:5:2,::3]
array([[ 7, 10, 13],
   [21, 24, 27]])

Note that both slices are passed in the same a[...] operation.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior for a 2D list

[John Gordon]

In mailman.793.1366317327.3114.python-l...@python.org Robrecht W. 
Uyttenhove ruyttenh...@gmail.com writes:

 I tried out the following code:
 y=[range(0,7),range(7,14),range(14,21),range(21,28),range(28,35)]
  y
 [[0, 1, 2, 3, 4, 5, 6],
  [7, 8, 9, 10, 11, 12, 13],
  [14, 15, 16, 17, 18, 19, 20],
  [21, 22, 23, 24, 25, 26, 27],
  [28, 29, 30, 31, 32, 33, 34]]

  y[1:5:2][::3]
 [[7, 8, 9, 10, 11, 12, 13]]

 I expected the 2D list:
 [[ 7, 10, 13],
  [21, 24, 27]]

 Any ideas?

y is just a list.  It happens to be a list of lists, but that doesn't make
it a 2D list.  It's an important distinction.

y[1:5:2] is the contents of y, starting at the second element and selecting
every second element after that:

[[7, 8, 9, 10, 11, 12, 13], [21, 22, 23, 24, 25, 26, 27]]

y[1:5:2][::3] is the contents of y[1:5:2], starting at the first element and
selecting every third element after that (and there are only two elements,
so it stops after the first one):

[[7, 8, 9, 10, 11, 12, 13]]

Why were you expecting the other result?

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, The Gashlycrumb Tinies

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior for a 2D list

[Wolfgang Maier]

Robrecht W. Uyttenhove ruyttenhove at gmail.com writes:

 
 Hello,
 I tried out the following
code:y=[range(0,7),range(7,14),range(14,21),range(21,28),range(28,35)]
  y[[0, 1, 2, 3, 4, 5, 6],  [7, 8, 9, 10, 11, 12, 13], 
   [14, 15, 16, 17, 18, 19, 20],  [21, 22, 23, 24, 25, 26, 27],  [28, 
   29, 30, 31, 32, 33, 34]]
  y[1:5:2][::3]
 [[7, 8, 9, 10, 11, 12, 13]]
 I expected the 2D list:[[ 7, 10, 13],
  [21, 24, 27]]
 
 Any ideas?
 
 Thanks,
 Rob
 PS: I used Python 2.7.3
 

The explanation is rather simple, just break up your complex slicing into
its parts:
y[1:5:2] = [[7, 8, 9, 10, 11, 12, 13],[21, 22, 23, 24, 25, 26, 27]]
and [::3] is asking for the first,4th,7th,... element from this list.
Obviously, only the first one's existing, so [7, 8, 9, 10, 11, 12, 13]

What you expected is kind of vertical slicing through the rows. I don't
think you can achieve this with slicing alone in standard Python, but it's
possible with numpy arrays.
In Python you will have to combine slicing with a comprehension, like this:
[x[::3] for x in y[1:5:2]]

Best,
Wolfgang

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior for a 2D list

[Wim R. Cardoen]

Thank you all for your replies.
I had the matrix concept in mind such as
explained in the numpy example.

Rob



On Thu, Apr 18, 2013 at 3:19 PM, Wolfgang Maier 
wolfgang.ma...@biologie.uni-freiburg.de wrote:

 Robrecht W. Uyttenhove ruyttenhove at gmail.com writes:

 
  Hello,
  I tried out the following
 code:y=[range(0,7),range(7,14),range(14,21),range(21,28),range(28,35)]
   y[[0, 1, 2, 3, 4, 5, 6],  [7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],  [21, 22, 23, 24, 25, 26, 27],  [28,
 
29, 30, 31, 32, 33, 34]]
   y[1:5:2][::3]
  [[7, 8, 9, 10, 11, 12, 13]]
  I expected the 2D list:[[ 7, 10, 13],
   [21, 24, 27]]
 
  Any ideas?
 
  Thanks,
  Rob
  PS: I used Python 2.7.3
 

 The explanation is rather simple, just break up your complex slicing into
 its parts:
 y[1:5:2] = [[7, 8, 9, 10, 11, 12, 13],[21, 22, 23, 24, 25, 26, 27]]
 and [::3] is asking for the first,4th,7th,... element from this list.
 Obviously, only the first one's existing, so [7, 8, 9, 10, 11, 12, 13]

 What you expected is kind of vertical slicing through the rows. I don't
 think you can achieve this with slicing alone in standard Python, but it's
 possible with numpy arrays.
 In Python you will have to combine slicing with a comprehension, like this:
 [x[::3] for x in y[1:5:2]]

 Best,
 Wolfgang

 --
 http://mail.python.org/mailman/listinfo/python-list




-- 
---
Wim R. Cardoen, PhD
Staff Scientist,
Center for High Performance Computing
University of Utah
(801)971-4184

*μὴ μου τοὺς κύκλους τάραττε! (Ἀρχιμήδης)*
---
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Virgil Stokes]

On 15-Aug-2012 02:19, Steven D'Aprano wrote:

On Tue, 14 Aug 2012 21:40:10 +0200, Virgil Stokes wrote:


You might find the following useful:

def testFunc(startingList):
  xOnlyList = []; j = -1
  for xl in startingList:
  if (xl[0] == 'x'):

That's going to fail in the starting list contains an empty string. Use
xl.startswith('x') instead.
Yes, but this was by design (tacitly assumed that startingList was both a list 
and non-empty).




  xOnlyList.append(xl)
  else:
  j += 1
  startingList[j] = xl

Very cunning, but I have to say that your algorithm fails the is this
obviously correct without needing to study it? test. Sometimes that is
unavoidable, but for something like this, there are simpler ways to solve
the same problem.

Sorry, but I do not sure what you mean here.




  if j == -1:
  startingList = []
  else:
  del startingList[j:-1]
  return(xOnlyList)



And here is another version using list comprehension that I prefer
def testFunc2(startingList):
  return([x for x in startingList if x[0] == 'x'], [x for x in
startingList if x[0] != 'x'])

This walks over the starting list twice, doing essentially the same thing
both times. It also fails to meet the stated requirement that
startingList is modified in place, by returning a new list instead.
This can meet the requirement that startingList is modified in place via the 
call to this function (see the attached code).

Here's an example of what I mean:

py mylist = mylist2 = ['a', 'x', 'b', 'xx', 'cx']  # two names for one
list
py result, mylist = testFunc2(mylist)
py mylist
['a', 'b', 'cx']
py mylist2  # should be same as mylist
['a', 'x', 'b', 'xx', 'cx']

Yes, I had a typo in my original posting --- sorry about that!


Here is the obvious algorithm for extracting and removing words starting
with 'x'. It walks the starting list only once, and modifies it in place.
The only trick needed is list slice assignment at the end.

def extract_x_words(words):
 words_with_x = []
 words_without_x = []
 for word in words:
 if word.startswith('x'):
 words_with_x.append(word)
 else:
 words_without_x.append(word)
 words[:] = words_without_x  # slice assignment
 return words_with_x

Suppose words was not a list --- you have tacitly assumed that words is a list.


The only downside of this is that if the list of words is so enormous
that you can fit it in memory *once* but not *twice*, this may fail. But
the same applies to the list comprehension solution.
But, this is not the only downside if speed is important --- it is slower than 
the list comprehension method (see results that follows).


Here is a summary of three algorithms (algorithm-1, algorithm-2, algorithm-2A) 
that I tested (see attached code). Note, algorithm-2A was obtained by removing 
the slice assignment in the above code and modifying the return as follows


def extract_x_words(words):
words_with_x = []
words_without_x = []
for word in words:
if word.startswith('x'):
words_with_x.append(word)
else:
words_without_x.append(word)
#words[:] = words_without_x  # slice assignment
return words_with_x, words_without_x

Of course, one needs to modify the call for in-place update of startingList as 
follows:


   xOnlyList,startingList = extract_x_words(startingList)

Here is a summary of my timing results obtained for 3 different algorithms for 
lists with 100,000 strings of length 4 in each list:


Method
average (sd) time in seconds
algorithm-1 (list comprehension)
0.11630 (0.0014)
algorithm-2 (S. D'Aprano)
0.17594 (0.0014)
algorithm-2A (modified S. D'Aprano)
0.18217 (0.0023)


These values  were obtained from 100 independent runs (MC simulations) on lists 
that contain 100,000 strings. Approximately 50% of these strings contained a 
leading 'x'. Note, that the results show that algorithm-2 (suggested by S. 
D'Aprano) is approximately 51% slower than algorithm-1 (list comprehensions) and 
algorithm-2A (simple modification of algorithm-2) is approximately 57% slower 
than algorithm-1. Why is algorithm-2A slower than algorithm-2?


I would be interested in seeing code that is faster than algorithm-1 --- any 
suggestions are welcomed.  And of course, if there are any errors in my attached 
code please inform me of them and I will try to correct them as soon as 
possible. Note, some of the code is actually irrelevant for the original 
Strange behavior post.


Have a good day!

'''
  Purpose: Time three different algorithms for the same task
  
  Author: V. Stokes (v...@it.uu.se, 2012-08-16)
  Refs:
   python-list@python.org list
* Strange behavior, 14-Aug-2012 17:38, light1qu...@gmail.com
* Re: Strange behavior, 14-Aug-2012 21:40, Stokes, Virgil
* Re: Strange behavior, 15-Aug-2012 02:19, Steven D'Aprano
  
  Note:
   1. The mean and standard deviation

Re: Strange behavior

[Peter Otten]

Virgil Stokes wrote:

 def testFunc(startingList):
xOnlyList = []; j = -1
for xl in startingList:
if (xl[0] == 'x'):
 That's going to fail in the starting list contains an empty string. Use
 xl.startswith('x') instead.
 Yes, but this was by design (tacitly assumed that startingList was both a
 list and non-empty).

You missunderstood it will fail if the list contains an empty string, not if 
the list itself is empty: 

 words = [alpha, , xgamma]
 [word for word in words if word[0] == x]
Traceback (most recent call last):
  File stdin, line 1, in module
IndexError: string index out of range

The startswith() version:

 [word for word in words if word.startswith(x)]
['xgamma']

Also possible:

 [word for word in words if word[:1] == x]
['xgamma']

 def testFunc1(startingList): 
  ''' 
Algorithm-1 
Note: 
  One should check for an empty startingList before 
  calling testFunc1 -- If this possibility exists! 
  ''' 
  return([x for x in startingList if x[0] == 'x'], 
 [x for x in startingList if x[0] != 'x']) 
  
 
 I would be interested in seeing code that is faster than algorithm-1

In pure Python? Perhaps the messy variant:

def test_func(words):
nox = []
append = nox.append
withx = [x for x in words if x[0] == 'x' or append(x)]
return withx, nox


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Virgil Stokes]

On 16-Aug-2012 15:02, Peter Otten wrote:

Virgil Stokes wrote:


def testFunc(startingList):
xOnlyList = []; j = -1
for xl in startingList:
if (xl[0] == 'x'):

That's going to fail in the starting list contains an empty string. Use
xl.startswith('x') instead.

Yes, but this was by design (tacitly assumed that startingList was both a
list and non-empty).

You missunderstood it will fail if the list contains an empty string, not if
the list itself is empty:


words = [alpha, , xgamma]
[word for word in words if word[0] == x]

Traceback (most recent call last):
   File stdin, line 1, in module
IndexError: string index out of range

The startswith() version:


[word for word in words if word.startswith(x)]

['xgamma']

Also possible:


[word for word in words if word[:1] == x]

['xgamma']


def testFunc1(startingList):
  '''
Algorithm-1
Note:
  One should check for an empty startingList before
  calling testFunc1 -- If this possibility exists!
  '''
  return([x for x in startingList if x[0] == 'x'],
 [x for x in startingList if x[0] != 'x'])
  


I would be interested in seeing code that is faster than algorithm-1

In pure Python? Perhaps the messy variant:

def test_func(words):
 nox = []
 append = nox.append
 withx = [x for x in words if x[0] == 'x' or append(x)]
 return withx, nox



Very nice Peter,

Here are the new results for timing with your method added (algorithm-3).

Method
average (sd) time in seconds
algorithm-1 (list comprehension)
0.11774 (0.002968)
algorithm-2 (S. D'Aprano)
0.17573 (0.003385)
algorithm-2A (modified S. D'Aprano)
0.18116 (0.003081)
algorithm-3 (improved list comprehension)
0.06639 (0.001728)


Algorithm-3 is 43% faster than algorithm-1.  Again, the code used to obtain 
these results is attached.


Thanks Peter for your contribution
'''
  Purpose: Time four different algorithms for the same task
  
  Author: V. Stokes (v...@it.uu.se, 2012-08-16 (15:46), 2012-08-16)
  Refs:
   python-list@python.org list
* Strange behavior, 14-Aug-2012 17:38, light1qu...@gmail.com
* Re: Strange behavior, 14-Aug-2012 21:40, Stokes, Virgil
* Re: Strange behavior, 15-Aug-2012 02:19, Steven D'Aprano
* Re: Strange behavior, 16-Aug-2012 15:02, Peter Otten
  
  Notes:
   1. The mean and standard deviation over the runs (MC simulations)
  are estimated using recursive equations.
   2. A seed (syd) is used with the RNG for repeatability. Each run is 
  started with a new seed to force the generation of independent 
  random sequences.
   3. Warning! No checks are made on the parameters passed to
  the functions (this was by design).
   4. No effort has been made to make this code elegant. My focus was
  to make the code clear and easy to understand.
   5. This was executed on a Windows Vista 32-bit platform with Python 2.6.6
   Processor: Intel(R) core(TM)2 Duo CPU E8500@3.16GHz 3.17GHz
   6. The estimated time to completion is displayed after each run.
  
'''
import random as random
import math as math
from time import clock # clock gives good resolution on MS Windows

def testFunc1(startingList): 
'''
  Algorithm-1
  Note: 
One should check for an empty startingList before 
calling testFunc1 -- If this possibility exists!
'''
return([x for x in startingList if x[0] == 'x'], 
   [x for x in startingList if x[0] != 'x'])

def testFunc2(words):
'''
  Algorithm-2
'''
words_with_x = []
words_without_x = []
for word in words:
if word.startswith('x'):
words_with_x.append(word)
else:
words_without_x.append(word)
words[:] = words_without_x  # slice assignment
return words_with_x

def testFunc2A(words):
'''
  Algorithm-2A
'''
words_with_x = []
words_without_x = []
for word in words:
if word.startswith('x'):
words_with_x.append(word)
else:
words_without_x.append(word)
#words[:] = words_without_x  # slice assignment
return words_with_x, words_without_x

def testFunc3(words):
'''
  Algorithm-3 (from: Peter Otten) 
'''
nox = []
append = nox.append
withx = [x for x in words if x[0] == 'x' or append(x)]
return withx, nox


def genStrList(NChar,NStrng,Alph,leadChr):
'''
 Purpose: Generate a list of NStrng elements with each element a string
  of length NChar and constrained such that approx. 50% of the 
  strings will begin with the character (symbol) leadChr
   Inputs: 
   NChar -- number of characters in each element
  NStrng -- number of elements in list (strgList)
Alph -- list of characters to be used (must contain the leadChr)
 leadChr -- leading character for strings to be generated from Alph
   Otputs:
strgList -- list with NString strings of NChar characters

Re: Strange behavior

[Steven D'Aprano]

On Thu, 16 Aug 2012 13:18:59 +0200, Virgil Stokes wrote:

 On 15-Aug-2012 02:19, Steven D'Aprano wrote:
 On Tue, 14 Aug 2012 21:40:10 +0200, Virgil Stokes wrote:

 You might find the following useful:

 def testFunc(startingList):
   xOnlyList = []; j = -1
   for xl in startingList:
   if (xl[0] == 'x'):
 That's going to fail in the starting list contains an empty string. Use
 xl.startswith('x') instead.

 Yes, but this was by design (tacitly assumed that startingList was both
 a list and non-empty).

As Peter already pointed out, I said it would fail if the list contains 
an empty string, not if the list was empty.


   xOnlyList.append(xl)
   else:
   j += 1
   startingList[j] = xl

 Very cunning, but I have to say that your algorithm fails the is this
 obviously correct without needing to study it? test. Sometimes that is
 unavoidable, but for something like this, there are simpler ways to
 solve the same problem.

 Sorry, but I do not sure what you mean here.

In a perfect world, you should be able to look at a piece of code, read 
it once, and see whether or not it is correct. That is what I mean by 
obviously correct. For example, if I have a function that takes an 
argument, doubles it, and prints the result:

def f1(x):
print(2*x)


that is obviously correct. Whereas this is not:

def f2(x):
y = (x + 5)**2 - (x + 4)**2
sys.stdout.write(str(y - 9) + '\n')


because you have to study it to see whether or not it works correctly.

Not all programs are simple enough to be obviously correct. Sometimes you 
have no choice but to write something which requires cleverness to get 
the right result. But this is not one of those cases. You should almost 
always prefer simple code over clever code, because the greatest expense 
in programming (time, effort and money) is to make code correct.

Most code does not need to be fast. But all code needs to be correct.


[...]
 This can meet the requirement that startingList is modified in place via
 the call to this function (see the attached code).

Good grief! See, that's exactly the sort of thing I'm talking about. 
Without *detailed* study of your attached code, how can I possibly know 
what it does or whether it does it correctly?

Your timing code calculates the mean using a recursive algorithm. Why 
don't you calculate the mean the standard way: add the numbers and divide 
by the total? What benefit do you gain from a more complicated algorithm 
when a simple one will do the job just as well?

You have spent a lot of effort creating a complicated, non-obvious piece 
of timing code, with different random seeds for each run, and complicated 
ways of calculating timing statistics... but unfortunately the most 
important part of any timing test, the actually *timing*, is not done 
correctly. Consequently, your code is not correct.

With an average time of a fraction of a second, none of those timing 
results are trustworthy, because they are vulnerable to interference from 
other processes, the operating system, and other random noise. You spend 
a lot of time processing the timing results, but it is Garbage In, 
Garbage Out -- the results are not trustworthy, and if they are correct, 
it is only by accident.

Later in your post, you run some tests, and are surprised by the result:

 Why is algorithm-2A slower than algorithm-2?

It isn't slower. It is physically impossible, since 2A does *less* work 
than 2. This demonstrates that you are actually taking a noisy 
measurement: the values you get have random noise, and you don't make any 
effort to minimise that noise. Hence GIGO.

The right way to test small code snippets is with the timeit module. It 
is carefully written to overcome as much random noise as possible. But 
even there, the authors of the timeit module are very clear that you 
should not try to calculate means, let alone higher order statistics like 
standard deviation. The only statistic which is trustworthy is to run as 
many trials as you can afford, and select the minimum value.

So here is my timing code, which is much shorter and simpler and doesn't 
try to do too much. You do need to understand the timeit.Timer class:

timeit.Timer creates a timer object; timer.repeat does the actual timing. 
The specific arguments to them are not vital to understand, but you can 
read the documentation if you wish to find out what they mean.

First, I define the two functions. I compare similar functions that have 
the same effect. Neither modifies the input argument in place. Copy and 
paste the following block into an interactive interpreter:

# Start block

def f1(startingList):
return ([x for x in startingList if x[0] == 'x'],
[x for x in startingList if x[0] != 'x'])

# Note that the above function is INCORRECT, it will fail if a string is
# empty; nevertheless I will use it for timing purposes anyway.


def f2(startingList):
words_without_x = []
words_with_x 

Re: Strange behavior

[Virgil Stokes]

On 16-Aug-2012 19:40, Steven D'Aprano wrote:

On Thu, 16 Aug 2012 13:18:59 +0200, Virgil Stokes wrote:


On 15-Aug-2012 02:19, Steven D'Aprano wrote:

On Tue, 14 Aug 2012 21:40:10 +0200, Virgil Stokes wrote:


You might find the following useful:

def testFunc(startingList):
   xOnlyList = []; j = -1
   for xl in startingList:
   if (xl[0] == 'x'):

That's going to fail in the starting list contains an empty string. Use
xl.startswith('x') instead.

Yes, but this was by design (tacitly assumed that startingList was both
a list and non-empty).

As Peter already pointed out, I said it would fail if the list contains
an empty string, not if the list was empty.



   xOnlyList.append(xl)
   else:
   j += 1
   startingList[j] = xl

Very cunning, but I have to say that your algorithm fails the is this
obviously correct without needing to study it? test. Sometimes that is
unavoidable, but for something like this, there are simpler ways to
solve the same problem.

Sorry, but I do not sure what you mean here.

In a perfect world, you should be able to look at a piece of code, read
it once, and see whether or not it is correct. That is what I mean by
obviously correct. For example, if I have a function that takes an
argument, doubles it, and prints the result:

def f1(x):
 print(2*x)


that is obviously correct. Whereas this is not:

def f2(x):
 y = (x + 5)**2 - (x + 4)**2
 sys.stdout.write(str(y - 9) + '\n')


because you have to study it to see whether or not it works correctly.

Not all programs are simple enough to be obviously correct. Sometimes you
have no choice but to write something which requires cleverness to get
the right result. But this is not one of those cases. You should almost
always prefer simple code over clever code, because the greatest expense
in programming (time, effort and money) is to make code correct.

Most code does not need to be fast. But all code needs to be correct.


[...]

This can meet the requirement that startingList is modified in place via
the call to this function (see the attached code).

Good grief! See, that's exactly the sort of thing I'm talking about.
Without *detailed* study of your attached code, how can I possibly know
what it does or whether it does it correctly?
Very strange question? Perhaps, you should work on understanding code that you 
have not written, or maybe you should learn more about Python, or   I really 
don't know how to help you with this question.


Your timing code calculates the mean using a recursive algorithm. Why
don't you calculate the mean the standard way: add the numbers and divide
by the total? What benefit do you gain from a more complicated algorithm
when a simple one will do the job just as well?
A lot of questions that suggest you have not made much of an effort to answer 
them yourself. Try a little numerical analysis/research before asking such 
questions (This is how you often respond to others on this list who would like 
help --- try apply your advice to other to yourself. I will give you a start:


* Knuth, D. E. (1998) /The Art of Computer Programming vol. 2: Seminumerical 
Algorithms/ /(3rd edition)/. Addison-Wesley, Boston.

[hint: study p. 232]

* Welford, B. P. (1962) Note on a method for calculating sums of squares and 
products. T/echnometrics/ *4*(3).

[hint: pp. 419-420]


You have spent a lot of effort creating a complicated, non-obvious piece
of timing code, with different random seeds for each run, and complicated
ways of calculating timing statistics... but unfortunately the most
important part of any timing test, the actually *timing*, is not done
correctly. Consequently, your code is not correct.
How do you know how much effort I used? Code non-obvious and complicated for 
you does not mean that this is also true for others. Could you please be more 
specific --- saying code is not correct without providing details is not very 
useful. I did say in an earlier email in reference to my code if there are any 
errors in my attached code please inform me of them and I will try to correct 
them as soon as possible.


With an average time of a fraction of a second, none of those timing
results are trustworthy, because they are vulnerable to interference from
other processes, the operating system, and other random noise.
Please explain what you mean by the timing results not being trustworthy and how 
this vulnerability works --- in detail please.

  You spend
a lot of time processing the timing results, but it is Garbage In,
Garbage Out -- the results are not trustworthy, and if they are correct,
it is only by accident.
Fantastic --- a lot of criticism but little that can be helpful. What 
specifically is the Garbage In?


Later in your post, you run some tests, and are surprised by the result:


Why is algorithm-2A slower than algorithm-2?

It isn't slower. It is physically impossible, since 2A does *less* work
than 2.
Please provide results 

Re: Strange behavior

[Alain Ketterlin]

Chris Angelico ros...@gmail.com writes:

 Other people have explained the problem with your code. I'll take this
 example as a way of introducing you to one of Python's handy features
 - it's an idea borrowed from functional languages, and is extremely
 handy. It's called the list comprehension, and can be looked up in
 the docs under that name,

 def testFunc(startingList):
 xOnlyList = [strng for strng in startingList if strng[0] == 'x']
 startingList = [strng for strng in startingList if strng[0] != 'x']
 print(xOnlyList)
 print(startingList)

 It's a compact notation for building a list from another list. (Note
 that I changed str to strng to avoid shadowing the built-in name
 str, as others suggested.)

Fully agree with you: list comprehension is, imo, the most useful
program construct ever. Extremely useful.

But not when it makes the program traverse twice the same list, where
one traversal is enough.

-- Alain.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Alain Ketterlin]

light1qu...@gmail.com writes:

 I got my answer by reading your posts and referring to:
 http://docs.python.org/reference/compound_stmts.html#the-for-statement
 (particularly the shaded grey box)

Not that the problem is not specific to python (if you erase the current
element when traversing a STL list in C++ you'll get a crash as well).

 I guess I should have (obviously) looked at the doc's before posting
 here; but im a noob.

Python has several surprising features. I think it is a good idea to
take some time to read the language reference, from cover to cover
(before or after the various tutorials, depending on your background).

-- Alain.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Alain Ketterlin]

light1qu...@gmail.com writes:

 However if you run the code you will notice only one of the strings
 beginning with 'x' is removed from the startingList.


 def testFunc(startingList):
   xOnlyList = [];
   for str in startingList:
   if (str[0] == 'x'):
   print str;
   xOnlyList.append(str)
   startingList.remove(str) #this seems to be the problem
   print xOnlyList;
   print startingList
 testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews'])

 #Thanks for your help!

Try with ['xasd', 'sefwr', 'xjkl', 'dfsews'] and you'll understand what
happens. Also, have a look at:

http://docs.python.org/reference/compound_stmts.html#the-for-statement

You can't modify the list you're iterating on, better use another list
to collect the result.

-- Alain.

P/S: str is a builtin, you'd better avoid assigning to it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Terry Reedy]

On 8/14/2012 11:59 AM, Alain Ketterlin wrote:

light1qu...@gmail.com writes:


However if you run the code you will notice only one of the strings
beginning with 'x' is removed from the startingList.




def testFunc(startingList):
xOnlyList = [];
for str in startingList:
if (str[0] == 'x'):
print str;
xOnlyList.append(str)
startingList.remove(str) #this seems to be the problem
print xOnlyList;
print startingList
testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews'])

#Thanks for your help!


Try with ['xasd', 'sefwr', 'xjkl', 'dfsews'] and you'll understand what
happens. Also, have a look at:

http://docs.python.org/reference/compound_stmts.html#the-for-statement

You can't modify the list you're iterating on,


Except he obviously did ;-).
(Modifying set or dict raises SomeError.)

Indeed, people routine *replace* items while iterating.

def squarelist(lis):
for i, n in enumerate(lis):
lis[i] = n*n
return lis

print(squarelist([0,1,2,3,4,5]))
# [0, 1, 4, 9, 16, 25]

Removals can be handled by iterating in reverse. This works even with 
duplicates because if the item removed is not the one tested, the one 
tested gets retested.


def removeodd(lis):
for n in reversed(lis):
if n % 2:
lis.remove(n)
print(n, lis)

ll = [0,1, 5, 5, 4, 5]
removeodd(ll)

5 [0, 1, 5, 4, 5]
5 [0, 1, 4, 5]
5 [0, 1, 4]
4 [0, 1, 4]
1 [0, 4]
0 [0, 4]


better use another list to collect the result.


If there are very many removals, a new list will be faster, even if one 
needs to copy the new list back into the original, as k removals from 
len n list is O(k*n) versus O(n) for new list and copy.



P/S: str is a builtin, you'd better avoid assigning to it.


Agreed. People have actually posted code doing something like

...
list = [1,2,3]
...
z = list(x)
...
and wondered and asked why it does not work.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Virgil Stokes]

On 2012-08-14 17:38, light1qu...@gmail.com wrote:

Hi, I am migrating from PHP to Python and I am slightly confused.

I am making a function that takes a startingList, finds all the strings in the 
list that begin with 'x', removes those strings and puts them into a xOnlyList.

However if you run the code you will notice only one of the strings beginning 
with 'x' is removed from the startingList.
If I comment out 'startingList.remove(str);' the code runs with both strings 
beginning with 'x' being put in the xOnlyList.
Using the print statement I noticed that the second string that begins with 'x' 
isn't even identified by the function. Why does this happen?

def testFunc(startingList):
xOnlyList = [];
for str in startingList:
if (str[0] == 'x'):
print str;
xOnlyList.append(str)
startingList.remove(str) #this seems to be the problem
print xOnlyList;
print startingList
testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews'])

#Thanks for your help!


You might find the following useful:

def testFunc(startingList):
xOnlyList = []; j = -1
for xl in startingList:
if (xl[0] == 'x'):
xOnlyList.append(xl)
else:
j += 1
startingList[j] = xl
if j == -1:
startingList = []
else:
del startingList[j:-1]

return(xOnlyList)


testList1 = ['xasd', 'xjkl', 'sefwr', 'dfsews']
testList2 = ['xasd', 'xjkl', 'xsefwr', 'xdfsews']
testList3 = ['xasd', 'jkl', 'sefwr', 'dfsews']
testList4 = ['asd', 'jkl', 'sefwr', 'dfsews']

xOnlyList = testFunc(testList1)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList1
xOnlyList = testFunc(testList2)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList2
xOnlyList = testFunc(testList3)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList3
xOnlyList = testFunc(testList4)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList4

And here is another version using list comprehension that I prefer

testList1 = ['xasd', 'xjkl', 'sefwr', 'dfsews']
testList2 = ['xasd', 'xjkl', 'xsefwr', 'xdfsews']
testList3 = ['xasd', 'jkl', 'sefwr', 'dfsews']
testList4 = ['asd', 'jkl', 'sefwr', 'dfsews']

def testFunc2(startingList):
return([x for x in startingList if x[0] == 'x'], [x for x in
startingList if x[0] != 'x'])

xOnlyList,testList = testFunc2(testList1)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList2)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList3)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList4)
print xOnlyList
print testList

--
http://mail.python.org/mailman/listinfo/python-list

Fwd: Re: Strange behavior

[Virgil Stokes]

 Original Message 
Subject:Re: Strange behavior
Date:   Tue, 14 Aug 2012 21:32:16 +0200
From:   Virgil Stokes v...@it.uu.se
To: light1qu...@gmail.com



On 2012-08-14 17:38, light1qu...@gmail.com wrote:

Hi, I am migrating from PHP to Python and I am slightly confused.

I am making a function that takes a startingList, finds all the strings in the 
list that begin with 'x', removes those strings and puts them into a xOnlyList.

However if you run the code you will notice only one of the strings beginning 
with 'x' is removed from the startingList.
If I comment out 'startingList.remove(str);' the code runs with both strings 
beginning with 'x' being put in the xOnlyList.
Using the print statement I noticed that the second string that begins with 'x' 
isn't even identified by the function. Why does this happen?

def testFunc(startingList):
xOnlyList = [];
for str in startingList:
if (str[0] == 'x'):
print str;
xOnlyList.append(str)
startingList.remove(str) #this seems to be the problem
print xOnlyList;
print startingList
testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews'])

#Thanks for your help!

You might find the following useful:

def testFunc(startingList):
xOnlyList = []; j = -1
for xl in startingList:
if (xl[0] == 'x'):
xOnlyList.append(xl)
else:
j += 1
startingList[j] = xl
if j == -1:
startingList = []
else:
del startingList[j:-1]

return(xOnlyList)


testList1 = ['xasd', 'xjkl', 'sefwr', 'dfsews']
testList2 = ['xasd', 'xjkl', 'xsefwr', 'xdfsews']
testList3 = ['xasd', 'jkl', 'sefwr', 'dfsews']
testList4 = ['asd', 'jkl', 'sefwr', 'dfsews']

xOnlyList = testFunc(testList1)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList1
xOnlyList = testFunc(testList2)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList2
xOnlyList = testFunc(testList3)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList3
xOnlyList = testFunc(testList4)
print 'xOnlyList = ',xOnlyList
print 'testList = ',testList4

And here is another version using list comprehension that I prefer

testList1 = ['xasd', 'xjkl', 'sefwr', 'dfsews']
testList2 = ['xasd', 'xjkl', 'xsefwr', 'xdfsews']
testList3 = ['xasd', 'jkl', 'sefwr', 'dfsews']
testList4 = ['asd', 'jkl', 'sefwr', 'dfsews']

def testFunc2(startingList):
return([x for x in startingList if x[0] == 'x'], [x for x in
startingList if x[0] != 'x'])

xOnlyList,testList = testFunc2(testList1)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList2)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList3)
print xOnlyList
print testList
xOnlyList,testList = testFunc2(testList4)
print xOnlyList
print testList




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Chris Angelico]

On Wed, Aug 15, 2012 at 1:38 AM,  light1qu...@gmail.com wrote:
 def testFunc(startingList):
 xOnlyList = [];
 for str in startingList:
 if (str[0] == 'x'):
 print str;
 xOnlyList.append(str)
 startingList.remove(str) #this seems to be the problem
 print xOnlyList;
 print startingList
 testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews'])

Other people have explained the problem with your code. I'll take this
example as a way of introducing you to one of Python's handy features
- it's an idea borrowed from functional languages, and is extremely
handy. It's called the list comprehension, and can be looked up in
the docs under that name,

def testFunc(startingList):
xOnlyList = [strng for strng in startingList if strng[0] == 'x']
startingList = [strng for strng in startingList if strng[0] != 'x']
print(xOnlyList)
print(startingList)

It's a compact notation for building a list from another list. (Note
that I changed str to strng to avoid shadowing the built-in name
str, as others suggested.)

(Unrelated side point: Putting parentheses around the print statements
makes them compatible with Python 3, in which 'print' is a function.
Unless something's binding you to Python 2, consider working with the
current version - Python 2 won't get any more features added to it any
more.)

Python's an awesome language. You may have to get your head around a
few new concepts as you shift thinking from PHP's, but it's well worth
while.

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior

[Steven D'Aprano]

On Tue, 14 Aug 2012 21:40:10 +0200, Virgil Stokes wrote:

 You might find the following useful:
 
 def testFunc(startingList):
  xOnlyList = []; j = -1
  for xl in startingList:
  if (xl[0] == 'x'):

That's going to fail in the starting list contains an empty string. Use 
xl.startswith('x') instead.


  xOnlyList.append(xl)
  else:
  j += 1
  startingList[j] = xl

Very cunning, but I have to say that your algorithm fails the is this 
obviously correct without needing to study it? test. Sometimes that is 
unavoidable, but for something like this, there are simpler ways to solve 
the same problem.


  if j == -1:
  startingList = []
  else:
  del startingList[j:-1]
  return(xOnlyList)


 And here is another version using list comprehension that I prefer

 def testFunc2(startingList):
  return([x for x in startingList if x[0] == 'x'], [x for x in
 startingList if x[0] != 'x'])

This walks over the starting list twice, doing essentially the same thing 
both times. It also fails to meet the stated requirement that 
startingList is modified in place, by returning a new list instead. 
Here's an example of what I mean:

py mylist = mylist2 = ['a', 'x', 'b', 'xx', 'cx']  # two names for one 
list
py result, mylist = testFunc2(mylist)
py mylist
['a', 'b', 'cx']
py mylist2  # should be same as mylist
['a', 'x', 'b', 'xx', 'cx']

Here is the obvious algorithm for extracting and removing words starting 
with 'x'. It walks the starting list only once, and modifies it in place. 
The only trick needed is list slice assignment at the end.

def extract_x_words(words):
words_with_x = []
words_without_x = []
for word in words:
if word.startswith('x'):
words_with_x.append(word)
else:
words_without_x.append(word)
words[:] = words_without_x  # slice assignment
return words_with_x


The only downside of this is that if the list of words is so enormous 
that you can fit it in memory *once* but not *twice*, this may fail. But 
the same applies to the list comprehension solution.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior on Python 3 Windows Command Line

[Arnaud Delobelle]

On 13 February 2012 19:50, waylan way...@gmail.com wrote:
 When I try running any Python Script on the command line with Python
 3.2 I get this weird behavior. The cursor dances around the command
 line window and nothing ever happens. Pressing Ctr+C does nothing.
 When I close the window (mouse click on X in top right corner), an
 error dialog appears asking me to force it to close.

I'm not a Windows user, so I can't be of assistance but it may help
others if you explained how you installed Python 3.2 on your computer.
 Also have you tried reinstalling it?

 Strangely it was working fine the other day. Then while debugging a
 script it suddenly started do this and now does this for every script

How were you debugging?

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior on Python 3 Windows Command Line

[Waylan Limberg]

On Mon, Feb 13, 2012 at 3:16 PM, Arnaud Delobelle arno...@gmail.com wrote:
 Strangely it was working fine the other day. Then while debugging a
 script it suddenly started do this and now does this for every script

 How were you debugging?

I think I may have been attempting to use pipes to redirect stdin
and/or stdout when the problem first presented itself.  Unfortunately,
once I closed the window, I lost whatever pipe combination I had
tried.

It just occurred to me that I was unsure if I had been doing that pipe
correctly, and that maybe I overwrote python.exe. Sure enough, the
modify date on that file indicated I overwrote it. A re-install has
resolved the problem.

It's just a little embarrassing that I didn't think of that until now,
but the fact that everything else seems to work was throwing me off.
Of course, everything else was running `pythonw.exe` not `python.exe`.

Anyway, thanks for the pointer Arnaud.

-- 

\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior from recursive generator

[Arnaud Delobelle]

On 23 September 2011 21:09, Dr. Phillip M. Feldman
phillip.m.feld...@gmail.com wrote:

 A few weeks ago, I wrote a class that creates an iterator for solving the
 general unlabeled-balls-in-labeled boxes occupancy problem. Chris Rebert
 converted my code to a generator, which made the code cleaner, and I
 subsequently simplified it somewhat further.

 My problem is the following: All of these versions of the code work fine for
 very small problems, but do not produce the full set of occupancy
 distributions for larger problems. The following sample input and output
 show what happens with two balls and two boxes (to keep things simple, I've
 made the boxes large enough so that each box can hold both balls).

 In [6]: x= balls_in_labeled_boxes(2,[2,2])

 In [7]: list(x)
 balls=2, box_sizes=[2, 2]
 About to make recursive call.  balls_in_other_boxes=0, box_sizes=[2]
 i=0, distribution_other=(0,)
 About to make recursive call.  balls_in_other_boxes=1, box_sizes=[2]
 i=0, distribution_other=(1,)
 About to make recursive call.  balls_in_other_boxes=2, box_sizes=[2]
 i=0, distribution_other=(2,)
 Out[7]: [(2, 0), (1, 1), (0, 2)]

 Note that Out[7] above gives the correct result, showing all three possible
 distributions. Now lets try the same thing with three boxes.

 In [8]: x= balls_in_labeled_boxes(2,[2,2,2])

 In [9]: list(x)
 balls=2, box_sizes=[2, 2, 2]
 About to make recursive call.  balls_in_other_boxes=0, box_sizes=[2, 2]
 i=0, distribution_other=(0, 0)
 About to make recursive call.  balls_in_other_boxes=1, box_sizes=[2, 2]
 i=0, distribution_other=(1, 0)
 About to make recursive call.  balls_in_other_boxes=2, box_sizes=[2, 2]
 i=0, distribution_other=(2, 0)
 i=1, distribution_other=(1, 1)
 Out[9]: [(2, 0, 0), (1, 1, 0), (0, 2, 0), (0, 1, 1)]

 When there are no balls in the initial box, the recursive call should
 produce the same three occupancy distributions that we saw above, but one of
 them is now missing. If someone can shed light on why this is happening, I'd
 be grateful.

Line 46:

 for distribution_other in _balls_in_unlabeled_boxes(

Should be:


 for distribution_other in _balls_in_labeled_boxes(

HTH

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior from recursive generator

[Phillip Feldman]

I don't know how many times I stared at that code without seeing the error.

Thanks so much!

Phillip

On Fri, Sep 23, 2011 at 1:26 PM, Arnaud Delobelle arno...@gmail.com wrote:

 On 23 September 2011 21:09, Dr. Phillip M. Feldman
 phillip.m.feld...@gmail.com wrote:
 
  A few weeks ago, I wrote a class that creates an iterator for solving the
  general unlabeled-balls-in-labeled boxes occupancy problem. Chris Rebert
  converted my code to a generator, which made the code cleaner, and I
  subsequently simplified it somewhat further.
 
  My problem is the following: All of these versions of the code work fine
 for
  very small problems, but do not produce the full set of occupancy
  distributions for larger problems. The following sample input and output
  show what happens with two balls and two boxes (to keep things simple,
 I've
  made the boxes large enough so that each box can hold both balls).
 
  In [6]: x= balls_in_labeled_boxes(2,[2,2])
 
  In [7]: list(x)
  balls=2, box_sizes=[2, 2]
  About to make recursive call.  balls_in_other_boxes=0, box_sizes=[2]
  i=0, distribution_other=(0,)
  About to make recursive call.  balls_in_other_boxes=1, box_sizes=[2]
  i=0, distribution_other=(1,)
  About to make recursive call.  balls_in_other_boxes=2, box_sizes=[2]
  i=0, distribution_other=(2,)
  Out[7]: [(2, 0), (1, 1), (0, 2)]
 
  Note that Out[7] above gives the correct result, showing all three
 possible
  distributions. Now lets try the same thing with three boxes.
 
  In [8]: x= balls_in_labeled_boxes(2,[2,2,2])
 
  In [9]: list(x)
  balls=2, box_sizes=[2, 2, 2]
  About to make recursive call.  balls_in_other_boxes=0, box_sizes=[2, 2]
  i=0, distribution_other=(0, 0)
  About to make recursive call.  balls_in_other_boxes=1, box_sizes=[2, 2]
  i=0, distribution_other=(1, 0)
  About to make recursive call.  balls_in_other_boxes=2, box_sizes=[2, 2]
  i=0, distribution_other=(2, 0)
  i=1, distribution_other=(1, 1)
  Out[9]: [(2, 0, 0), (1, 1, 0), (0, 2, 0), (0, 1, 1)]
 
  When there are no balls in the initial box, the recursive call should
  produce the same three occupancy distributions that we saw above, but one
 of
  them is now missing. If someone can shed light on why this is happening,
 I'd
  be grateful.

 Line 46:

 for distribution_other in _balls_in_unlabeled_boxes(

 Should be:


 for distribution_other in _balls_in_labeled_boxes(

 HTH

 --
 Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Aahz]

In article 626f24e5-4d8e-416c-b3ed-dc56a88dc...@s21g2000prm.googlegroups.com,
Lambda  stephenh...@gmail.com wrote:

def matrix_power(m, n):
  result = m[:]
  print result is m

Use copy.deepcopy()
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

You could make Eskimos emigrate to the Sahara by vigorously arguing --
at hundreds of screens' length -- for the wonder, beauty, and utility of
snow.  --PNH to rb in r.a.sf.f
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Lambda]

On Oct 28, 10:40 am, Chris Rebert c...@rebertia.com wrote:
 On Tue, Oct 27, 2009 at 7:15 PM, Lambda stephenh...@gmail.com wrote:
  I defined a function to raise a 2x2 matrix to nth power:

  def matrix_power(m, n):
   result = m[:]

 Note that this only copies the *outer* list. It does NOT copy any of
 the inner, nested lists, it just makes fresh *references* to them,
 which is what's causing your problem; the inner lists get shared, thus
 the 2D lists are almost the same list for practical purposes.

 You want:
 result = [row[:] for row in m]
 Which explicitly copies the inner lists.

 snip

  I find the matrix and m lists are always equal.
  But these two lists are separate objects, right?

 Right, but the *inner* list objects *are* the same in your code, and
 lists with the same contents (in this case, the same inner lists)
 compare as equal (==) but not identical (`is`). Remember that all
 lists in Python are 1-dimensional; multidimensional lists are merely
 by convention, hence you have to think a bit more carefully when
 working with them.

 Cheers,
 Chris
 --http://blog.rebertia.com

Thank you!
Following is my final code:

def matrix_power(m, n):
  
  Raise 2x2 matrix m to nth power.
  
  if n == 0: return [[1, 0], [0, 1]]

  x = matrix_power(m, n / 2)
  square = matrix_mul(x, x)
  if n % 2 == 0:
return square
  else:
return matrix_mul(m, square)

def matrix_mul(a, b):
  result = [row[:] for row in a]
  result[0][0] = a[0][0] * b[0][0] + a[0][1] * b[1][0]
  result[0][1] = a[0][0] * b[0][1] + a[0][1] * b[1][1]
  result[1][0] = a[1][0] * b[0][0] + a[1][1] * b[1][0]
  result[1][1] = a[1][0] * b[0][1] + a[1][1] * b[1][1]
  return result
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Mark Dickinson]

On Oct 28, 8:24 am, Lambda stephenh...@gmail.com wrote:
 Thank you!
 Following is my final code:

Looks good, but are you sure about that word 'final'?  ;-)


 def matrix_power(m, n):
   
   Raise 2x2 matrix m to nth power.
   
   if n == 0: return [[1, 0], [0, 1]]

   x = matrix_power(m, n / 2)

I suggest using n // 2 instead of n / 2 here:  n // 2 always
does integer divsion (aka 'floor division' in Python), while
the behaviour of n/2 depends on whether you're using Python
2.x or 3.1, and (in 2.x) on whether anyone's put a 'from
__future__ import division' at the top of the file.

   square = matrix_mul(x, x)
   if n % 2 == 0:
     return square
   else:
     return matrix_mul(m, square)

 def matrix_mul(a, b):
   result = [row[:] for row in a]
   result[0][0] = a[0][0] * b[0][0] + a[0][1] * b[1][0]
   result[0][1] = a[0][0] * b[0][1] + a[0][1] * b[1][1]
   result[1][0] = a[1][0] * b[0][0] + a[1][1] * b[1][0]
   result[1][1] = a[1][0] * b[0][1] + a[1][1] * b[1][1]
   return result

It's slightly more natural to build the list up as you go,
instead of creating a list of the right size and assigning
to its entries.  E.g.,

def matrix_mul(a, b):
row0 = [a[0][0]*b[0][0] + a[0][1]*b[1][0],
a[0][0]*b[0][1] + a[0][1]*b[1][1]]
row1 = similar
return [row0, row1]

Better still, use for loops to loop over the elements and
accumulate the sums, and then your code won't need rewriting
when you want to use it to take the power of 3-by-3 matrices.
And then check out the builtin 'sum' function, and consider
replacing some or all of the for loops with list
comprehensions (depending on personal taste)...

Not-so-final'ly yours,

Mark
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Peter Otten]

Lambda wrote:

 I defined a function to raise a 2x2 matrix to nth power:

 BTW, numpy has such a function, but it doesn't support really large
 number.
 Does numpy has some functions that support arbitrarily large number?

You can tell it to use Python instead of C integers:

 import numpy
 m = numpy.matrix([[1,2],[3,4]], dtype=object)
 m**100
matrix([[2477769229755647135316619942790719764614291718779321308132883358976984999,
 
3611168042210422417525189130504685706095268999850398812464239094782237250],

[5416752063315633626287783695757028559142903499775598218696358642173355875,
 
7894521293071280761604403638547748323757195218554919526829242001150340874]], 
dtype=object)

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Dave Angel]

Mark Dickinson wrote:

On Oct 28, 8:24 am, Lambda stephenh...@gmail.com wrote:
  

Thank you!
Following is my final code:



Looks good, but are you sure about that word 'final'?  ;-)

  

def matrix_power(m, n):
  
  Raise 2x2 matrix m to nth power.
  
  if n =0: return [[1, 0], [0, 1]]

  x =atrix_power(m, n / 2)



I suggest using n // 2 instead of n / 2 here:  n // 2 always
does integer divsion (aka 'floor division' in Python), while
the behaviour of n/2 depends on whether you're using Python
2.x or 3.1, and (in 2.x) on whether anyone's put a 'from
__future__ import division' at the top of the file.

  

  square =atrix_mul(x, x)
  if n % 2 =0:
return square
  else:
return matrix_mul(m, square)

def matrix_mul(a, b):
  result =row[:] for row in a]
  result[0][0] =[0][0] * b[0][0] + a[0][1] * b[1][0]
  result[0][1] =[0][0] * b[0][1] + a[0][1] * b[1][1]
  result[1][0] =[1][0] * b[0][0] + a[1][1] * b[1][0]
  result[1][1] =[1][0] * b[0][1] + a[1][1] * b[1][1]
  return result



It's slightly more natural to build the list up as you go,
instead of creating a list of the right size and assigning
to its entries.  E.g.,

def matrix_mul(a, b):
row0 =a[0][0]*b[0][0] + a[0][1]*b[1][0],
a[0][0]*b[0][1] + a[0][1]*b[1][1]]
row1 =similar
return [row0, row1]

Better still, use for loops to loop over the elements and
accumulate the sums, and then your code won't need rewriting
when you want to use it to take the power of 3-by-3 matrices.
And then check out the builtin 'sum' function, and consider
replacing some or all of the for loops with list
comprehensions (depending on personal taste)...

Not-so-final'ly yours,

  

exp = 600
mat = [ [3, 0], [0, 3]]
print mat, exp, matrix_power(mat, exp)
mat = [ [3.0, 0], [0, 3.0]]
print mat, exp, matrix_power(mat, exp); print power(mat, exp)

[[3, 0], [0, 3]] 600 
[[18739277038847939886754019920358123424308469030992781557966909983211910963157763678726120154469030856807730587971859910379069087693119051085139566217370635083384943613868029545256897117998608156843699465093293765833141309526696357142600866935689483770877815014461194837692223879905132001L, 
0L], [0L, 
18739277038847939886754019920358123424308469030992781557966909983211910963157763678726120154469030856807730587971859910379069087693119051085139566217370635083384943613868029545256897117998608156843699465093293765833141309526696357142600866935689483770877815014461194837692223879905132001L]]
[[3.0, 0], [0, 3.0]] 600 [[1.8739277038847955e+286, 0.0], [0.0, 
1.8739277038847955e+286]]

[[1.8739277038847965e+286, 0.0], [0.0, 1.8739277038847965e+286]]


Mark

  
That square/multiply technique brought back fond memories.  I used it 
when I did the microcode for floating point arithmetic in 1975 or so.  
Exponentiation used logs in the general case, but for integers below 100 
used this technique.  (At the time, the hardware instruction set was 
capable of multiplying only single digit numbers, the rest was loops in 
my microcode)


Probably I should also mention that, if the values had been floats 
instead of integers, the square/multiply technique generally improves 
accuracy, not just speed.  While repeatedly multiplying int/long values 
will eventually get the same answer, the quantization errors if they're 
floats do add up faster.  If we use the matrix


mat = [ [3, 0], [0, 3]]   n = 600
The first bunch of digits of the upper left corner are:
1873927703884793988675401...
If we give mat floating values, and run it again:
1.8739277038847955e+286
and if we use simple loop, multiplying 600 times, we get:
1.8739277038847965e+286

DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Chris Rebert]

On Tue, Oct 27, 2009 at 7:15 PM, Lambda stephenh...@gmail.com wrote:
 I defined a function to raise a 2x2 matrix to nth power:

 def matrix_power(m, n):
  result = m[:]

Note that this only copies the *outer* list. It does NOT copy any of
the inner, nested lists, it just makes fresh *references* to them,
which is what's causing your problem; the inner lists get shared, thus
the 2D lists are almost the same list for practical purposes.

You want:
result = [row[:] for row in m]
Which explicitly copies the inner lists.

snip
 I find the matrix and m lists are always equal.
 But these two lists are separate objects, right?

Right, but the *inner* list objects *are* the same in your code, and
lists with the same contents (in this case, the same inner lists)
compare as equal (==) but not identical (`is`). Remember that all
lists in Python are 1-dimensional; multidimensional lists are merely
by convention, hence you have to think a bit more carefully when
working with them.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior related to value / reference

[Terry Reedy]

Lambda wrote:

I defined a function to raise a 2x2 matrix to nth power:


There is a much faster way to raise x to a count power n than the 
definitional but naive method of multipling 1 by x n times. It is based 
on the binary representation of n.


Example: x**29 = x**(16+8+4+1) = x**16 * x**8 * x**4 * x * 1
So square x 4 times and do 4 more multiplies (total 8) instead of 28!

General algorithm is something like (untested):

def expo(x, n): # assume n is a count
  if n%2: res = x
  else:   res = 1 # or the identity for type(x)
  base = x
  n //= 2 # if n  2, we are done
  while n:
base *= base # or appropriate mul function
if n%2: res *= base # ditto
n //= 2
  return res

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior with os.system

[kmw]

That's it. I am calling my own program and not coreutils' sort, what
explains the unrequested output. Many thanks.

Cheers,
Kay

On 16 Jun., 22:16, Piet van Oostrum p...@cs.uu.nl wrote:
  kmw wuerz...@gmail.com (k) wrote:
 k Hi,
 k I wanted to write a simple script (in 5 minutes or so) which replaces
 k the option '+1' given to the command 'sort' by '-k 2' and than runs
 k 'sort' with the modified argument list. After two hours I am giving up
 k and ask you for help. This is what I tried (please excuse the verbose
 k code, it is due to my various efforts to understand the error):

 [snip]

 k Please note the unrequested output of 'type 'NoneType''. The strange
 k thing about this all is the fact that  the whole thing works as
 k expected when typed  into the interpreter. I would be glad if anyone
 k could help.

 MRAB has already given you some insight, I hope.
 But are you aware that you are calling your own program again?
 Or did you want to call the standard sort program? In that case you
 shouldn't call ./sort as it is in argv[0], but just sort (assuming '.'
 is not in your PATH) or the full path, like /usr/bin/sort.
 --
 Piet van Oostrum p...@cs.uu.nl
 URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4]
 Private email: p...@vanoostrum.org

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior with os.system

[MRAB]

kmw wrote:

Hi,

I wanted to write a simple script (in 5 minutes or so) which replaces
the option '+1' given to the command 'sort' by '-k 2' and than runs
'sort' with the modified argument list. After two hours I am giving up
and ask you for help. This is what I tried (please excuse the verbose
code, it is due to my various efforts to understand the error):

#!/usr/bin/python
import sys, os, re
arguments = sys.argv[0]
for i in sys.argv[1:]:
arguments +=   + i


Shorter:

arguments =  .join(sys.argv)


p = re.compile ( (\+(\d+)) )


This looks for a + followed by digits.


m = p.search ( arguments )
print type ( m )
m_list = list ( m.groups () )
print type ( m_list )
from1 = str ( m_list[0] )
to1 = -k  + str ( int ( m_list[1] ) + 1 )
cmd1 = str ( arguments.replace ( from1, to1 ) )
print cmd1
os.system ( cmd1 )

Now, this is what I get (on three different machines with different
versions of python):
type '_sre.SRE_Match'
type 'list'
./sort -F -k 2 -e


No +s, so the p.search(arguments) returns None.


type 'NoneType'
Traceback (most recent call last):
  File ./sort, line 9, in module
m_list = list ( m.groups () )
AttributeError: 'NoneType' object has no attribute 'groups'

Please note the unrequested output of 'type 'NoneType''. The strange
thing about this all is the fact that  the whole thing works as
expected when typed  into the interpreter. I would be glad if anyone
could help.



--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior with os.system

[Piet van Oostrum]

 kmw wuerz...@gmail.com (k) wrote:

k Hi,
k I wanted to write a simple script (in 5 minutes or so) which replaces
k the option '+1' given to the command 'sort' by '-k 2' and than runs
k 'sort' with the modified argument list. After two hours I am giving up
k and ask you for help. This is what I tried (please excuse the verbose
k code, it is due to my various efforts to understand the error):

[snip]

k Please note the unrequested output of 'type 'NoneType''. The strange
k thing about this all is the fact that  the whole thing works as
k expected when typed  into the interpreter. I would be glad if anyone
k could help.

MRAB has already given you some insight, I hope.
But are you aware that you are calling your own program again?
Or did you want to call the standard sort program? In that case you
shouldn't call ./sort as it is in argv[0], but just sort (assuming '.'
is not in your PATH) or the full path, like /usr/bin/sort.
-- 
Piet van Oostrum p...@cs.uu.nl
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Eric Kemp]

Tim Roberts wrote:

Scott David Daniels scott.dani...@acm.org wrote:

I avoid using single-letter variables except where I know the types
from the name (so I use i, j, k, l, m, n as integers, s as string,
and w, x, y, and z I am a little looser with (but usually float or
complex).


It's amazing to me that Fortran continues to live on in the hearts and
minds of today's programmers.


Fortran is alive and well.  If you can find a Fortran 2003 compiler, you 
can even use classes.

--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Steve Holden]

Tim Roberts wrote:
 Scott David Daniels scott.dani...@acm.org wrote:
 I avoid using single-letter variables except where I know the types
from the name (so I use i, j, k, l, m, n as integers, s as string,
 and w, x, y, and z I am a little looser with (but usually float or
 complex).
 
 It's amazing to me that Fortran continues to live on in the hearts and
 minds of today's programmers.

Well I think it's more that the original Fortran programmers were
applied mathematicians, who have always tended to use i through m as
integer variables, and that usage continues both in programming and
mathematics today.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Tim Roberts]

Scott David Daniels scott.dani...@acm.org wrote:

I avoid using single-letter variables except where I know the types
from the name (so I use i, j, k, l, m, n as integers, s as string,
and w, x, y, and z I am a little looser with (but usually float or
complex).

It's amazing to me that Fortran continues to live on in the hearts and
minds of today's programmers.
-- 
Tim Roberts, t...@probo.com
Providenza  Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Scott David Daniels]

David Lemper wrote:
I'm a newbee trying 3.0   Please help with  math.sqrt() 

At the command line this function works correctly 
   import math

  n = input(enter a number  )
  s = math.sqrt(n)
 An entry of 9 or 9.0  will yield 3.0



Yet the same code in a script gives an error message
 Script1
   import math
   n = input(enter a number  )
   s = math.sqrt(n)
 ... TypeError : a float is required



Strangely the above code runs fine in version 2.5  ( ? ) ...


OK, here's what's going on:
at the command line in 2.X, the builtin function input reads a string
and returns the eval of that string.  This is a bit of a safety issue.
I suspect when it worked from the command line, you were using a 2.X
command line inadvertently.  in 2.X, you'll get similar errors if you
use raw_input instead of input.

The input function in 3.0 is the same as the raw_input function
in 2.X.  I would suggest using:
import math
value = float(input(enter a number  ))
root = math.sqrt(value)
print('root(%s) == %s' % (value, root))

I avoid using single-letter variables except where I know the types
from the name (so I use i, j, k, l, m, n as integers, s as string,
and w, x, y, and z I am a little looser with (but usually float or
complex).

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Chris Rebert]

On Fri, Dec 26, 2008 at 1:52 PM,  da...@bag.python.org wrote:
 I'm a newbee trying 3.0   Please help with  math.sqrt()

 At the command line this function works correctly
   import math
  n = input(enter a number  )

raw_input() was renamed input() in Python 3.0, and it returns a
*string*, not a *number*.
Therefore, you need to convert the string to an int or float.

Change the line to:

n = float(input(enter a number  ))

And it should work just fine.

  s = math.sqrt(n)
 An entry of 9 or 9.0  will yield 3.0

 Yet the same code in a script gives an error message
 Script1
   import math
   n = input(enter a number  )
   s = math.sqrt(n)

   Traceback (most recent call last) :
  File stdin, line 1, in module
  File script1.py line 3 in module
 s = math.sqrt(n)
   TypeError : a float is required

You're currently giving it a string, not a number, which is
nonsensical, hence the TypeError. I presume ints would be coerced to
floats by the function.

 Entering 9 or 9.0 gives same error message.

   According to the math module the results of all
   functions are floats.  However it says nothing about
   inputs.

 Strangely the above code runs fine in version 2.5  ( ? )
 and will handle large integers.

 I've read the documentation for 3.0 including the section
 Floating Point Arithmetic: Issues  Limitations and it
 helps nada.

You should read What's New in Python 3.0, it covers changes such as
the one you've encountered -
http://docs.python.org/3.0/whatsnew/3.0.html

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[Gabriel Genellina]

En Fri, 26 Dec 2008 19:52:24 -0200, da...@bag.python.org escribió:


I'm a newbee trying 3.0   Please help with  math.sqrt()

At the command line this function works correctly
   import math
  n = input(enter a number  )
  s = math.sqrt(n)
 An entry of 9 or 9.0  will yield 3.0

Yet the same code in a script gives an error message
 Script1
   import math
   n = input(enter a number  )
   s = math.sqrt(n)
  Traceback (most recent call last) :
  File stdin, line 1, in module
  File script1.py line 3 in module
 s = math.sqrt(n)
   TypeError : a float is required
 Entering 9 or 9.0 gives same error message.

   According to the math module the results of all
   functions are floats.  However it says nothing about
   inputs.

Strangely the above code runs fine in version 2.5  ( ? )
and will handle large integers.

I've read the documentation for 3.0 including the section
Floating Point Arithmetic: Issues  Limitations and it
helps nada.


And you won't find nothing - the change is in input behavior, not in the  
math functions.

For versions prior to 3.0, there are:

raw_input(message) - string typed
input(message) - result of evaluating the string typed

raw_input just returns whatever you type, as a string. Using the input  
function, Python evaluates whatever you type to obtain a result: if you  
type the three characters nine dot zero the result is the double  
9.0; you can even type (17+1)/2.0 to get the same value (try it with your  
Python 2.5)


Since version 3.0, input behaves as raw_input in the older versions, and  
there is no builtin function equivalent to the old input function.

Use this instead:

n = float(input(enter a number  ))


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: strange behavior of math.sqrt() in new 3.0 version

[John Machin]

On Dec 27, 8:52 am, David Lemper wrote:
 I'm a newbee trying 3.0   Please help with  math.sqrt()

math.sqrt() is not the problem.

 At the command line this function works correctly
        import math
               n = input(enter a number  )
               s = math.sqrt(n)
      An entry of 9 or 9.0  will yield 3.0

I can't reproduce this. See below. Are you sure that you weren't using
Python 2.x? Start with a fresh Python 3.0, and please copy/paste
*exactly* what is on the screen, like I've done:

C:\junk\python30\python
Python 3.0 (r30:67507, Dec  3 2008, 20:14:27) [MSC v.1500 32 bit
(Intel)] on win32
Type help, copyright, credits or license for more information.
 n = input(float- )
float- 9
 type(n)
class 'str'
 repr(n)
'9'
 import math
 s = math.sqrt(n)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: a float is required

All of the above is exactly as expected. input() returns a string.
math.sqrt expects a float.


 Yet the same code in a script gives an error message
      Script1
                    import math
                    n = input(enter a number  )
                    s = math.sqrt(n)

                Traceback (most recent call last) :
                   File stdin, line 1, in module
                   File script1.py line 3 in module
                      s = math.sqrt(n)
                TypeError : a float is required
      Entering 9 or 9.0 gives same error message.

Again, as expected.


    According to the math module the results of all
    functions are floats.  However it says nothing about
    inputs.

 Strangely the above code runs fine in version 2.5  ( ? )
 and will handle large integers.

Suggestion: either choose ONE of 2.6 and 3.0 to learn Python, or plan
to spend a lot of time being confused, or reading What's new in
Python 3.0 -- a relevant snippet of which is:

PEP 3111: raw_input() was renamed to input(). That is, the new input()
function reads a line from sys.stdin and returns it with the trailing
newline stripped. It raises EOFError if the input is terminated
prematurely. To get the old behavior of input(), use eval(input()).


HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior of the Eclipse embedded console

[hellt]

On 20 мар, 14:31, hellt [EMAIL PROTECTED] wrote:
 i've faced with some strangeness while executing this sample:

 choice = raw_input(your choice: )
 print len(choice)

 when i run this sample in my eclipse console with CPython and print
 Yes, i have this output
 4 #trailing \t is the fourth element

 but when i use command line method
 python sample.py
 i have the correct length = 3

 i'm curious now, can anyone explain that?


just took a look into PyDEV-FAQ.

The eclipse console is not an exact copy of a shell... one of the
changes is that when you press ENTER in a shell, it may give you a
\r, \n or \r\n as an end-line char, depending on your platform. Python
does not expect this -- from the docs it says that it will remove the
last \n (checked in version 2.4), but, in some platforms that will
leave a \r there. This means that the raw_input() should usually be
used as raw_input().replace('\r', ''), and input() should be changed
for: eval(raw_input().replace('\r', '')).



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior of the Eclipse embedded console

[Preston Landers]

On Mar 20, 9:09 am, hellt [EMAIL PROTECTED] wrote:

 The eclipse console is not an exact copy of a shell... one of the
 changes is that when you press ENTER in a shell, it may give you a
 \r, \n or \r\n as an end-line char, depending on your platform. Python
 does not expect this -- from the docs it says that it will remove the
 last \n (checked in version 2.4), but, in some platforms that will
 leave a \r there. This means that the raw_input() should usually be
 used as raw_input().replace('\r', ''), and input() should be changed
 for: eval(raw_input().replace('\r', '')).

While I do love Eclipse as a Python code editor, interactive debugger,
front-end to SVN / Trac, and for many other things... it is
unfortunately terrible as an interactive console.

But then again, despite being a long time Emacs user I never did
anything complex inside Emacs' shell / terminal emulator, instead
preferring a 'real' console using xterm or rxvt or konsole or what
have you.

By the way, so far the only text editing feature from Emacs I am
missing in Eclipse is the ability to reformat line endings - i.e., M-x
fill-paragraph-or-region

regards
Preston
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior: csv module IDLE

[Marc 'BlackJack' Rintsch]

On Fri, 28 Dec 2007 18:12:58 -0800, t_rectenwald wrote:

 Within the program, the snippet where I use the csv module is below:
 
 ==
 csvfile = open('foo.csv', 'w')
 writer = csv.writer(csvfile)
 
 for row in rows:
 writer.writerow(row[0:3])
 
 csvfile.close
 ==
 
 The rows object is returned from a database query and is a list of
 tuples.  Now here is the strange thing.  If I run this program
 directly from the command line, i.e.,
 
 D:\test D:\python25\python foo.py
 
 It runs fine, foo.csv is created and all is well.  However, when I run
 it through the IDLE shell as described above, the foo.csv file is
 created but remains empty at 0 bytes.  When I try to delete the file,
 Windows says it is in use.  The only way I can break out of this is by
 restarting the IDLE shell.  In other words, it appears that the shell
 is hanging.
 
 This will run through Task Scheduler, so shouldn't be a problem, but
 I'm worried that I'm coding this wrong for it to be acting this way
 under IDLE.  Any help or explanation would be appreciated.

You are not closing the file so the buffered data is not written to disk. 
To call a function you need the parenthesis, otherwise you are just
referencing it without any effect.

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior: csv module IDLE

[John Machin]

On Dec 29, 1:12 pm, t_rectenwald [EMAIL PROTECTED] wrote:
 I've noticed an oddity when running a program, using the csv module,
 within IDLE.  I'm new to Python so am confused by what is happening.
 Here is what I'm doing:

 1) Open the IDLE Shell.
 2) Select File | Open...
 3) Choose my file, foo.py, opening it in a window.
 4) From that window, I hit F5 to run the module.

 Within the program, the snippet where I use the csv module is below:

Forget snippet, show us a *whole* program. Cut out the database
stuff; just use some simple made-up value for rows.


 ==
 csvfile = open('foo.csv', 'w')

Always use 'wb' -- not the cause of the current problem but it will
bite you later.

 writer = csv.writer(csvfile)

 for row in rows:
 writer.writerow(row[0:3])


Adding

del writer

may help

 csvfile.close

The above statement does nothing. You meant csvfile.close(), I
presume.


 ==

 The rows object is returned from a database query and is a list of
 tuples.  Now here is the strange thing.  If I run this program
 directly from the command line, i.e.,

 D:\test D:\python25\python foo.py

 It runs fine, foo.csv is created and all is well.  However, when I run
 it through the IDLE shell as described above, the foo.csv file is
 created but remains empty at 0 bytes.  When I try to delete the file,
 Windows says it is in use.  The only way I can break out of this is by
 restarting the IDLE shell.  In other words, it appears that the shell
 is hanging.

No it's not hanging, it's just that the file is still open; you
haven't closed it. It won't be closed until you exit IDLE.


 This will run through Task Scheduler, so shouldn't be a problem, but
 I'm worried that I'm coding this wrong for it to be acting this way
 under IDLE.  Any help or explanation would be appreciated.


Do these things inside a function, so that the objects get garbage-
collected on exit.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior: csv module IDLE

[t_rectenwald]

On Dec 28, 9:43 pm, John Machin [EMAIL PROTECTED] wrote:
 On Dec 29, 1:12 pm, t_rectenwald [EMAIL PROTECTED] wrote:

  I've noticed an oddity when running a program, using the csv module,
  within IDLE.  I'm new to Python so am confused by what is happening.
  Here is what I'm doing:

  1) Open the IDLE Shell.
  2) Select File | Open...
  3) Choose my file, foo.py, opening it in a window.
  4) From that window, I hit F5 to run the module.

  Within the program, the snippet where I use the csv module is below:

 Forget snippet, show us a *whole* program. Cut out the database
 stuff; just use some simple made-up value for rows.

  ==
  csvfile = open('foo.csv', 'w')

 Always use 'wb' -- not the cause of the current problem but it will
 bite you later.

  writer = csv.writer(csvfile)

  for row in rows:
      writer.writerow(row[0:3])

 Adding

 del writer

 may help

  csvfile.close

 The above statement does nothing. You meant csvfile.close(), I
 presume.

  ==

  The rows object is returned from a database query and is a list of
  tuples.  Now here is the strange thing.  If I run this program
  directly from the command line, i.e.,

  D:\test D:\python25\python foo.py

  It runs fine, foo.csv is created and all is well.  However, when I run
  it through the IDLE shell as described above, the foo.csv file is
  created but remains empty at 0 bytes.  When I try to delete the file,
  Windows says it is in use.  The only way I can break out of this is by
  restarting the IDLE shell.  In other words, it appears that the shell
  is hanging.

 No it's not hanging, it's just that the file is still open; you
 haven't closed it. It won't be closed until you exit IDLE.



  This will run through Task Scheduler, so shouldn't be a problem, but
  I'm worried that I'm coding this wrong for it to be acting this way
  under IDLE.  Any help or explanation would be appreciated.

 Do these things inside a function, so that the objects get garbage-
 collected on exit.

Thanks for all of the help.  I'm still learning Python so dorked up
here and didn't add the empty parenthesis around csvfile.close as I
should have.  So, it never called the close() function, but just
referenced it as was noted in your responses.  After doing that,
everything works fine and the file is closed properly.  I do have this
in a function in the actual script I'm writing, just sort of made a
dummy, foo.py, to do some testing and didn't have it in a function
there.

I'll research wb now to figure out what that does.  Thanks again for
the help!
Tom
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior of __get__ in a descriptor in IPython

[Diez B. Roggisch]

Jakub Hegenbart wrote:

 Hi,
 
 I'm studying the descriptor protocol and its usage from the following
 document:
 
 http://users.rcn.com/python/download/Descriptor.htm
 
 There is some sample code:
 
 http://users.rcn.com/python/download/Descriptor.htm#descriptor-example
 
 that behaves in a different way on my machine than the example suggests:
 
 In [2]: a=MyClass()
 
 In [3]: a.x
 Retrieving var x
 Retrieving var x
 Out[3]: 1
 
 On the other hand, in the 'plain' Python shell, it's invoked only once as
 expected:
 
 a=desc.MyClass()
 a.x
 Retrieving var x
 10

 
 Should I take as granted that IPython might in some cases access an
 attribute
 of an object more than once even in face of side effects, or is this a
 bug?

I'm not sure, but you could try and set up a counter that counts the number
of accesses.

But to be honest: I don't think it's a bug - this is one hell of an
essential part of python, not working here would cause all kinds of
troubles.

Instead, I presume the printing/shell-part of IPython is responsible.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior of __get__ in a descriptor in IPython

[Fernando Perez]

Jakub Hegenbart wrote:

 Hi,
 
 I'm studying the descriptor protocol and its usage from the following
 document:
 
 http://users.rcn.com/python/download/Descriptor.htm
 
 There is some sample code:
 
 http://users.rcn.com/python/download/Descriptor.htm#descriptor-example
 
 that behaves in a different way on my machine than the example suggests:
 
 In [2]: a=MyClass()
 
 In [3]: a.x
 Retrieving var x
 Retrieving var x
 Out[3]: 1
 
 On the other hand, in the 'plain' Python shell, it's invoked only once as
 expected:
 
 a=desc.MyClass()
 a.x
 Retrieving var x
 10

 
 Should I take as granted that IPython might in some cases access an
 attribute
 of an object more than once even in face of side effects, or is this a bug?

Yup, IPython does access attributes more than once in an attempt to determine
if things can be called as functions.  This behavior, however, only exists
if 'autocall' is active.  Here's an example:

In [1]: run desc

In [2]: m.x
Retrieving var x
Retrieving var x
Out[2]: 10

In [3]: m.x
Retrieving var x
Retrieving var x
Out[3]: 10

In [4]: autocall 0
Automatic calling is: OFF

In [5]: m.x
Retrieving var x
Out[5]: 10


As you can see, once autocall is disabled, the double access goes away.  There
really is no way to provide the autocall feature without any side effects
whatsoever, so if you need to avoid them at all costs, disable this feature. 
You can do it permanently by editing your ipythonrc file.

Cheers,

f

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior in Windows

[Douglas Woodrow]

On Mon, 4 Jun 2007 21:34:36, David Stockwell wxp [EMAIL PROTECTED] 
wrote

in DOS you can try this to see what your path is:

echo My path is %PATH%

or more simply:

,
| C: path
`
-- 
Doug Woodrow

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior in Windows

[Gabriel Genellina]

En Sat, 02 Jun 2007 15:31:40 -0300, Jakob Svavar Bjarnason  
[EMAIL PROTECTED] escribió:

 I have been trying to switch to Python from other languages such as Perl  
 and now I have a newbie question.

 I have been trying to run a setup script to install a python library. To  
 do that I need to run c: python setup.py install but the strange  
 thing is is that when it runs the command:

 from distutils import log

 I get the error ONLY while in cmd. If I run this from the interactive  
 shell that comes with IDLE it works. But if I go into the interactive  
 shell in the Windows shell cmd then I get:

 ImportError: cannot import log

Maybe you have two different versions installed? Try this and post what  
you get:

import sys
print sys.version
print sys.executable
print sys.path

both from inside IDLE and the console.
Python versions earlier than 2.3 did not have a distutils.log module.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior in Windows

[David Stockwell wxp]

my guess its your path.  I'm not familiar with IDLE but if you try getting 
the properties of IDLE, chances are its got the patch set up correctly.

in DOS you can try this to see what your path is:

echo My path is %PATH%

- Original Message - 
From: Jakob Svavar Bjarnason [EMAIL PROTECTED]
To: python-list@python.org
Sent: Saturday, June 02, 2007 2:31 PM
Subject: Strange behavior in Windows




Greetings.

I have been trying to switch to Python from other languages such as Perl and 
now I have a newbie question.

I have been trying to run a setup script to install a python library. To do 
that I need to run c: python setup.py install but the strange thing is is 
that when it runs the command:

 from distutils import log

I get the error ONLY while in cmd. If I run this from the interactive shell 
that comes with IDLE it works. But if I go into the interactive shell in the 
Windows shell cmd then I get:

ImportError: cannot import log

Any ideas on how I can fix this

Vinsamlega athugið að upplýsingar í tölvupósti þessum og viðhengi eru 
eingöngu ætlaðar þeim sem póstinum er beint til og gætu innihaldið 
upplýsingar sem eru trúnaðarmál. Sjá nánar: http://www.ru.is/trunadur

Please note that this e-mail and attachments are intended for the named 
addresses only and may contain information that is confidential and 
privileged. Further information: http://www.ru.is/trunadur


-- 
http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior with Old-Style classes and implicit __contains__

[Gabriel Genellina]

En Wed, 11 Apr 2007 20:37:35 -0300, [EMAIL PROTECTED] escribió:

 First, we create an instance of an Old-Style class without defining a
 __contains__ but instead define a __getitem__ method in which we raise
 KeyError. Next we repeatedly use the 'in' operator to test to see
 whether something, a string, an int, etc is an attribute of this new
 instance.

 Here's the strange part: The first test will return False as if the
 __getitem__ was never called. The next test will raise a KeyError as
 we'd expect. The test after that will again return False. This goes on
 ad infinitum.

  In Code:
   Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
   [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
   Type help, copyright, credits or license for more
 information.
class Foo:
   ... def __getitem__(self, key):
   ... raise KeyError
   ...
foo = Foo()
asdf in foo
   False
asdf in foo
   Traceback (most recent call last):
 File stdin, line 1, in module
 File stdin, line 3, in __getitem__
   KeyError

First I want to say that __getitem__ should raise IndexError, not  
KeyError, to indicate not found - just to make clear the observed  
behavior. (Using IndexError, you always get False, as expected).
Python 2.4 and 2.3 never return False, always showing the KeyError  
exception, also as expected.

 foo = Foo()
 asdf in foo
 False
 1 in set([1,2,3])   So the prior KeyError from another class  
 is interacting and producing bad output
 Traceback (most recent call last):
   File stdin, line 1, in module
   File stdin, line 3, in __getitem__
 KeyError

I have a displayhook installed, and it also were interfering with the  
results; it appears that the next __getitem__ call after the KeyError was  
raised (wherever it is called) sees the previous exception.

You should file a bug at http://sourceforge.net/bugs/?group_id=5470

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior with Old-Style classes and implicit __contains__

[Steven D'Aprano]

On Wed, 11 Apr 2007 16:37:35 -0700, rconradharris wrote:

 A co-worker of mine came across some interesting behavior in the
 Python interpreter today and I'm hoping someone more knowledgeable in
 Python internals can explain this to me.
 
 First, we create an instance of an Old-Style class without defining a
 __contains__ but instead define a __getitem__ method in which we raise
 KeyError. Next we repeatedly use the 'in' operator to test to see
 whether something, a string, an int, etc is an attribute of this new
 instance.
 
 Here's the strange part: The first test will return False as if the
 __getitem__ was never called. The next test will raise a KeyError as
 we'd expect. The test after that will again return False. This goes on
 ad infinitum.

I can confirm that. It looks like __getitem__ is only being called every
second time.

class Parrot:
def __getitem__(self, n):
print Checking index %s... % n
raise KeyError

def tester(n):
parrot = Parrot()
results = []
for i in range(n):
try:
results.append(i in parrot)
except KeyError:
results.append(KeyError)
return results



Here are the results under Python 2.5:

 tester(10)
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
[False, 'KeyError', False, 'KeyError', False, 
'KeyError', False, 'KeyError', False, 'KeyError']


And here are the results under Python 2.4.3:

 tester(10)
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
['KeyError', 'KeyError', 'KeyError', 'KeyError', 'KeyError', 
'KeyError', 'KeyError', 'KeyError', 'KeyError', 'KeyError']


Looks like a bug to me.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior with Old-Style classes and implicit __contains__

[Steven D'Aprano]

On Thu, 12 Apr 2007 00:32:32 -0300, Gabriel Genellina wrote:

 First I want to say that __getitem__ should raise IndexError, not  
 KeyError, to indicate not found

How do you know the Original Poster's class was meant to be a sequence
rather than a mapping?

-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior with Old-Style classes and implicit __contains__

[Scott David Daniels]

Steven D'Aprano wrote:
 On Wed, 11 Apr 2007 16:37:35 -0700, rconradharris wrote:
 ...
 Here are the results under Python 2.5:
 
 tester(10)
 Checking index 0...
 Checking index 0...
 Checking index 0...
 Checking index 0...
 Checking index 0...
 [False, 'KeyError', False, 'KeyError', False, 
 'KeyError', False, 'KeyError', False, 'KeyError']
 
 
 And here are the results under Python 2.4.3:
 
 tester(10)
[works]
 
 Looks like a bug to me.

No problem with 2.5.1c1 here.

--Scott David Daniels
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior with Old-Style classes and implicit __contains__

[Gabriel Genellina]

En Thu, 12 Apr 2007 01:23:08 -0300, Steven D'Aprano  
[EMAIL PROTECTED] escribió:

 On Thu, 12 Apr 2007 00:32:32 -0300, Gabriel Genellina wrote:

 First I want to say that __getitem__ should raise IndexError, not
 KeyError, to indicate not found

 How do you know the Original Poster's class was meant to be a sequence
 rather than a mapping?

Ouch, no, sorry, disregard that comment.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior when printing a returned closure function

[Jean-Paul Calderone]

On 25 Mar 2007 03:59:52 -0700, [EMAIL PROTECTED] wrote:
Hello,

when I execute the following code (python 2.5)

def f(x):
def g():
return x
return g

print f(1)
print f(2)

I get an output like

function g at 0x00AFC1F0
function g at 0x00AFC1F0

So according to print I get the same function object returned at both
calls.
That's surprising, I would expect to get two distinct function objects
because their func_closure attribute has to be different. And indeed,
if I do

print f(1) is f(2)

instead, it prints False. Even more confusing, if I do

g1 = f(1)
g2 = f(2)
print g1
print g2

I get something like

function g at 0x00AFC1B0
function g at 0x00AFC1F0

ie. two distinct function objects are printed.

What's happening here?
Some clever optimization reusing function objects in special cases or
what ...?

They're _not_ the same function object, just like the `is' test told you.
They just happen to have been allocated at the same memory address.

Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior when printing a returned closure function

[dartsch]

On Mar 25, 1:04 pm, Jean-Paul Calderone [EMAIL PROTECTED] wrote:
 On 25 Mar 2007 03:59:52 -0700, [EMAIL PROTECTED] wrote:



 Hello,

 when I execute the following code (python 2.5)

 def f(x):
 def g():
 return x
 return g

 print f(1)
 print f(2)

 I get an output like

 function g at 0x00AFC1F0
 function g at 0x00AFC1F0

 So according to print I get the same function object returned at both
 calls.
 That's surprising, I would expect to get two distinct function objects
 because their func_closure attribute has to be different. And indeed,
 if I do

 print f(1) is f(2)

 instead, it prints False. Even more confusing, if I do

 g1 = f(1)
 g2 = f(2)
 print g1
 print g2

 I get something like

 function g at 0x00AFC1B0
 function g at 0x00AFC1F0

 ie. two distinct function objects are printed.

 What's happening here?
 Some clever optimization reusing function objects in special cases or
 what ...?

 They're _not_ the same function object, just like the `is' test told you.
 They just happen to have been allocated at the same memory address.

 Jean-Paul

ah yes, I see, thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior when printing a returned closure function

[Steven D'Aprano]

On Sun, 25 Mar 2007 03:59:52 -0700, dartsch wrote:


 I get an output like
 
 function g at 0x00AFC1F0
 function g at 0x00AFC1F0
 
 So according to print I get the same function object returned at both
 calls.

Not the same function object. The first object is printed, then deleted
by the garbage collector because it goes out of scope. Then the second one
is created and just happens to end up in the same memory location. That's
an accident of the garbage collector implementation.


 That's surprising, I would expect to get two distinct function objects
 because their func_closure attribute has to be different. And indeed,
 if I do

[snip]

 function g at 0x00AFC1B0
 function g at 0x00AFC1F0
 
 ie. two distinct function objects are printed.

This time the first function still exists when the second is created, so
the second naturally can't be in the same memory location.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Rob Williscroft]

abcd wrote in news:[EMAIL PROTECTED] in 
comp.lang.python:

 class Foo:
 def __init__(self, name, data=[]):

http://docs.python.org/ref/function.html#l2h-619

Rob.
-- 
http://www.victim-prime.dsl.pipex.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[abcd]

Rob Williscroft wrote:
 http://docs.python.org/ref/function.html#l2h-619


thanks.  weird that it works that way since they even state This is
generally not what was intended.

oh well.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Steven D'Aprano]

On Mon, 16 Oct 2006 07:26:05 -0700, abcd wrote:

 class Foo:
 def __init__(self, name, data=[]):

The binding of the name data to the empty list happens at compile time,
not runtime.

 self.name = name
 self.data = data
 
 def addData(self, val):
 self.data.append(val)

Every time you call addData on an instance, it appends to the same list.
So all instances created with Foo(name) share the same list in data.

Think of it like this:

some_list = []
x = Foo(fred, some_list)
y = Foo(wilma, some_list)

Isn't it obvious now that both instances share the same list? That x.data
and y.data don't just have the same value, but are the same object? The
same thing happens when you set the default.


 f = Foo('a')
 f.addData(1)
 f.addData(2)
 
 f2 = Foo('b', [])

And in this case, you've passed a DIFFERENT empty list as an argument.


The normal Python way for handling this situation is to not use mutable
objects as defaults unless you want this behaviour. Instead, use None as
the default value:

class Foo:
def __init__(self, name, data=None):
self.name = name
if data is None: self.data = []
else: self.data = data


 Any ideas?  is this a bug?

Well, it's a bug in your code :)

It isn't a bug in Python. At worst, it is a gotcha, but it is a
deliberate design decision, and quite useful. For example, this is good
for caching complicated calculations:

def function(x, _cache={}):
# _cache is initialised to an empty dictionary at compile time
if _cache.has_key(x):
return _cache[x]
else:
# complicated and time consuming calculation happens
_cache[x] = result
return result




-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Paul Rubin]

Steven D'Aprano [EMAIL PROTECTED] writes:
 It isn't a bug in Python. At worst, it is a gotcha, but it is a
 deliberate design decision, and quite useful. For example, this is good
 for caching complicated calculations:
 
 def function(x, _cache={}):
 # _cache is initialised to an empty dictionary at compile time
 if _cache.has_key(x):
 return _cache[x]

The above can be done explicitly:

 def function(x):
 if function._cache.has_key(x):
 return function._cache[x]
 ...
 # function gets an initially-empty cache
 function._cache = {}

So the existing behavior, while not a bug (since it's documented), may
well be a wart.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Neil Cerutti]

On 2006-10-16, Steven D'Aprano
[EMAIL PROTECTED] wrote:
 Well, it's a bug in your code :)

 It isn't a bug in Python. At worst, it is a gotcha, but it is
 a deliberate design decision, and quite useful. For example,
 this is good for caching complicated calculations:

I'd say the feature is usable rather than useful, like
bitfields in C.

-- 
Neil Cerutti
Next Sunday Mrs. Vinson will be soloist for the morning service.
The pastor will then speak on It's a Terrible Experience.
--Church Bulletin Blooper 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Fredrik Lundh]

Steven D'Aprano wrote:

 It isn't a bug in Python. At worst, it is a gotcha, but it is a
 deliberate design decision, and quite useful. For example, this is good
 for caching complicated calculations:

it's also used to pass in *objects* instead of names into an inner scope.

/F

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Diez B. Roggisch]

abcd wrote:

 Rob Williscroft wrote:
 http://docs.python.org/ref/function.html#l2h-619
 
 
 thanks.  weird that it works that way since they even state This is
 generally not what was intended.

The not intended refers to the programmer making the mistake of creating a
shared instance - which usually isn't intended, as you yourself are an
example of.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Carsten Haese]

On Mon, 2006-10-16 at 10:51, Steven D'Aprano wrote:
 On Mon, 16 Oct 2006 07:26:05 -0700, abcd wrote:
 
  class Foo:
  def __init__(self, name, data=[]):
 
 The binding of the name data to the empty list happens at compile time,
 not runtime.

I think this statement needs to be clarified. The binding of data to
the empty list *does* happen at runtime, not at compile time. However,
the binding happens only once, when the def statement is executed, as
opposed to every time the __init__ function is called.

-Carsten


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange Behavior

[Fredrik Lundh]

Carsten Haese wrote:

 I think this statement needs to be clarified. The binding of data to
 the empty list *does* happen at runtime, not at compile time. However,
 the binding happens only once, when the def statement is executed, as
 opposed to every time the __init__ function is called.

to be precise, it happens every time the def statement is executed.

/F

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[[EMAIL PROTECTED]]

Gary Herron wrote:
 List comprehension is a great shortcut, but when the shortcut starts
 causing trouble, better to go with the old ways. You need to reopen each
 file each time you want to iterate through it. You should be able to
 understand the difference between these two bits of code.

 The first bit opens each file but uses (two of them) multiple times.
 Reading from a file at EOF returns an empty sequence.

 The second bit opened the file each time you want to reuse it. That
 works correctly.

 And that suggest the third bit of correctly working code which uses list
 comprehension.

 # Fails because files are opened once but reused
 f1 = open('word1.txt')
 f2 = open('word2.txt')
 f3 = open('word3.txt')
 for i1 in f1:
   for i2 in f2:
 for i3 in f3:
   print (i1.strip(),i2.strip(),i3.strip())

 and

 # Works because files are reopened for each reuse:
 f1 = open('word1.txt')
 for i1 in f1:
 f2 = open('word2.txt')
 for i2 in f2:
 f3 = open('word3.txt')
 for i3 in f3:
 print (i1.strip(),i2.strip(),i3.strip())

 and

 # Also works because files are reopened for each use:
 print [(i1.strip(),i2.strip(),i3.strip())
   for i1 in open('word1.txt')
 for i2 in open('word2.txt')
   for i3 in open('word3.txt')]

 Hope that's clear!

 Gary Herron


My original problem was with recursion.  I explicitly nested it out to
try and understand the behavior - and foolishly looked in the wrong
spot for the problem, namely that file is not reitreable.  In truth I
was never concerned about file objects, the problem was failing with my
own custom iterators (wich also were not reiterable) and I switched to
file, to eliminate possible code deficiencies on my own part.  I was
simply chasing down the wrong problem.   As was pointed out to me in a
nother thread - the cleanest implementation which would allow me to use
one copy of the file (in my example the files are identical) would be
to use a trivial iterator class that opens the file, uses tell to track
position and seek to set position, and returns the appropriate line for
that instance - thus eliminating unnecessary file opens and closes.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[Gary Herron]

[EMAIL PROTECTED] wrote:

Gary Herron wrote:
  

List comprehension is a great shortcut, but when the shortcut starts
causing trouble, better to go with the old ways. You need to reopen each
file each time you want to iterate through it. You should be able to
understand the difference between these two bits of code.

The first bit opens each file but uses (two of them) multiple times.
Reading from a file at EOF returns an empty sequence.

The second bit opened the file each time you want to reuse it. That
works correctly.

And that suggest the third bit of correctly working code which uses list
comprehension.

# Fails because files are opened once but reused
f1 = open('word1.txt')
f2 = open('word2.txt')
f3 = open('word3.txt')
for i1 in f1:
  for i2 in f2:
for i3 in f3:
  print (i1.strip(),i2.strip(),i3.strip())

and

# Works because files are reopened for each reuse:
f1 = open('word1.txt')
for i1 in f1:
f2 = open('word2.txt')
for i2 in f2:
f3 = open('word3.txt')
for i3 in f3:
print (i1.strip(),i2.strip(),i3.strip())

and

# Also works because files are reopened for each use:
print [(i1.strip(),i2.strip(),i3.strip())
  for i1 in open('word1.txt')
for i2 in open('word2.txt')
  for i3 in open('word3.txt')]

Hope that's clear!

Gary Herron




My original problem was with recursion.  I explicitly nested it out to
try and understand the behavior - and foolishly looked in the wrong
spot for the problem, namely that file is not reitreable.  In truth I
was never concerned about file objects, the problem was failing with my
own custom iterators (wich also were not reiterable) and I switched to
file, to eliminate possible code deficiencies on my own part.  I was
simply chasing down the wrong problem.   As was pointed out to me in a
nother thread - the cleanest implementation which would allow me to use
one copy of the file (in my example the files are identical) would be
to use a trivial iterator class that opens the file, uses tell to track
position and seek to set position, and returns the appropriate line for
that instance - thus eliminating unnecessary file opens and closes.

  

I see. 

I wouldn't call tell and seek clean.  Here's another suggestion.  Use
  l1 = open(...).readlines()
to read the whole file into a (nicely reiterable) list residing in 
memory, and then iterate through the list as you wish.  Only if your 
files are MANY megabytes long would this be a problem with memory 
consumption.  (But if they were that big, you wouldn't be trying to find 
all permutations would you!)

Gary Herron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[[EMAIL PROTECTED]]

My original concern and reason for goint the iterator/generator route
was exactly for large large lists :)  Unnecessary in this example, but
exactly what I was exploring.  I wouldn't be using list comprehension
for generating the permutiations.  Where all this came from was
creating a generator/iterator to handle very large permutations.



Gary Herron wrote:
 [EMAIL PROTECTED] wrote:

 Gary Herron wrote:
 
 
 List comprehension is a great shortcut, but when the shortcut starts
 causing trouble, better to go with the old ways. You need to reopen each
 file each time you want to iterate through it. You should be able to
 understand the difference between these two bits of code.
 
 The first bit opens each file but uses (two of them) multiple times.
 Reading from a file at EOF returns an empty sequence.
 
 The second bit opened the file each time you want to reuse it. That
 works correctly.
 
 And that suggest the third bit of correctly working code which uses list
 comprehension.
 
 # Fails because files are opened once but reused
 f1 = open('word1.txt')
 f2 = open('word2.txt')
 f3 = open('word3.txt')
 for i1 in f1:
   for i2 in f2:
 for i3 in f3:
   print (i1.strip(),i2.strip(),i3.strip())
 
 and
 
 # Works because files are reopened for each reuse:
 f1 = open('word1.txt')
 for i1 in f1:
 f2 = open('word2.txt')
 for i2 in f2:
 f3 = open('word3.txt')
 for i3 in f3:
 print (i1.strip(),i2.strip(),i3.strip())
 
 and
 
 # Also works because files are reopened for each use:
 print [(i1.strip(),i2.strip(),i3.strip())
   for i1 in open('word1.txt')
 for i2 in open('word2.txt')
   for i3 in open('word3.txt')]
 
 Hope that's clear!
 
 Gary Herron
 
 
 
 
 My original problem was with recursion.  I explicitly nested it out to
 try and understand the behavior - and foolishly looked in the wrong
 spot for the problem, namely that file is not reitreable.  In truth I
 was never concerned about file objects, the problem was failing with my
 own custom iterators (wich also were not reiterable) and I switched to
 file, to eliminate possible code deficiencies on my own part.  I was
 simply chasing down the wrong problem.   As was pointed out to me in a
 nother thread - the cleanest implementation which would allow me to use
 one copy of the file (in my example the files are identical) would be
 to use a trivial iterator class that opens the file, uses tell to track
 position and seek to set position, and returns the appropriate line for
 that instance - thus eliminating unnecessary file opens and closes.
 
 
 
 I see.

 I wouldn't call tell and seek clean.  Here's another suggestion.  Use
   l1 = open(...).readlines()
 to read the whole file into a (nicely reiterable) list residing in
 memory, and then iterate through the list as you wish.  Only if your
 files are MANY megabytes long would this be a problem with memory
 consumption.  (But if they were that big, you wouldn't be trying to find
 all permutations would you!)
 
 Gary Herron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[Terry Reedy]

[EMAIL PROTECTED] wrote in message 
news:[EMAIL PROTECTED]
 Ok, I am confused about this one.  I'm not sure if it's a bug or a
 feature.. but

  RESTART
 f1 = open('word1.txt')
 f2 = open('word2.txt')
 f3 = open('word3.txt')
 print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in f1 for i2 in f2 
 for i3 in f3]
 [('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c')]

A file is something like an iterator and something like an iterable.  At 
this point, the internal cursur for f3 points at EOF.  To reiterate thru 
the file, you must rewind in the inner loops.  So try (untest by me)

def initf(fil):
   f.seek(0)
   return f

and ...for i2 in initf(f2) for i3 in initf(f3)


 l1 = ['a\n','b\n','c\n']
 l2 = ['a\n','b\n','c\n']

 l3 = ['a\n','b\n','c\n']
 print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in l1 for i2 in l2 
 for i3 in l3]
 [('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('a', 'b', 'a'),
 ('a', 'b', 'b'), ('a', 'b', 'c'), ('a', 'c', 'a'), ('a', 'c', 'b'),
 ('a', 'c', 'c'), ('b', 'a', 'a'), ('b', 'a', 'b'), ('b', 'a', 'c'),
 ('b', 'b', 'a'), ('b', 'b', 'b'), ('b', 'b', 'c'), ('b', 'c', 'a'),
 ('b', 'c', 'b'), ('b', 'c', 'c'), ('c', 'a', 'a'), ('c', 'a', 'b'),
 ('c', 'a', 'c'), ('c', 'b', 'a'), ('c', 'b', 'b'), ('c', 'b', 'c'),
 ('c', 'c', 'a'), ('c', 'c', 'b'), ('c', 'c', 'c')]

 explanation of code:  the files word1.txt, word2.txt and word3.txt are
 all identical conataining the letters a,b and c one letter per line.
 The lists I've added the \n so that the lists are identical to what
 is returned by the file objects.  Just eliminating any possible
 differences.

But lists are not file objects and you did not eliminate the crucial 
difference in reiterability.  Try your experiment with StringIO objects, 
which are more nearly identical to file objects.

Terry Jan Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[Inyeol Lee]

On Tue, May 30, 2006 at 01:11:26PM -0700, [EMAIL PROTECTED] wrote:
[...]
   RESTART
  f1 = open('word1.txt')
  f2 = open('word2.txt')
  f3 = open('word3.txt')
  print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in f1 for i2 in f2 for 
  i3 in f3]
 [('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c')]
  l1 = ['a\n','b\n','c\n']
  l2 = ['a\n','b\n','c\n']
 
  l3 = ['a\n','b\n','c\n']
  print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in l1 for i2 in l2 for 
  i3 in l3]
 [('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('a', 'b', 'a'),
 ('a', 'b', 'b'), ('a', 'b', 'c'), ('a', 'c', 'a'), ('a', 'c', 'b'),
 ('a', 'c', 'c'), ('b', 'a', 'a'), ('b', 'a', 'b'), ('b', 'a', 'c'),
 ('b', 'b', 'a'), ('b', 'b', 'b'), ('b', 'b', 'c'), ('b', 'c', 'a'),
 ('b', 'c', 'b'), ('b', 'c', 'c'), ('c', 'a', 'a'), ('c', 'a', 'b'),
 ('c', 'a', 'c'), ('c', 'b', 'a'), ('c', 'b', 'b'), ('c', 'b', 'c'),
 ('c', 'c', 'a'), ('c', 'c', 'b'), ('c', 'c', 'c')]
 
 explanation of code:  the files word1.txt, word2.txt and word3.txt are
 all identical conataining the letters a,b and c one letter per line.
 The lists I've added the \n so that the lists are identical to what
 is returned by the file objects.  Just eliminating any possible
 differences.

You're comparing file, which is ITERATOR, and list, which is ITERABLE,
not ITERATOR. To get the result you want, use this instead;

 print [(i1.strip(),i2.strip(),i3.strip(),)
for i1 in open('word1.txt')
for i2 in open('word2.txt')
for i3 in open('word3.txt')]

FIY, to get the same buggy(?) result using list, try this instead;

 l1 = iter(['a\n','b\n','c\n'])
 l2 = iter(['a\n','b\n','c\n'])
 l3 = iter(['a\n','b\n','c\n'])
 print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in l1 for i2 in l2 for i3 
 in l3]
[('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c')]



-Inyeol Lee
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[Gary Herron]

[EMAIL PROTECTED] wrote:

Ok, I am confused about this one.  I'm not sure if it's a bug or a
feature.. but
  

List comprehension is a great shortcut, but when the shortcut starts 
causing trouble, better to go with the old ways. You need to reopen each 
file each time you want to iterate through it. You should be able to 
understand the difference between these two bits of code.

The first bit opens each file but uses (two of them) multiple times. 
Reading from a file at EOF returns an empty sequence.

The second bit opened the file each time you want to reuse it. That 
works correctly.

And that suggest the third bit of correctly working code which uses list 
comprehension.

# Fails because files are opened once but reused
f1 = open('word1.txt')
f2 = open('word2.txt')
f3 = open('word3.txt')
for i1 in f1:
  for i2 in f2:
for i3 in f3:
  print (i1.strip(),i2.strip(),i3.strip())

and 

# Works because files are reopened for each reuse:
f1 = open('word1.txt')
for i1 in f1:
f2 = open('word2.txt')
for i2 in f2:
f3 = open('word3.txt')
for i3 in f3:
print (i1.strip(),i2.strip(),i3.strip())

and

# Also works because files are reopened for each use:
print [(i1.strip(),i2.strip(),i3.strip()) 
  for i1 in open('word1.txt')
for i2 in open('word2.txt') 
  for i3 in open('word3.txt')]

Hope that's clear!

Gary Herron





  

 RESTART
f1 = open('word1.txt')
f2 = open('word2.txt')
f3 = open('word3.txt')
print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in f1 for i2 in f2 for i3 
in f3]


[('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c')]
  

l1 = ['a\n','b\n','c\n']
l2 = ['a\n','b\n','c\n']

l3 = ['a\n','b\n','c\n']
print [(i1.strip(),i2.strip(),i3.strip(),) for i1 in l1 for i2 in l2 for i3 
in l3]


[('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('a', 'b', 'a'),
('a', 'b', 'b'), ('a', 'b', 'c'), ('a', 'c', 'a'), ('a', 'c', 'b'),
('a', 'c', 'c'), ('b', 'a', 'a'), ('b', 'a', 'b'), ('b', 'a', 'c'),
('b', 'b', 'a'), ('b', 'b', 'b'), ('b', 'b', 'c'), ('b', 'c', 'a'),
('b', 'c', 'b'), ('b', 'c', 'c'), ('c', 'a', 'a'), ('c', 'a', 'b'),
('c', 'a', 'c'), ('c', 'b', 'a'), ('c', 'b', 'b'), ('c', 'b', 'c'),
('c', 'c', 'a'), ('c', 'c', 'b'), ('c', 'c', 'c')]

explanation of code:  the files word1.txt, word2.txt and word3.txt are
all identical conataining the letters a,b and c one letter per line.
The lists I've added the \n so that the lists are identical to what
is returned by the file objects.  Just eliminating any possible
differences.


If you notice, when using the file objects I don't get the proper set
of permutations.  I was playing around with doing this via recursion,
etc.  But nothing was working so I made a simplest case nesting.  Still
no go.
Why does this not work with the file objects?  Or any other class I''ve
made which implements __iter__ and next?

Seems like a bug to me, but maybe I am missing something.  Seems to
happen in 2.3 and 2.4.

  


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with iterables - is this a bug?

[[EMAIL PROTECTED]]

DOH!!

thanks a lot.  had to be something stupid on my part.  

Now I get it :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Rene Pijlman]

Vinay Sajip:
Rene Pijlman:
 It struck me as somewhat complicated as well.

You should look at later versions of Python - your points above about
easier configuration have already been addressed: here's a link from
the current (2.4) docs:

http://docs.python.org/lib/minimal-example.html

Yes, that looks good. Thanks for pointing that out. So... my advice to OP
(if still alive) is:

Add logging to your program:
http://docs.python.org/lib/minimal-example.html

And log the environment:
http://www.python.org/dev/doc/newstyle/lib/os-procinfo.html

 :-)

-- 
René Pijlman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Jim]

 But I do think that adding logging to a cgi script is a sensible thing to
 do for a beginner.
Let me second that.  I happen to write a lot of CGI, and ISTM that
while you can do it without logging, you are condemming yourself to a
lot of staring at the screen, head-scratching, and saying ``Now what
could *that* mean?''  That is, CGI programming without logging seems to
me to ba a lot like general programming in Perl.  :-}

Jim

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Vinay Sajip]

Rene Pijlman wrote:
 It struck me as somewhat complicated as well.

 Looking at the basic example:
 http://www.python.org/doc/2.3.5/lib/node304.html

 ... the things that first-time users shouldn't be bothered with IMO are:

 1. Getting a logger by name from a hierarchical namespace. There should be
 a module-level log function that logs through the root logger.
 2. Formatting the message (use a sensible default)
 3. Adding a handler to a logger (artefact of the logging system).
 4. Setting a log level (use a sensible default).

 Perhaps there should be some module-level functions that make it easier to
 'just log this message to that file'.

 But I do think that adding logging to a cgi script is a sensible thing to
 do for a beginner. Getting that to run in a debugger is probably way more
 complicated.

You should look at later versions of Python - your points above about
easier configuration have already been addressed: here's a link from
the current (2.4) docs:

http://docs.python.org/lib/minimal-example.html

Regards,

Vinay Sajip

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Duncan Booth]

Steve Holden wrote:

 Add logging to your program:
 http://www.python.org/doc/2.3.5/lib/module-logging.html
 
 Probably oiverkill, particularly for a beginner (is it only me that 
 thinks the logging module is either way over-complicated or way 
 under-documented?).

No, I would agree with you on that. I get the impression that it was 
designed by studying some pre-existing logging packages and saying 'what 
neat features can we take from each of these' rather than by writing up a 
set of use-cases and saying 'what is the simplest way we can make all of 
these possible'. 

The way it is now means there is quite a step from the simplest possible 
use to slightly more complex use. For example, the support for reading 
configuration files is wonderfully general purpose, but far too much for 
the vast majority of applications: you really don't want to expose a 
typical end user to that sort of configuration soup. If all you want in 
your application is to let the user specify the name of the logfile and the 
logging level you are on your own (and you can't even convert the log level 
name from a string to the required number without accessing an _ prefixed 
variable in the logging module).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Rene Pijlman]

Steve Holden:
Rene Pijlman:
 Add logging to your program:
 http://www.python.org/doc/2.3.5/lib/module-logging.html
 
Probably oiverkill, particularly for a beginner (is it only me that 
thinks the logging module is either way over-complicated or way 
under-documented?).

It struck me as somewhat complicated as well.

Looking at the basic example:
http://www.python.org/doc/2.3.5/lib/node304.html

... the things that first-time users shouldn't be bothered with IMO are:

1. Getting a logger by name from a hierarchical namespace. There should be
a module-level log function that logs through the root logger.
2. Formatting the message (use a sensible default)
3. Adding a handler to a logger (artefact of the logging system).
4. Setting a log level (use a sensible default).

Perhaps there should be some module-level functions that make it easier to
'just log this message to that file'.

But I do think that adding logging to a cgi script is a sensible thing to
do for a beginner. Getting that to run in a debugger is probably way more
complicated.

-- 
René Pijlman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Fredrik Lundh]

Rene Pijlman wrote:

 But I do think that adding logging to a cgi script is a sensible thing to
 do for a beginner. Getting that to run in a debugger is probably way more
 complicated.

on the other hand, adding

if 1: # set to 0 when deploying
print pre
print cgi.escape(repr(os.environ))
print /pre

to the CGI script isn't that hard... (if you're willing to do view source in
the browser, you can skip the pre and cgi.escape() stuff...)

/F



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Dennis Benzinger]

sophie_newbie schrieb:
 I have written a cgi script that seems to run perfectly from the
 command line when I simulate some cgi input using
 os.environ['QUERY_STRING'].
 
 The thing is that when I run it from the browser, one of my os.system
 calls, which gets excecuted fine when running the program in the
 interpreter, doesn't seem to get excecuted, or gets excecuted but does
 nothing.
 
 Does anyone know whats going on or how I could debug this problem?
  [...]

Probably the environment is different when your program is executed by 
your web server. So either PATH is wrong and your program is not found 
or some other environment variable is wrong and you could debug it by 
printing the environment in your program.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[sophie_newbie]

OK, interesting, but just how dow I print he environment in the
program??

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Rene Pijlman]

sophie_newbie:
OK, interesting, but just how dow I print he environment in the
program??

Add logging to your program:
http://www.python.org/doc/2.3.5/lib/module-logging.html

And log the environment:
http://www.python.org/dev/doc/newstyle/lib/os-procinfo.html

-- 
René Pijlman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior with os call in cgi script

[Steve Holden]

Rene Pijlman wrote:
 sophie_newbie:
 
OK, interesting, but just how dow I print he environment in the
program??
 
 
 Add logging to your program:
 http://www.python.org/doc/2.3.5/lib/module-logging.html
 
Probably oiverkill, particularly for a beginner (is it only me that 
thinks the logging module is either way over-complicated or way 
under-documented?).

 And log the environment:
 http://www.python.org/dev/doc/newstyle/lib/os-procinfo.html
 

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange behavior of int()

[Rob E]

 Why is int(r/k), where r = 0.5 and k = 0.5 = 0?  Shouldn't it be 1?
 And why is the last one = 4 and not 5?

I dont' know why the differences in your exact case.  However, please
realise that Regardless of the programming language good programming 
practice is to never rely on the int of a floating point division 
-- or even the int of a floating point constant to be exactly 
the integer value you expect it to be.  This is because both 
constant representations and math operations have rounding 
error.  Instead do less precise conversions by rounding.  For 
example:

a=b/c
if (a = 0.0):
   d = int(a+0.5)
else:
   d = int(a-0.5)

If you don't do this sort of thing your programs generally 
are  senstivie to the details of foating point rounding -- 
which is generally dependent on processor, compilier and
in pythons case probably the runtime system.

Rob
-- 
http://mail.python.org/mailman/listinfo/python-list