[OT] Simulation Results Managment

[moogyd]

Hi,
This is a general question, loosely related to python since it will be the 
implementation language.
I would like some suggestions as to manage simulation results data from my ASIC 
design. 

For my design, 
- I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of 
these test cases we have: 
  - a number of properties (P_AA_BB_CC) 
  - For each property, the following information is given
- Property name (P_NAME) 
- Number of times it was checked (within the testcase) N_CHECKED
- Number of times if failed (within the testcase) N_FAILED
- A simulation runs a testcase with a set of parameters.
  - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
- For the design, I will run regression every night (at least), so I will have 
results from multiple timestamps
We have < 1000 TESTCASES, and < 1000 PROPERTIES.

At the moment, I have a script that extracts property information from 
simulation logfile, and provides single PASS/FAIL and all logfiles stored in a 
directory structure with timestamps/testnames and other parameters embedded in 
paths

I would like to be easily look at (visualize) the data and answer the questions
- When did this property last fail, and how many times was it checked
- Is this property checked in this test case.

Initial question: How to organize the data within python?
For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, 
N_FAILED
I then have to store multiple instances of testcase based on date (and 
simulation parameters.

Any comments, suggestions?
Thanks,
Steven







-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[rusi]

On Jul 14, 8:43 am, Steven D'Aprano  wrote:
> On Fri, 13 Jul 2012 19:31:24 -0700, rusi wrote:
> > Consider the following
>
> > def foo(x):
> >     i = 100
> >     if x:
> >         j = [i for i in range(10)]
> >         return i
> >     else:
> >         return i
>
> A simpler example:
>
> def foo():
>     i = 100
>     j = [i for i in range(10)]
>     return i
>
> In Python 3, foo() returns 100; in Python 2, it returns 9.

You did not get the point.

Converting my example to your format:

def foo_steven(n):
i = 100
j = [i for i in range(n)]
return i

$ python3
Python 3.2.3 (default, Jun 26 2012, 00:38:09)
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo_steven(n):
... i = 100
... j = [i for i in range(n)]
... return i
...
>>> foo_steven(0)
100
>>> foo_steven(4)
100
>>>
$ python
Python 2.7.3rc2 (default, Apr 22 2012, 22:35:38)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo_steven(n):
... i = 100
... j = [i for i in range(n)]
... return i
...
>>> foo_steven(0)
100
>>> foo_steven(3)
2
>>>

Python 2:
When n>0 comprehension scope i is returned
When n=0 function scope i is returned

Python 3: The return statement is lexically outside the comprehension
and so that outside-scope's i is returned in all cases.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[Steven D'Aprano]

On Fri, 13 Jul 2012 19:31:24 -0700, rusi wrote:

> Consider the following
> 
> def foo(x):
> i = 100
> if x:
> j = [i for i in range(10)]
> return i
> else:
> return i

A simpler example:

def foo():
i = 100
j = [i for i in range(10)]
return i

In Python 3, foo() returns 100; in Python 2, it returns 9.


> In python 2  two different 'i's could be returned. In python3 only one i
> can be returned.
> One could call it dynamic binding. 

One could also call it a tractor, but it isn't either of those things.

The difference is whether or not list comprehensions create their own 
scope. In Python 2, they don't; in Python 3, they do. Personally I don't 
have an opinion as to which is better, but in neither case does this have 
any thing to do with lexical versus dynamic binding.


> Evidently Guido thinks it a bug which is why he changed it.

It was a case that either list comps be changed to *not* expose their 
loop variable, or generator expressions be changed *to* expose their loop 
variable. The Python 2 situation where list comps and gen expressions 
have opposite behaviour was unfortunate.


> The leakage of i in the OPs question is the same kind of bug.

Not at all. The OP was *deliberately* creating a closure using i -- under 
those circumstances, it would be a bug if i *didn't* leak.

The OP's gotcha was:

1) he expected the closure to use early binding, where i in each function 
was bound to the value of i in the enclosing scope at the moment the 
function was defined; 

2) but Python actually uses late binding for closures, where the value of 
i in each function is set to the value of i in the enclosing scope when 
the function is called.

As far as I can tell, Python always uses late binding for scopes; the 
only time it does early binding is for default values of function 
parameters.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: howto do a robust simple cross platform beep

[Steven D'Aprano]

On Sat, 14 Jul 2012 03:00:05 +0200, Gelonida N wrote:

> How do others handle simple beeps?
> 
> I just want to use them as alert, when certain events occur within a
> very long running non GUI application.

Why? Do you hate your users?


> What I do at the moment is:
> 
> For Windows I use winsound.Beep
> 
> For Linux I create some raw data and pipe it into sox's 'play' command.
> 
> I don't consider this very elegant.

There is no cross-platform way to play a beep.

Every few years, people complain that Python doesn't have a standard way 
to play a simple alert sound. Why ask for volunteers to write and 
maintain the code, and suddenly they go silent.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[rusi]

On Jul 13, 10:53 pm, Hans Mulder  wrote:
> If you add `global VERBOSE` to `caller`, then there is only one
> variable named `VERBOSE` and what `function` does, depends on
> the most recent assignment to that variable.
>
> If you remove your `global VERBOSE`, then there are two
> variables by that name, one global and one local to `caller`.
> In that case, there is the question of which one `function`
> will use.

Good point. See below.

>
> The function `function` refers to a variable `VERBOSE` that
> isn't local.  In some programming langauages, the interpreter
> would then scan the call stack at run-time, looking for a scope
> where that name is defined.  It would find the local one in
> `caller`.  This is known as "dynamic binding".
>
> Other interpreters use the `VERBOSE` that was in scope at
> the point in the program text where `function` was defined.
> In this case, that would be the global one.  This is called
> "lexical binding".
>
> Some programming languages allow you to indicate on a per-
> variable basis whether you want dynamic or lexical binding.
>
> Python is firmly in the lexical camp.  Dynamic binding is not
> available in Python, and never will be.

Thats a good intention.  The question is whether it is uniformly
realized.

Consider the following

def foo(x):
i = 100
if x:
j = [i for i in range(10)]
return i
else:
return i

In python 2  two different 'i's could be returned. In python3 only one
i can be returned.
One could call it dynamic binding. Evidently Guido thinks it a bug
which is why he changed it.

The leakage of i in the OPs question is the same kind of bug.

tl;dr version:
This is 2012. Dynamic binding is considered a bug even by the lisp
community where it originated.
Lets at least call it a feature and a gotcha
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to safely maintain a status file

[Steven D'Aprano]

On Fri, 13 Jul 2012 15:15:13 -0500, Chris Gonnerman wrote:

> On 07/13/2012 12:59 PM, Prasad, Ramit wrote:
>> I lean slightly towards the POSIX handling with the addition that any
>> additional write should throw an error. You are now saving to a file
>> that will not exist the moment you close it and that is probably not
>> expected. Ramit
> But if I created, then deleted it while holding an open file descriptor,
> it is entirely likely that I intend to write to it. I'll admit, these
> days there are those in the Unix/Linux community that consider using an
> anonymous file a bad idea; I'm just not one of them.

A badly-behaved application can write oodles and oodles of data to an 
unlinked file, which has the result of temporarily using up disk space 
that doesn't show up when you do an ls. For an inexperienced system 
administrator, this may appear mysterious.

The solution is to us lsof to identify the unlinked file, which gives you 
the process id of the application, which you can then kill. As soon as 
you do that, the space is freed up again.

Like all powerful tools, unlinked files can be abused. Underpowered tools 
can't be abused, but nor can they be used.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

howto do a robust simple cross platform beep

[Gelonida N]

Hi,


I just want to use a beep command that works cross platform.


I tried the simplest approach (just printing the BEL character '\a' 
chr(7) to the console.



This fails on my Ubuntu 12.04 host, as the pcspkr is in the list of the 
blacklisted kernel modules.


I found another snippet trying to push a sine wave  directly to /dev/audio

but I don't have write permissions to /dev/audio.

Other solutions seem to suggest to play a wav file, but of course first 
I had to write code creating me a wav file.


How do others handle simple beeps?


I just want to use them as alert, when certain events occur within a 
very long running non GUI application.



Thanks for any info.


What I do at the moment is:

For Windows I use winsound.Beep

For Linux I create some raw data and pipe it into sox's
'play' command.

I don't consider this very elegant.











--
http://mail.python.org/mailman/listinfo/python-list

Re: Python professional certification

[Ethan Furman]

Mark Lawrence wrote:
Google tells me that various certifications are available but I'd like 
to know if any of these are approved by the PSF or whoever would be 
responsible?  If there's anything out there I've missed it :-(




There is an O'Reilly Python Certification class offered in conjunction 
with the Illinois Institute of Technology (or something like that) which 
was created by Steve Holden, and taught by him and a couple others.


It's a great course of four classes.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter.event.widget: handler gets name instead of widget.

[Terry Reedy]

On 7/13/2012 4:24 PM, Frederic Rentsch wrote:

On Fri, 2012-07-13 at 09:26 +0200, Peter Otten wrote:



Another random idea: run your code on a more recent python/tcl installation.


That might have been clearer as python + tcl/tk installation.


I next spent a day with an attempt to upgrade to Python 2.7.3,
figuring that that might simultaneously take care of upgrading tcl.


No, two completely separate actions.


... build finished, but the necessary bits to build these modules were
not found:

_bsddb
_curses
_curses_panel
_sqlite3
_ssl
_tkinter
bsddb185
bz2
dbm
gdbm
readline
sunaudiodev


I believe _tkinter is the only one of those you need to run idle.

You need tcl/tk installed/compiled first to compile python with 
_tkinter. Easier on *nix than windows. Many *nix systems come with 
tcl/tk or easily install it with their package managers (same with some 
of the other prerequisites for other modules).


--
Terry Jan Reedy



--
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[Terry Reedy]

From now on, for each operator I would have to remember wether it
is a supposedly comparison operator or not.


I believe the following rule is true: if a op b is True or False raises, 
then op is a potentially chained comparison operation. They are (not) 
equal (and (not) is), the 4 order comparisons, and (not) in. 'in' should 
be the only surprise and most confusing.

>>> 1 < 3 in {3,4}
True
>>> 3 in {3,4} < {4}
False

'and' and 'or' are not included because they do not always return a 
bool, and indeed, they are not binary operators in the usual sense 
because of short-circuiting.


--
Terry Jan Reedy



--
http://mail.python.org/mailman/listinfo/python-list

Python professional certification

[Mark Lawrence]

Google tells me that various certifications are available but I'd like 
to know if any of these are approved by the PSF or whoever would be 
responsible?  If there's anything out there I've missed it :-(


--
Cheers.

Mark Lawrence.


--
http://mail.python.org/mailman/listinfo/python-list

ANN: Urwid 1.0.2

[Ian Ward]

Announcing Urwid 1.0.2
--

Urwid home page:
  http://excess.org/urwid/

Manual:
  http://excess.org/urwid/wiki/UrwidManual

Tarball:
  http://excess.org/urwid/urwid-1.0.2.tar.gz


About this release:
===

This is a stable bug-fix-only release.

A number of bugs that could cause Urwid to crash in normal use have
been fixed.  Upgrading is recommended.


New in 1.0.2:
=

 * Fix for a bug when entering Unicode text into an Edit widget with
   a bytes caption

 * Fix a regression when not running in UTF-8 mode

 * Fix for a MainLoop.remove_watch_pipe() bug

 * Fix for a bug when packing empty Edit widgets

 * Fix for a ListBox "contents too long" error with very large
   Edit widgets

 * Prevent ListBoxes from selecting 0-height selectable widgets
   when moving up or down

 * Fix a number of bugs caused by 0-height widgets in a ListBox


About Urwid
===

Urwid is a console UI library for Python. It features fluid interface
resizing, Unicode support, multiple text layouts, simple attribute
markup, powerful scrolling list boxes and flexible interface design.

Urwid is released under the GNU LGPL.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and Qt4 Designer

[Jean Dubois]

Op vrijdag 13 juli 2012 03:52:51 UTC+2 schreef Vincent Vande Vyvre het volgende:
> On 12/07/12 08:42, Jean Dubois wrote:
> > On 12 jul, 02:59, Vincent Vande Vyvre 
> > wrote:
> >> On 11/07/12 17:37, Jean Dubois wrote:
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>> I'm trying to combine python-code made with QT4 designer 
> with plain
> >>> python statements like
> >>> file = open("test","w")
> >>> Can anyone tell me what I have to  add to the following code 
> just to
> >>> open a file when clicking on the load-button and closing it by
> >>> clicking on the save button.
> >>> #!/usr/bin/env python
> >>> # -*- coding: utf-8 -*-
> >>> # Form implementation generated from reading ui file 
> 'test.ui'
> >>> #
> >>> # Created: Wed Jul 11 17:21:35 2012
> >>> #  by: PyQt4 UI code generator 4.8.3
> >>> #
> >>> # WARNING! All changes made in this file will be lost!
> >>> from PyQt4 import QtCore, QtGui
> >>> try:
> >>> _fromUtf8 = QtCore.QString.fromUtf8
> >>> except AttributeError:
> >>> _fromUtf8 = lambda s: s
> >>> class Ui_Form(object):
> >>> def setupUi(self, Form):
> >>> Form.setObjectName(_fromUtf8("Form"))
> >>> Form.resize(400, 300)
> >>> self.widget = QtGui.QWidget(Form)
> >>> self.widget.setGeometry(QtCore.QRect(10, 20, 146, 25))
> >>> self.widget.setObjectName(_fromUtf8("widget"))
> >>> self.horizontalLayout = QtGui.QHBoxLayout(self.widget)
> >>> self.horizontalLayout.setMargin(0)
> >>> 
> self.horizontalLayout.setObjectName(_fromUtf8("horizontalLayout"))
> >>> self.pushButton_2 = QtGui.QPushButton(self.widget)
> >>> 
> self.pushButton_2.setObjectName(_fromUtf8("pushButton_2"))
> >>> self.horizontalLayout.addWidget(self.pushButton_2)
> >>> self.pushButton = QtGui.QPushButton(self.widget)
> >>> 
> self.pushButton.setObjectName(_fromUtf8("pushButton"))
> >>> self.horizontalLayout.addWidget(self.pushButton)
> >>> self.retranslateUi(Form)
> >>> QtCore.QMetaObject.connectSlotsByName(Form)
> >>> def retranslateUi(self, Form):
> >>> 
> Form.setWindowTitle(QtGui.QApplication.translate("Form",
> >>> "Form", None, QtGui.QApplication.UnicodeUTF8))
> >>> 
> self.pushButton_2.setText(QtGui.QApplication.translate("Form",
> >>> "Save file", None, QtGui.QApplication.UnicodeUTF8))
> >>> 
> self.pushButton.setText(QtGui.QApplication.translate("Form",
> >>> "Load file", None, QtGui.QApplication.UnicodeUTF8))
> >>> if __name__ == "__main__":
> >>> import sys
> >>> app = QtGui.QApplication(sys.argv)
> >>> Form = QtGui.QWidget()
> >>> ui = Ui_Form()
> >>> ui.setupUi(Form)
> >>> Form.show()
> >>> sys.exit(app.exec_())
> >>> thanks in advance
> >>> jean
> >> Connect the signal clicked of your's buttons to your's 
> functions.
> >>
> >> self.pushButton.clicked.connect(self.my_func)
> >>
> >> Here's all the truth:
> >>
> >> 
> http://www.riverbankcomputing.co.uk/static/Docs/PyQt4/html/new_style_...
> >>
> >> --
> >> Vincent V.V.
> >> Oqapy ; . Qarte+7
> >> ; . PaQager 
> ;
> > thanks for the reference, could you just supply a small example for
> > the code above to get me started?
> >
> > thanks in advance
> > jean
> Just add the connection at the end of the Ui_Form class and, of course,
> your function.
> 
> You can find numbers of examples in your PyQt4 install folder.
> On my machine is located at /usr/share/doc/python-qt4-doc/examples
> 
> And, for more inspiration, have a look at this site:
>http://diotavelli.net/PyQtWiki/
> 
> -- 
> Vincent V.V.
> Oqapy ; . Qarte
> ; . PaQager 
> ;

Thanks for the extra docu references

regards,

jean
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter.event.widget: handler gets name instead of widget.

[Frederic Rentsch]

On Fri, 2012-07-13 at 09:26 +0200, Peter Otten wrote:
> Frederic Rentsch wrote:
> 
> > I'm sorry I can't post an intelligible piece that does NOT work. I
> > obviously can't post the whole thing. 
> 
> How about a pastebin then? Or even bitbucket/github as you need to track 
> changes anyway?
> 
> > It is way too convoluted.
> 
> "Convoluted" code is much easier to debug than no code ;)
> 
> Another random idea: run your code on a more recent python/tcl installation. 
> If you are lucky you get a different error.
> 

So many good ideas! I can hardly keep up. Let me try anyway.

I hesitate to ask dumb questions, but I guess I have to. What is
python/tcl? I enlisted Google, Synaptic, apt-cache, apt-get, dpkg and
scouring the profusion I couldn't detect any actionable piece of
information, undoubtedly due to my modest expertise in matters of system
administration.
   I next spent a day with an attempt to upgrade to Python 2.7.3,
figuring that that might simultaneously take care of upgrading tcl.
Accustomed to installing packages I had to venture into the unknown
territory of compiling source, because no package was available.
(Windows, Apple, yes. Linux, no). The compile went smoothly, but ended
like this:

... build finished, but the necessary bits to build these modules were
not found:

_bsddb
_curses
_curses_panel
_sqlite3
_ssl
_tkinter
bsddb185
bz2
dbm
gdbm
readline
sunaudiodev

To find the necessary bits, look in setup.py in detect_modules() for the
module's name.

I didn't know what to look for in setup.py, spent a couple of hours
turning stones and encountered evidence of a bunch of missing header
files, probably of other modules which I had installed rather than
compiled.
   2.7.3 came up in terminals, but not in an IDLE window. No wonder,
_tkinter was reported not found; and so many others with it that,
anxious to get on, I stopped venturing further into this labyrinth,
erased everything 2.7.3 and now I'm back to 2.6 and would greatly
appreciate advice on upgrading python/tcl.

I shall look at pastebin and bitbucket/github right away.

Frederic


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python] RE: How to safely maintain a status file

[Christian Heimes]

Am 13.07.2012 21:57, schrieb MRAB:
> It's possible to create a temporary file even in Windows.

Windows has a open() flag named O_TEMPORARY for temporary files. With
O_TEMPORARY the file is removed from disk as soon as the file handle is
closed. On POSIX OS it's common practice to unlink temporary files
immediately after the open() call.

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: How to safely maintain a status file

[Chris Gonnerman]

On 07/13/2012 12:59 PM, Prasad, Ramit wrote:
I lean slightly towards the POSIX handling with the addition that any 
additional write should throw an error. You are now saving to a file 
that will not exist the moment you close it and that is probably not 
expected. Ramit
But if I created, then deleted it while holding an open file descriptor, 
it is entirely likely that I intend to write to it. I'll admit, these 
days there are those in the Unix/Linux community that consider using an 
anonymous file a bad idea; I'm just not one of them.


-- Chris.


--
http://mail.python.org/mailman/listinfo/python-list

Re: [Python] RE: How to safely maintain a status file

[MRAB]

On 13/07/2012 19:28, Hans Mulder wrote:

On 13/07/12 19:59:59, Prasad, Ramit wrote:


I lean slightly towards the POSIX handling with the addition that
any additional write should throw an error. You are now saving to
a file that will not exist the moment you close it and that is
probably not expected.



Strictly speaking, the file does exist, it's just that there are no
names referring to it. When any handles to it are also closed, the file
_can_ truly be deleted.

As has been said before, in the *nix world, "unlink" _doesn't_ delete
a file, it deletes a name.


I'd say: it depends.

If the amount of data your script needs to process does not fit
in RAM, then you may want to write some of it to a temporary file.
On a Posix system, it's entirely normal to unlink() a temp file
first thing after you've created it.  The expectation is that the
file will continue to exists, and be writeable, until you close it.

In fact, there's a function in the standard library named
tempfile.TemporaryFile that does exactly that: create a file
and unlink it immediately.  This function would be useless
if you couldn't write to your temporary file.


It's possible to create a temporary file even in Windows.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Keeping the Console Open with IDLE

[Rodrick Brown]

I think you can use pythonw.exe which will read stdin and for any
input before closing.

(I read this a while back, ma guy here.)

Sent from my iPhone

On Jul 13, 2012, at 7:27 AM, "summerholidaylearn...@gmail.com"
 wrote:

> On Friday, February 20, 2009 4:06:42 AM UTC, W. eWatson wrote:
>> I'm using IDLE for editing, but execute programs directly. If there are
>> execution or "compile" errors, the console closes before I can see 
>> what it
>> contains. How do I prevent that?
>> --
>>W. eWatson
>>
>>  (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
>>   Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 
>> feet
>>
>> Web Page: 
>
> Thanks this solved my problem too(the same issue - I was also double clicking 
> instead of right clicking - edit with IDLE - F5 to run)
> --
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[Hans Mulder]

On 13/07/12 20:54:02, Ian Kelly wrote:
> I've also seen the distinction described as "early" vs. "late" binding
> on this list, but I'm not sure how precise that is -- I believe that
> terminology more accurately describes whether method and attribute
> names are looked up at compile-time or at run-time, late binding being
> the feature that makes duck typing possible.

I think that these terms describe the problem at the start of
this thread: the OP was expecting early binding.  However, Python
does late binding, or "acts funny" as it says in the subject line.


-- HansM


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[Ian Kelly]

On Fri, Jul 13, 2012 at 11:53 AM, Hans Mulder  wrote:
> The function `function` refers to a variable `VERBOSE` that
> isn't local.  In some programming langauages, the interpreter
> would then scan the call stack at run-time, looking for a scope
> where that name is defined.  It would find the local one in
> `caller`.  This is known as "dynamic binding".
>
> Other interpreters use the `VERBOSE` that was in scope at
> the point in the program text where `function` was defined.
> In this case, that would be the global one.  This is called
> "lexical binding".
>
> Some programming languages allow you to indicate on a per-
> variable basis whether you want dynamic or lexical binding.
>
> Python is firmly in the lexical camp.  Dynamic binding is not
> available in Python, and never will be.

I don't believe that dynamic vs. lexical binding is what rusi was
attempting to describe.  If he was, then Python and Haskell would be a
bad comparison since both are lexical.  Rather, I think what he was
trying to show was capture by reference vs. capture by value in the
context of closures.  Python uses capture by reference, and so the
upvalue is the value of that reference at the time the closure is
called.  Haskell uses capture by value, and the upvalue is the value
at the time of definition.

I've also seen the distinction described as "early" vs. "late" binding
on this list, but I'm not sure how precise that is -- I believe that
terminology more accurately describes whether method and attribute
names are looked up at compile-time or at run-time, late binding being
the feature that makes duck typing possible.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python] RE: How to safely maintain a status file

[Hans Mulder]

On 13/07/12 19:59:59, Prasad, Ramit wrote:

> I lean slightly towards the POSIX handling with the addition that 
> any additional write should throw an error. You are now saving to 
> a file that will not exist the moment you close it and that is
> probably not expected.

I'd say: it depends.

If the amount of data your script needs to process does not fit
in RAM, then you may want to write some of it to a temporary file.
On a Posix system, it's entirely normal to unlink() a temp file
first thing after you've created it.  The expectation is that the
file will continue to exists, and be writeable, until you close it.

In fact, there's a function in the standard library named
tempfile.TemporaryFile that does exactly that: create a file
and unlink it immediately.  This function would be useless
if you couldn't write to your temporary file.

Hope this helps,

-- HansM


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python] RE: How to safely maintain a status file

[Chris Angelico]

On Sat, Jul 14, 2012 at 3:59 AM, Prasad, Ramit
 wrote:
> I lean slightly towards the POSIX handling with the addition that
> any additional write should throw an error. You are now saving to
> a file that will not exist the moment you close it and that is probably
> not expected.

There are several different possible "right behaviors" here, but they
depend more on the application than anything else. With a log file,
for instance, the act of deleting it is more a matter of truncating it
(dispose of the old history), so the right thing to do is to start a
fresh file. Solution: Close the file and re-open it periodically. But
I don't know of an efficient way to do that with Windows semantics.
Renaming/moving an open file in order to perform log rotation isn't
all that easy.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: lambda in list comprehension acting funny

[Prasad, Ramit]

> >> VERBOSE = True
> >>
> >> def function(arg):
> >> if VERBOSE:
> >>print("calling function with arg %r" % arg)
> >> process(arg)
> >>
> >> def caller():
> >> VERBOSE = False
> >> function(1)
> >>
> >> -
> >> Python semantics: function sees VERBOSE False
> >> Haskell semantics: function sees VERBOSE True
> 
>  def caller():
> > ... VERBOSE = False
> > ... function(1)
>  def function(arg):
> > ... if VERBOSE:
> > ...print("calling function with arg %r" % arg)
> > ...
>  VERBOSE = True
>  caller()
> > calling function with arg 1
> >
> > I might be being OCD, but caller needs `global VERBOSE` for that to
> > work as you explain.
> 
> That would be quite different from what Rusi is after.
> 
> If you add `global VERBOSE` to `caller`, then there is only one
> variable named `VERBOSE` and what `function` does, depends on
> the most recent assignment to that variable.
> 
> If you remove your `global VERBOSE`, then there are two
> variables by that name, one global and one local to `caller`.
> In that case, there is the question of which one `function`
> will use.
>

But that is not what Rusi writes.
"Python semantics: function sees VERBOSE False" <- function
will not see False without the use of global. 

> The function `function` refers to a variable `VERBOSE` that
> isn't local.  In some programming langauages, the interpreter
> would then scan the call stack at run-time, looking for a scope
> where that name is defined.  It would find the local one in
> `caller`.  This is known as "dynamic binding".
> 
> Other interpreters use the `VERBOSE` that was in scope at
> the point in the program text where `function` was defined.
> In this case, that would be the global one.  This is called
> "lexical binding".
> 
> Some programming languages allow you to indicate on a per-
> variable basis whether you want dynamic or lexical binding.
> 
> Python is firmly in the lexical camp.  Dynamic binding is not
> available in Python, and never will be.

True and a good explanation, but not what I understood
Rusi to mean.

Ramit



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: [Python] RE: How to safely maintain a status file

[Prasad, Ramit]

> >> Well "neat tricks" aside, I am of the firm belief that deleting files
> should
> >> never be possible whilst they are open.
> > This is one of the few instances I think Windows does something better
> > than OS X. Windows will check before you attempt to delete (i.e. move
> > to Recycling Bin) while OS X will move a file to Trash quite happily
> > only tell me it cannot remove the file when I try to empty the Trash.
> While I was trained in the Unix way, and believe it is entirely
> appropriate to delete an open file.  Even if I my program is the opener.
>   It's just too handy to have temp files that disappear on their own.
> 
> As opposed to periodically going to %TEMP% and deleting them manually.  Gah.

In my experience things that are "too handy" are usually breaking
what I consider "right". That being said, I am not entirely sure
what I think is "right" in this circumstance. I suppose it depends
on if I am the person deleting or the person who is looking at
a file that is being deleted. Or the user who just wants the stupid
computer to just Work.

I lean slightly towards the POSIX handling with the addition that 
any additional write should throw an error. You are now saving to 
a file that will not exist the moment you close it and that is probably 
not expected.





Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[Hans Mulder]

On 13/07/12 18:12:40, Prasad, Ramit wrote:
>> VERBOSE = True
>>
>> def function(arg):
>> if VERBOSE:
>>print("calling function with arg %r" % arg)
>> process(arg)
>>
>> def caller():
>> VERBOSE = False
>> function(1)
>>
>> -
>> Python semantics: function sees VERBOSE False
>> Haskell semantics: function sees VERBOSE True

 def caller():
> ... VERBOSE = False
> ... function(1)
 def function(arg):
> ... if VERBOSE:
> ...print("calling function with arg %r" % arg)
> ... 
 VERBOSE = True
 caller()
> calling function with arg 1
> 
> I might be being OCD, but caller needs `global VERBOSE` for that to
> work as you explain.

That would be quite different from what Rusi is after.

If you add `global VERBOSE` to `caller`, then there is only one
variable named `VERBOSE` and what `function` does, depends on
the most recent assignment to that variable.

If you remove your `global VERBOSE`, then there are two
variables by that name, one global and one local to `caller`.
In that case, there is the question of which one `function`
will use.

The function `function` refers to a variable `VERBOSE` that
isn't local.  In some programming langauages, the interpreter
would then scan the call stack at run-time, looking for a scope
where that name is defined.  It would find the local one in
`caller`.  This is known as "dynamic binding".

Other interpreters use the `VERBOSE` that was in scope at
the point in the program text where `function` was defined.
In this case, that would be the global one.  This is called
"lexical binding".

Some programming languages allow you to indicate on a per-
variable basis whether you want dynamic or lexical binding.

Python is firmly in the lexical camp.  Dynamic binding is not
available in Python, and never will be.


Hope this helps,

-- HansM


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python] RE: How to safely maintain a status file

[Chris Gonnerman]

On 07/13/2012 11:00 AM, Prasad, Ramit wrote:

Well "neat tricks" aside, I am of the firm belief that deleting files should
never be possible whilst they are open.

This is one of the few instances I think Windows does something better
than OS X. Windows will check before you attempt to delete (i.e. move
to Recycling Bin) while OS X will move a file to Trash quite happily
only tell me it cannot remove the file when I try to empty the Trash.
While I was trained in the Unix way, and believe it is entirely 
appropriate to delete an open file.  Even if I my program is the opener. 
 It's just too handy to have temp files that disappear on their own.


As opposed to periodically going to %TEMP% and deleting them manually.  Gah.

-- Chris.
--
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[Chris Angelico]

On Sat, Jul 14, 2012 at 2:46 AM, rusi  wrote:
> Ok let me restate: if python were to work that way (without the
> global) we could say either
> a Python chooses to have dynamic scoping of variables
> or
> b There is a bug in python's scoping rules

Or c, there's a declaration at the top:

from __future__ import variable_declarations

Like braces, it's a MASSIVE improvement to the language, and it's part
of a huge conspiracy that it isn't there by default. But all it takes
is a little future statement and you're on Python 4000!



ChrisA
(I shouldn't post at half past three in the morning. I get stupid... er.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: adding a simulation mode

[Chris Angelico]

On Sat, Jul 14, 2012 at 3:08 AM, Prasad, Ramit
 wrote:
> I would say the opposite. In production code usually I want it
> to recover, log as much information as I need (including sending
> any notifications), and NOT just die.
>
> In development, not catching the exception will give me a full
> trace back automatically.

Here's another take on the matter. In development, your script is your
execution unit, so you let the interpreter print out your tracebacks.
In production, there will usually be one, maybe two subunits (for
instance, a TCP-based server might have the socket connection as an
execution unit, and possibly a command parser inside that), and at the
top of that subunit, you have a broad exception handler that resets
that one unit (goes back and accepts another client, or waits for
another command). Otherwise, wide-scope exception handling is usually
a bad thing.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: adding a simulation mode

[Prasad, Ramit]

> > Please do NOT catch BaseException, since that is the wrong thing to do.
> 
> I would agree if you had said "in production code".
> 
> If you are investigating why a third-party function is stopping your
> interpreter, then catching BaseException may tell you that the code
> is raising the wrong kind of Exception.  Once you know what kind the
> function is raising, you should catch only that particular excpetion
> subclass.

I would say the opposite. In production code usually I want it
to recover, log as much information as I need (including sending
any notifications), and NOT just die.

In development, not catching the exception will give me a full 
trace back automatically. Why bother with trying to catch and 
print something when the interpreter will do it for me? Not
to mention that removes any hassle of trying to catch the
"right" exception or figuring out the best way to print it.
I suppose if there are arguments on the exception that were not
printed then I might want to catch it, but has been rare
in my experience.

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[rusi]

On Jul 13, 9:12 pm, "Prasad, Ramit"  wrote:
> > VERBOSE = True
>
> > def function(arg):
> >     if VERBOSE:
> >        print("calling function with arg %r" % arg)
> >     process(arg)
>
> > def caller():
> >     VERBOSE = False
> >     function(1)
>
> > -
> > Python semantics: function sees VERBOSE False
> > Haskell semantics: function sees VERBOSE True
> >>> def caller():
>
> ...     VERBOSE = False
> ...     function(1)>>> def function(arg):
>
> ...     if VERBOSE:
> ...        print("calling function with arg %r" % arg)
> ...    >>> VERBOSE = True
> >>> caller()
>
> calling function with arg 1
>
> I might be being OCD, but caller needs `global VERBOSE` for that to
> work as you explain.

Ok let me restate: if python were to work that way (without the
global) we could say either
a Python chooses to have dynamic scoping of variables
or
b There is a bug in python's scoping rules

I would guess that most younger folks (who've not seen lisp and apl)
would choose b

We can say the same analogously in the context of ZF expressions when
the i leaks as it does in the OPs example.

Another tack on the same question: python 3 cleaned up the variable
leakage from inside ZFs to outside.  It missed cleaning up the leakage
from one step to next.

tl;dr version: Beware of mixing up functional and imperative
programming
Double-beware when you are a language designer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: adding a simulation mode

[Hans Mulder]

On 13/07/12 04:16:53, Steven D'Aprano wrote:
> On Thu, 12 Jul 2012 16:37:42 +0100, andrea crotti wrote:
> 
>> 2012/7/12 John Gordon :
>>> In  andrea crotti
>>>  writes:
>>>
 Well that's what I thought, but I can't find any explicit exit
 anywhere in shutil, so what's going on there?
>>>
>>> Try catching SystemExit specifically (it doesn't inherit from
>>> Exception, so "except Exception" won't catch it.)
>>>
>>
>> Ah yes that actually works, but I think is quite dodgy, why was it done
>> like this?

It may be that the function you're calling found a problem that the
author thinks is so grave that they shouldn't give you an opportunity
to deal with it.

If that's the case, I would be inclined to think that they are wrong.

> Built-in exceptions SystemExit, KeyboardInterrupt and GeneratorExit 
> deliberately do not inherit from Exception since they are not meant to be 
> caught by "catch-all" try...except Exception clauses.
> 
> You can see the exception hierarchy here:
> 
> http://docs.python.org/library/exceptions.html#exception-hierarchy
> 
> Please do NOT catch BaseException, since that is the wrong thing to do. 

I would agree if you had said "in production code".

If you are investigating why a third-party function is stopping your
interpreter, then catching BaseException may tell you that the code
is raising the wrong kind of Exception.  Once you know what kind the
function is raising, you should catch only that particular excpetion
subclass.


> If you must catch SystemExit, KeyboardInterrupt, etc. they you should do 
> so as separate catch clauses:
> 
> try:
> main()
> except SystemExit as e:
> print(e)  # see if we can find out who is raising this

If you want to find out who is raising the exception, you could
try this:

except SystemExit:
import traceback
traceback.print_exc()

That will print a complete stack trace.

If you only need to know what kind of exception you have,
you can do:

print(repr(e))

A simple print(e) will print str(e), which in the case of
SystemExit, is an empty string.  That's not very informative.

> except KeyboardInterrupt:
> print("Mwahahaha my pretty, you cannot cancel this!!!")
> print("...er, now what do I do?")
> except Exception:
> print("why am I catching exceptions I can't recover from?")


Hope this helps,

-- HansM
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[rusi]

On Jul 13, 8:36 pm, Chris Angelico  wrote:
> On Sat, Jul 14, 2012 at 1:04 AM, Steven D'Aprano
>
>  wrote:
> > Actually, no. Is True less than False, or is it greater? In boolean
> > algebra, the question has no answer. It is only an implementation detail
> > of Python that chooses False < True.
>
> Maybe in boolean algebra, but in code, it's handy to have sortable
> bools. In SQL, for instance, I can use a boolean in an ORDER BY,
> perhaps followed by another criterion, and it's effectively sorting by
> "1 if some_boolean else 0" or in C notation "some_boolean ? 0 : 1"
>
> ChrisA

Actually a boolean algebra is a lattice with some additional
properties
A lattice is a poset (partially ordered set) with suprema and infimas
And so there is one natural order relation on any boolean algebra
which may be defined as
a ≤ b iff a ⋀ b = a

tl;dr version: In a boolean algebra, False is bottom and True is top
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: lambda in list comprehension acting funny

[Prasad, Ramit]

> VERBOSE = True
> 
> def function(arg):
> if VERBOSE:
>print("calling function with arg %r" % arg)
> process(arg)
> 
> def caller():
> VERBOSE = False
> function(1)
> 
> -
> Python semantics: function sees VERBOSE False
> Haskell semantics: function sees VERBOSE True

>>> def caller():
... VERBOSE = False
... function(1)
>>> def function(arg):
... if VERBOSE:
...print("calling function with arg %r" % arg)
... 
>>> VERBOSE = True
>>> caller()
calling function with arg 1

I might be being OCD, but caller needs `global VERBOSE` for that to
work as you explain.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: How to safely maintain a status file

[Prasad, Ramit]

> Well "neat tricks" aside, I am of the firm belief that deleting files should
> never be possible whilst they are open.

This is one of the few instances I think Windows does something better 
than OS X. Windows will check before you attempt to delete (i.e. move
to Recycling Bin) while OS X will move a file to Trash quite happily
only tell me it cannot remove the file when I try to empty the Trash.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[Roy Smith]

On Friday, July 6, 2012 9:58:10 AM UTC-4, Steven D'Aprano wrote:
> (Sadly, when I say "we" I mean 
> collectively. Many language designers, and programmers, don't have the 
> foggiest clue as to what makes a good clean design. Hence C++ and PHP.)

I'm not going to defend C++, but to be fair, a major driver of the design is 
that it had to be plug-compatible with C.  From that you're stuck with the 
preprocessor and pointers.  Much goes downhill when you start from there.

PHP, yeah, that's just charlie-foxtrot from start to finish.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[Chris Angelico]

On Sat, Jul 14, 2012 at 1:04 AM, Steven D'Aprano
 wrote:
> Actually, no. Is True less than False, or is it greater? In boolean
> algebra, the question has no answer. It is only an implementation detail
> of Python that chooses False < True.

Maybe in boolean algebra, but in code, it's handy to have sortable
bools. In SQL, for instance, I can use a boolean in an ORDER BY,
perhaps followed by another criterion, and it's effectively sorting by
"1 if some_boolean else 0" or in C notation "some_boolean ? 0 : 1"

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[Steven D'Aprano]

On Fri, 13 Jul 2012 12:30:47 +, Albert van der Horst wrote:

>>Apart from Python, Mathematica, Perl 6, CoffeeScript, Cobra and Clay
>>give chained comparisons the standard meaning. It is, or was, a feature
>>request for Boo, but I can't tell whether it has been implemented or
>>not.
> 
> Algol 68 does not. It has promoted operator symbols to first class
> citizens. In that context chained comparison operators cannot be made to
> work.
> Both Mathematica and Perl are ad-hoc-ish languages. I would hate Python
> go that way.

Perl 5 does not have chained comparisons. Perl 6, which is more designed 
and less ad-hoc, does.

> From now on, for each operator I would have to remember wether it is a
> supposedly comparison operator or not.

Are you often in the habit of using operators *without* remembering what 
they do? 

I can't speak for you, but it isn't often that I've found myself not 
remembering whether the less-than operator <  means "compare two values" 
or "multiply two values".


> Comparison operations on booleans make sense.

Actually, no. Is True less than False, or is it greater? In boolean 
algebra, the question has no answer. It is only an implementation detail 
of Python that chooses False < True.

Of course, equality and inequality make sense for all values. But the 
other comparison operators, < <= >= > only make sense for values which 
are ordered, like real numbers (but not complex numbers), strings, and 
similar, or for set comparisons (subset and superset). Arbitrary types 
may not define comparison operators.


> I remember a FORTRAN
> compiler complaining about me comparing LOGICALS. The lack of
> abstraction!

The opposite: LOGICALS are abstract values which do not define greater 
than or less than.

> The worst of is, of course, = for assignment instead of := . This is a
> convention that Python follows, to my dismay.

*shrug*

The worst is to use = for both equality and assignment, like some BASICs. 
At least Python does not allow assignment as an expression, so you can't 
make the typical C error of:

if x = y: do_something()  # oops meant x == y



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[rusi]

To come back to the OPs question.

Variables can be assigned. Or they can be bound.
[C++ programmers will be familiar with the difference between
initialization and assignment]

List comprehensions are defined in terms of assignment to the local
variable rather than binding.
Hence the issue.

Below are two functions that simulate mapping (lambda x: x**i) where
the i's come from some given list

def binding_version(lst):
if not lst: return []
i = lst[0]
return [(lambda x: x ** i)] + binding_version(lst[1:])

def assign_version(lst):
ret = []
for i in lst:
ret.append(lambda x: x**i)
return ret
--

>>> fs= binding_version([0,1,2])
>>> fs[0](2)
1
>>> fs[1](2)
2
>>> fs[2](2)
4

>>> fs= assign_version([0,1,2])
>>> fs[0](2)
4
>>> fs[1](2)
4
>>> fs[2](2)
4
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lambda in list comprehension acting funny

[rusi]

On Jul 13, 11:36 am, Steven D'Aprano  wrote:
> On Thu, 12 Jul 2012 21:33:40 -0700, rusi wrote:
> > On Jul 11, 11:41 am, Daniel Fetchinson 
> > wrote:
> >> funcs = [ lambda x: x**i for i in range( 5 ) ] print funcs[0]( 2 )
> >> print funcs[1]( 2 )
> >> print funcs[2]( 2 )
>
> >> This gives me
>
> >> 16
> >> 16
> >> 16
>
> >> When I was excepting
>
> >> 1
> >> 2
> >> 4
>
> >> Does anyone know why?
>
> >> Cheers,
> >> Daniel
>
> > Your expectations are reasonable.
>
> You forget to finish that sentence.
>
> "Your expectations are reasonable, for languages that don't have
> variables which can actually vary."
>
> *wink*
>
> For purely functional languages like Haskell, the behaviour you show
> below makes sense. Since Haskell doesn't allow variables to change their
> value, once a closure sees i=1 (say), then it must *always* see i=1.
>
> But that's not the case in Python, where the Haskell behaviour would be
> unfortunate. Imagine if you did this:
>
> VERBOSE = True
>
> def function(arg):
>     if VERBOSE:
>         print("calling function with arg %r" % arg)
>     process(arg)
>
> You would expect the function to honour changes to the variable VERBOSE,
> would you not? Using Python's scoping rules for closures, it does. Using
> Haskell's rules, it wouldn't.

Heres a more appropriate analog

--
VERBOSE = True

def function(arg):
if VERBOSE:
   print("calling function with arg %r" % arg)
process(arg)

def caller():
VERBOSE = False
function(1)

-
Python semantics: function sees VERBOSE False
Haskell semantics: function sees VERBOSE True
-- 
http://mail.python.org/mailman/listinfo/python-list

Initial nose experience

[Roy Smith]

I've been using unittest for many years, but have steadfastly (perhaps 
stubbornly) avoided newfangled improvements like nose.  I finally 
decided to take a serious look at nose.  There were a few pain points I 
had to work through to get our existing collection of tests to run under 
nose.  I figured I'd share them for the benefit of others who may be 
going through the same process.

First nose won't import executable files, at least not by default.

All of our test files are executable, with a "#!/usr/bin/env python" 
line at the top, and a "if __name__ == '__main__'" block, which does 
some setup and invokes unittest.main(), at the bottom.  Going to the top 
of our source tree and typing "nosetests" was an uninspiring experience:

> $ nosetests
> 
> --
> Ran 0 tests in 0.001s
> 
> OK

The fix is either make them non-executable, or do "nosetests --exe".

Next up was the the setup in the "if __name__ == '__main__'" block 
wasn't running.  The solution here is to move all the setup to 
setUpModule(), where it belongs.  SetUpModule() is new in Python 2.7 but 
it turns out it's trivial to drop that version into older systems 
(http://pypi.python.org/pypi/unittest2).

We found a bunch of tests which require some specific setup before they 
could run (most blatantly, some selenium tests which depend on X11).  
When we were running tests semi-manually, that was not a big deal.  With 
nose's "find them all and run them" strategy, this fails.  The obvious 
fix is that every test needs to either set up the right environment, or 
be protected with the appropriate @skip decorator so it doesn't run if 
the environment isn't good.

Lastly, nose, by default, doesn't say much.  When things go wrong and 
you have no clue what's happening, --verbose and --debug are your 
friends.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using a CMS for small site?

[Albert van der Horst]

In article ,
Gilles   wrote:
>Hello
>
>Someone I know with no computer knowledge has a studio appartment to
>rent in Paris and spent four months building a small site in Joomla to
>find short-time renters.
>
>The site is just...
>- a few web pages that include text (in four languages) and pictures
>displayed in a Flash slide show
>- a calendar to show availability
>- a form to send e-mail with anti-SPAM support
>- (ASAP) online payment
>
>Out of curiosity, are there CMS/frameworks in Python that can do this?
>Django? Other?
>
>Is a full-fledged CMS even needed for something like that?

Good old rcs would be fine. It is oldfashioned, so you need only
4 commands, compared to a bewildering display of icons.

mkdir RCS
ci *
rcs -NSTABLE1: RCS/*

Backup by
tar cf /media/MYSTICK/project.tar RCS

>
>Thank you.


--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: code review

[Albert van der Horst]

In article <4ff0f8e0$0$29988$c3e8da3$54964...@news.astraweb.com>,
Steven D'Aprano   wrote:
>On Sun, 01 Jul 2012 05:55:24 -0400, Terry Reedy wrote:
>
>> On 7/1/2012 2:54 AM, Steven D'Aprano wrote:
>>
>>> So no, Python has always included chained comparisons, and yes, it is
>>> shameful that a language would force you to unlearn standard notation
>>> in favour of a foolish consistency with other operators. Comparisons
>>> aren't special because they return bools. They are special because of
>>> the way they are used.
>>>
>>> C treats comparison operators as if they were arithmetic operators, and
>>> so the behaviour of a chained comparison is the same as the behaviour
>>> as a sequence of arithmetic operators: a foolish consistency. Python
>>> treats comparison operators as comparison operators, and gives them
>>> behaviour appropriate to comparisons.
>>
>> I considered this a great feature of Python when I first learned it.
>> Reading about how rare it is among programming languages to treat
>> comparisons in the standard way in mathematics reinforces that.
>
>Apart from Python, Mathematica, Perl 6, CoffeeScript, Cobra and Clay give
>chained comparisons the standard meaning. It is, or was, a feature
>request for Boo, but I can't tell whether it has been implemented or not.

Algol 68 does not. It has promoted operator symbols to first
class citizens. In that context chained comparison operators
cannot be made to work.
Both Mathematica and Perl are ad-hoc-ish languages. I would
hate Python go that way.

>From now on, for each operator I would have to remember wether it
is a supposedly comparison operator or not.

Comparison operations on booleans make sense.
I remember a FORTRAN compiler complaining about me
comparing LOGICALS. The lack of abstraction!

>
>
>C-like semantics are next to useless, except perhaps for obfuscation:
>
>http://stackoverflow.com/questions/4089284/why-does-0-5-3-return-true/
>
>And surprising:
>
>http://answers.yahoo.com/question/index?qid=20090923172909AA4O9Hx
>
>C-like semantics are a clear case of purity of implementation overruling
>functional usefulness.

The worst of is, of course, = for assignment instead of := .
This is a convention that Python follows, to my dismay.

>
>--
>Steven

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keeping the Console Open with IDLE

[summerholidaylearning]

On Friday, February 20, 2009 4:06:42 AM UTC, W. eWatson wrote:
> I'm using IDLE for editing, but execute programs directly. If there are 
> execution or "compile" errors, the console closes before I can see 
> what it 
> contains. How do I prevent that?
> -- 
> W. eWatson
> 
>   (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
>Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 
> feet
> 
>  Web Page: 

Thanks this solved my problem too(the same issue - I was also double clicking 
instead of right clicking - edit with IDLE - F5 to run)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mocked object returning another Mocked object

[Ulrich Eckhardt]

Am 13.07.2012 12:09, schrieb Jean-Michel Pichavant:

I have an App object with the 'target' attribute. This target is
controlling a piece of hardware. The hardware itself holds a software,
hence the target object having an 'api' attribute. I hope I make sense.

So basically I'd like

self.target.api.()

to return a Mocked object (Mocking an Api object response).

** Question **

I do I make *all* method calls return a specifik Mock?

target = Mock()

result = target.api.start()


I'm not sure where the "target" here comes in. As I understand it, the 
goal is to write the "api" object so that you can call any function on it...



I'd like result to be a Mock I defined
with the 'returnCode' attribute

print result.returnCode
1


...and every function should just return the same return code. Right?



But I would like to do it for any method of api (the list is huge,
setting each of them is not an option I think) so that in the end,

result = target.api.start()
result = target.api.stop()
result = target.api.reset()
result = target.api.loadSw()

return all the same Mock object (with 'returnCode')


There are two options I could think of:

1. Intercept attribute lookup

From the top of my head, the syntax is something like this:

class TargetMock(object):
def __getattr__(self, name):
def default_result(*vargs, **kwargs):
return ReturnCode(1)
return default_result

This just ignores the name and returns a function returning the mock 
return code. I think you get the idea.



2. Intercept individual lookups

class TargetMock(object):
def _default(self, *vargs, **kwargs):
return ReturnCode(1)
start = _default
stop = _default
reset = _default
loadSW = _default

Yes, this ignores your claim that the list of functions is too big to 
add every function individually. I'd just add them on demand when a test 
accesses a function that isn't there yet. The advantage is that it shows 
clearly which of the functions are just stubs if you actually implement 
a few of them differently.



If the list functions is really that huge but you have a class (the real 
driver class) that defines this API, then you could extract this list 
programmatically. That way, you could also ensure that your mock API 
doesn't provide functions not supported by the original.



Good luck!

Uli

--
http://mail.python.org/mailman/listinfo/python-list

Mocked object returning another Mocked object

[Jean-Michel Pichavant]

Greetings Fellow Group,


I'd need your help to fix some issues I have with my unitary tests.
I'm trying to bring them to the next level by Mocking equipments 
controlled by my application.


Despite me reading http://www.voidspace.org.uk/python/mock/index.html I 
cannot figure out how to do the following :


I have an App object with the 'target' attribute. This target is 
controlling a piece of hardware. The hardware itself holds a software, 
hence the target object having an 'api' attribute. I hope I make sense.


So basically I'd like

self.target.api.()

to return a Mocked object (Mocking an Api object response).



** Question **

I do I make *all* method calls return a specifik Mock?

target = Mock()

result = target.api.start() # I'd like result to be a Mock I defined 
with the 'returnCode' attribute


print result.returnCode
1

But I would like to do it for any method of api (the list is huge, 
setting each of them is not an option I think) so that in the end,


result = target.api.start()
result = target.api.stop()
result = target.api.reset()
result = target.api.loadSw()

return all the same Mock object (with 'returnCode')


Any idea ?

Cheers,

JM


--
http://mail.python.org/mailman/listinfo/python-list

New Python build regr test___all__ fails at 'ctypes.macholib.dyld'

[John Pote]

Hi,

I have built Python 2.7.3 from source and although the interpreter 
starts up and runs scripts (so far OK) the 'test___all__' regression 
test fails.


The machine I'm building on is a virtual server running a version of Red 
Hat Linux


Build commands:
./configure --prefix /usr
make

At the end of the make the following note is shown

	Python build finished, but the necessary bits to build these 	modules 
were not found:

_bsddb _sqlite3   bsddb185
sunaudiodev



Test command
./python Lib/test/regrtest.py -v test___all__
produces the following:-

== CPython 2.7.3 (default, Jul 13 2012, 10:26:48) [GCC 3.2.2 20030217 
(Red Hat Linux 8.0 3.2.2-2)]

==   Linux-2.4.29-hs-17-i686-with-glibc2.2 little-endian
== 
/usr/local/home/aquamaster3/Python-2.7.3/Python-2.7.3/build/test_python_4517
Testing with flags: sys.flags(debug=0, py3k_warning=0, 
division_warning=0, division_new=0, inspect=0, interactive=0, 
optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, 
ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0, 
hash_randomization=0)

test___all__
test_all (test.test___all__.AllTest) ... BaseHTTPServer
Bastion
CGIHTTPServer
ConfigParser
Cookie
DocXMLRPCServer
HTMLParser
MimeWriter
Queue
SimpleHTTPServer
SimpleXMLRPCServer
SocketServer
StringIO
UserDict
UserList
UserString
_LWPCookieJar
_MozillaCookieJar
__phello__.foo
_abcoll
_pyio
_strptime
_threading_local
_weakrefset
abc
aifc
antigravity
anydbm
argparse
ast
asynchat
asyncore
atexit
audiodev
base64
bdb
binhex
bisect
bsddb
bsddb.db
bsddb.dbobj
bsddb.dbrecio
bsddb.dbshelve
bsddb.dbtables
bsddb.dbutils
bsddb.test
bsddb.test.test_all
bsddb.test.test_associate
bsddb.test.test_basics
bsddb.test.test_compare
bsddb.test.test_compat
bsddb.test.test_cursor_pget_bug
bsddb.test.test_db
bsddb.test.test_dbenv
bsddb.test.test_dbobj
bsddb.test.test_dbshelve
bsddb.test.test_dbtables
bsddb.test.test_distributed_transactions
bsddb.test.test_early_close
bsddb.test.test_fileid
bsddb.test.test_get_none
bsddb.test.test_join
bsddb.test.test_lock
bsddb.test.test_misc
bsddb.test.test_pickle
bsddb.test.test_queue
bsddb.test.test_recno
bsddb.test.test_replication
bsddb.test.test_sequence
bsddb.test.test_thread
cProfile
calendar
cgi
cgitb
chunk
cmd
code
codecs
codeop
collections
colorsys
commands
compileall
compiler
compiler.ast
compiler.consts
compiler.future
compiler.misc
compiler.pyassem
compiler.pycodegen
compiler.symbols
compiler.syntax
compiler.transformer
compiler.visitor
contextlib
cookielib
copy
copy_reg
csv
ctypes
ctypes._endian
ctypes.macholib
ctypes.macholib.dyld
Aborted


Any help in resolving this problem would be appreciated

Regards,

John

--- Posted via news://freenews.netfront.net/ - Complaints to n...@netfront.net 
---
--
http://mail.python.org/mailman/listinfo/python-list

Re: Writing a wrapper - any tips?

[Stefan Behnel]

Martin P. Hellwig, 13.07.2012 09:39:
> On Friday, 13 July 2012 05:03:23 UTC+1, Temia Eszteri  wrote:
>> I'm going to be looking into writing a wrapper for the Allegro 5 game
>> development libraries, either with ctypes or Cython. They technically
>> have a basic 1:1 ctypes wrapper currently, but I wanted to make
>> something more pythonic, because it'd be next to impossible to deal
>> with the memory management cleanly in the script itself.
>>
>> Anything I should keep in mind? Any tips to pass on to a first-time
>> module writer, pitfalls to watch out for, etc.?
> 
> I would split the wrapping in layers, the lowest layer is a one on one
> exposure of the library with your wrapper, I would rather avoid ctypes
> for performance reasons, however if performance is not a concern ctypes
> is excellent and broadly available.
> 
> The next layer is purely there to make the lower layer pythonic, i.e.
> apply namespaces, automatic handling of memory, PEP8 naming convetions,
> etc. etc. just what you would expect from a modern pure python module
> 
> The next layer, if you want to, contains tools that are often used in
> that concept, think in the line of design patterns.

And the good thing about Cython in this context is that even if performance
*is* a concern, you can move code around between all three layers freely in
order to adjust it to your performance requirements and even drop the
lowest layer entirely. In fact, my advice is to really skip that lowest
layer in a Cython wrapper, because if it's really just a 1:1 mapping, it's
going to end up with a lot of boring and redundant code and you won't gain
anything from it.

Stefan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: adding a simulation mode

[andrea crotti]

2012/7/13 Steven D'Aprano :

> Well of course it does. If copytree fails, the try block ends and
> execution skips straight to the except block, which runs, and then the
> program halts because there's nothing else to be done.
>
> That at least is my guess, based on the described symptoms.
>

Well I think that's what I was stupidly missing, I always had only one
possibly failing thing in a try/except block,
and I always gave for granted that it doesn't jump to the except block
on first error, but of course it makes
more sense if it does...

Thanks a lot
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing a wrapper - any tips?

[Martin P. Hellwig]

On Friday, 13 July 2012 05:03:23 UTC+1, Temia Eszteri  wrote:
> I'm going to be looking into writing a wrapper for the Allegro 5 game
> development libraries, either with ctypes or Cython. They technically
> have a basic 1:1 ctypes wrapper currently, but I wanted to make
> something more pythonic, because it'd be next to impossible to deal
> with the memory management cleanly in the script itself.
> 
> Anything I should keep in mind? Any tips to pass on to a first-time
> module writer, pitfalls to watch out for, etc.?

I would split the wrapping in layers, the lowest layer is a one on one exposure 
of the library with your wrapper, I would rather avoid ctypes for performance 
reasons, however if performance is not a concern ctypes is excellent and 
broadly available.

The next layer is purely there to make the lower layer pythonic, i.e. apply 
namespaces, automatic handling of memory, PEP8 naming convetions, etc. etc.
just what you would expect from a modern pure python module

The next layer, if you want to, contains tools that are often used in that 
concept, think in the line of design patterns.

hth
-- 
mph
-- 
http://mail.python.org/mailman/listinfo/python-list

help needed with subprocess, pipes and parameters

[nuffi]

If I copy and paste the following command into a command window,   it does what 
I need.  

c:\Programs\bob\bob.exe -x -y "C:\text\path\to some\file.txt" | 
c:\Programs\kate\kate.exe -A 2 --dc "Print Media Is Dead" --da "Author" --dt 
"Title" --hf "Times" --bb "14" --aa "" --font "Ariel" - "C:\rtf\path\to 
some\file.rtf"

My mission is to recreate this command within a python script,  so that I can 
pass a bunch of different parameters into it,  and use it as a batch over a 
bunch of different papers.

http://docs.python.org/library/subprocess.html seems to be the thing to use in 
python 2.7.3.  I also checked out 
http://www.doughellmann.com/PyMOTW/subprocess/.

My attempts run fine,  create destination folders ok and prints done but don't 
actually seem to process the file.  Is there some way to get subprocess to 
output the command it's generating so I can see what I'm doing wrong,  rather 
than just the output of the command?

How can I chekc that kate's opening the pipe left from bob?Bob may take 
some time to execute,  will that matter?


The code I came up with looks like this:

import os, glob, subprocess

sourceDir = "c:\\text\\"
destDir = "c:\\rtf\\"
bobPath = "C:\\Programs\\bob\\bob.exe"
katePath = "C:\\Programs\\kate\\kate.exe"

def get_new_path(doc):
blah = doc.replace(sourceDir, destDir)
if not os.path.isdir(blah):
os.makedirs(blah)
rtf = blah.replace('.txt', '.rtf')
pathString = '- "' + (os.path.join(rtf)) + '"'
return(pathString)


def convert_doc(doc):
dc = '--dc "Print Media Is Dead"'
da = '--da "Author"'
dt = '--dt "Title"'
hf = '--hf "Times"'
fn = '--font "Ariel"'
bb = '--bb "14"'
docpath = '"' + (os.path.join(doc)) + '"'
path = get_new_path(doc)
A = '-A 2'
bob = subprocess.Popen([bobPath, '-x', '-y', docpath], stdout = 
subprocess.PIPE,)
kate = subprocess.Popen([katePath, A, dc, da, dt, hf, fn, bb, path], stdin 
= bob.stdout, stdout = subprocess.PIPE,)
end_of_pipe = kate.stdout
#print kate
#subprocess.call(['echo', "Test peice of text", '>', 'D:\\output.txt'])
for line in end_of_pipe:
print '\t', blah.strip()
#raw_input()
return('done')


test = convert_doc("c:\\text\\path to\\some\\specific text file.txt")
print(blah)


==

Thanks for looking  :-)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter.event.widget: handler gets name instead of widget.

[Peter Otten]

Frederic Rentsch wrote:

> I'm sorry I can't post an intelligible piece that does NOT work. I
> obviously can't post the whole thing. 

How about a pastebin then? Or even bitbucket/github as you need to track 
changes anyway?

> It is way too convoluted.

"Convoluted" code is much easier to debug than no code ;)

Another random idea: run your code on a more recent python/tcl installation. 
If you are lucky you get a different error.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to safely maintain a status file

[Steven D'Aprano]

On Thu, 12 Jul 2012 21:26:20 -0700, rantingrickjohnson wrote:

> On Thursday, July 12, 2012 10:13:47 PM UTC-5, Steven D'Aprano wrote:
>> Rick has obviously never tried to open a file for reading when somebody
>> else has it opened, also for reading, and discovered that despite
>> Windows being allegedly a multi-user operating system, you can't
>> actually have multiple users read the same files at the same time.
> 
> You misread my response. My comment was direct result of Christian
> stating:
> 
> (paraphrase) "On some systems you are not permitted to delete a file
> whilst the file is open "
> 
> ...which seems to be consistent to me. Why would *anybody* want to
> delete a file whilst the file is open? 

Because it is useful and a sensible thing to do.

Why should one misbehaved application, keeping a file open, be allowed to 
hold every other application, and the file system, hostage?

This is one of the many poor decisions which makes Windows so vulnerable 
to viruses and malware. If malware can arrange to keep itself open, you 
can't delete it. Thanks guys!


> Bringing back the car analogy
> again: Would you consider jumping from a moving vehicle a consistent
> interaction with the interface of a vehicle? Of course not. The
> interface for a vehicle is simple and consistent:
> 
>  1. You enter the vehicle at location A 
>  2. The vehicle transports you to location B 
>  3. You exit the vehicle

Amusingly, you neglected to specify "the vehicle stops" -- and rightly 
so, because of course having to stop the vehicle is not a *necessary* 
condition for exiting it, as tens of thousands of stunt men and women can 
attest.

Not to mention people parachuting out of an airplane, pirates or 
commandos boarding a moving ship, pedestrians transferring from a slow 
moving walkway to a faster moving walkway, farmers jumping off a trailer 
while it is still being towed behind a tractor (and jumping back on 
again), and Bruce Willis in "Red" in very possibly the best slow-motion 
action sequence in the history of Hollywood.

http://www.youtube.com/watch?v=xonMpj2YyDU


> At no time during the trip would anyone expect you to leap from the
> vehicle. 

Expected or not, you can do so.


> But when you delete open files, you are essentially leaping
> from the moving vehicle! This behavior goes against all expectations of
> consistency in an API -- and against all sanity when riding in a
> vehicle!

Fortunately, files on a file system are not cars, and deleting open files 
is a perfectly reasonable thing to do, no more frightening than in Python 
deleting a reference to an object using the del statement. Imagine how 
stupid it would be if this happened:


py> x = 42
py> y = x
py> del y
Traceback (most recent call last):
  File "", line 1, in 
DeleteError: cannot delete reference to object '42' until no other 
references to it exist


Fortunately, Python doesn't do that -- it tracks when the object is no 
longer being accessed, and only then physically reclaims the memory used. 
And so it is on POSIX file systems: the file system keeps track of when 
the file on disk is no longer being accessed, and only then physically 
reclaims the blocks being used. Until then, deleting the file merely 
unlinks the file name from the blocks on disk, in the same way that 
"del y" merely unlinks the name y from the object 42.


>> Opening files for exclusive read *by default* is a pointless and silly
>> limitation. It's also unsafe: if a process opens a file for
>> exclusive read, and then dies, *no other process* can close that file.
> 
> Oh come on. Are you actually going to use "errors" or "unintended
> consequences", or even "Acts of God" to defend your argument? 

Features have to be judged by their actual consequences, not some 
unrealistic sense of theoretical purity. The actual consequences of 
mandatory exclusive file locking is, *it sucks*.

Windows users are used to having to reboot their server every few days 
because something is broken, so they might not mind rebooting it because 
some file is locked in a mandatory open state and not even the operating 
system can unlock it. But for those with proper operating systems who 
expect months of uninterrupted service, mandatory locking is a problem to 
be avoided, not a feature.


> Okay.
> Okay. I suppose "IF" the car spontaneously combusted "THEN" the
> passengers would be wise to jump out, leaving the vehicle to the whims
> of inertia.

In this analogy, is the car the file name, the inode, or the directory? 
Are the passengers the file name(s), or the file contents, or the inode? 
Is the driver meant to be the file system? If I have a hard link to the 
file, does that mean the passengers are in two cars at once, or two lots 
of passengers in the same car?


>> One neat trick is to open a file, then delete it from disk while it is
>> still open. So long as your process is still running, you can write to
>> this ghost file, as normal, but no other process can (easily) see it.
>>