Re: value of pi and 22/7

[Tim Harig]

On 2016-06-18, Steven D'Aprano  wrote:
> On Sat, 18 Jun 2016 09:49 am, Ian Kelly wrote:
>
>> If I tell you that the speed of light is 300,000,000 m/s, do you think
>> that measurement has 9 significant digits? If you do, then you would be
>> wrong.
> What if the figure to nine significant digits *actually is* three followed
> by eight zeroes?

The you can either write it as 3. (notice the trailing decimal
indicating that all of the zeros are indeed significant) or write it it
scientific notation.

> For all that it is in widespread use, I think the concept of "significant
> figures" is inherently ambiguous.

Only for those who do not understand it.

The main problem I have with significant figures is that measurement
accuracy is often not constrained to a decimal system.  A scale that can
measure in 1/5 units is more accurate than a scale that can measure only
in whole units but it is not as accurate as a scale that can measure
all 1/10 units.  Therefore it effectively has a fractional number of
significant figures.

I could just cut my loses and express the lower number of significant
figures but, I usually express the error explicitly instead:

 +- 0.2 units

where +- looks like the  html entity.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Message passing syntax for objects

[Tim Harig]

On 2013-03-18, Mark Janssen dreamingforw...@gmail.com wrote:
 Alan Kay's idea of message-passing in Smalltalk are interesting, and
 like the questioner says, never took off.  My answer was that Alan
 Kay's abstraction of Everything is an object fails because you can't
 have message-passing, an I/O task, working in the same space as your
 objects -- they are two very different functionalities and they have
 to be preserved **for the programmer**.

Without concurrency, message passing and interacting through functions
are semantically similar.  You have operating for sending and receiving
messages.  Python does this through methods which potentially have inputs
and outputs.  Function syntax does have two slight advantages over pure
message passing.

1. A command is sent with the message as opposed to requiring requests to
be parsed from the message separately from the data.

2. Functions constrain what actions are available and what inputs they can
legal accept.  This is doubly true

With concurrency, CSP has shown us that message passing is valuable for
syncronization.  Most CSP languages retain the idea of passing messages.
This need not be the case.  The same semantics could be acheived through
function call syntax, although it would likely be less intuitive.

If you want to pursue your ideas further, and you have not already,
I suggest that you investigate Erlang and its CSP based concurrent
oriented programming.  Alan Kay, himself, commented that he felt it
matched his ideas of message based OOP.  When you return, there is a
Candygram package providing similar constructs (last I knew, it did not
yet impliment light weight processes) for Python.

You might also look at using Go which has CSP based channel and coroutine
semantics.  It has many things in common with Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which is the best book to learn python

[Tim Harig]

On 2011-01-25, Mark Summerfield l...@qtrac.plus.com wrote:
 On Jan 24, 5:09 pm, santosh hs santosh.tron...@gmail.com wrote:
 Hi All,
 i am beginner to python please tell me which is the best available
 reference for beginner to start from novice

 If you want to learn Python 3 and have some prior programming
 experience (in any modern procedural or object oriented language), you
 might find
 Programming in Python 3 (http://www.qtrac.eu/py3book.html) to be a
 good choice. (I'm biased though since I wrote it;-)

I am usually a big fan for O'reilly books and I started learning Python
from the first edition of _Learning Python_.  It's not a bad book and it
will get you started.  I cannot speak for the latest edition which seems to
contain quite a bit more then the version I read.

When Python 3 was released, I decided to try relearn Python 3 from scratch
rather then trying to simply figure out the differences between versions.  I
picked up Mr. Summerfield's book because it seemed to be the first book to
cover Python 3 excusively and I was rather impressed.  I would definitely
recommend it to others.

[OT] P.S. to Mark Summerfield.  You have been hanging around in the Go Nuts
mailing list.  Is that any indication that you might be considering writing
a book on Go?  If you do, you will have at least one customer.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-19, Tim Roberts t...@probo.com wrote:
 Tim Harig user...@ilthio.net wrote:
On 2011-01-17, carlo syseng...@gmail.com wrote:

 2- If that were true, can you point me to some documentation about the
 math that, as Mark says, demonstrates this?

It is true because UTF-8 is essentially an 8 bit encoding that resorts
to the next bit once it exhausts the addressible space of the current
byte it moves to the next one.  Since the bytes are accessed and assessed
sequentially, they must be in big-endian order.

 You were doing excellently up to that last phrase.  Endianness only applies
 when you treat a series of bytes as a larger entity.  That doesn't apply to
 UTF-8.  None of the bytes is more significant than any other, so by
 definition it is neither big-endian or little-endian.

It depends how you process it and it doesn't generally make much
difference in Python.  Accessing UTF-8 data from C can be much trickier
if you use a multibyte type to store the data.  In that case, if happen
to be on a little-endian architecture, it may be necessary to remember
that the data is not in the order that your processor expects it to be
for numeric operations and comparisons.  That is why the FAQ I linked to
says yes to the fact that you can consider UTF-8 to always be in big-endian
order.  Essentially all byte based data is big-endian.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

Considering you post contained no information or evidence for your
negations, I shouldn't even bother responding.  I will bite once.
Hopefully next time your arguments will contain some pith.

On 2011-01-19, Antoine Pitrou solip...@pitrou.net wrote:
 On Wed, 19 Jan 2011 11:34:53 + (UTC)
 Tim Harig user...@ilthio.net wrote:
 That is why the FAQ I linked to
 says yes to the fact that you can consider UTF-8 to always be in big-endian
 order.

 It certainly doesn't. Read better.

- Q: Can a UTF-8 data stream contain the BOM character (in UTF-8 form)? If
- yes, then can I still assume the remaining UTF-8 bytes are in big-endian
^^
- order?
  ^^
- 
- A: Yes, UTF-8 can contain a BOM. However, it makes no difference as
 ^^^ 
- to the endianness of the byte stream. UTF-8 always has the same byte
    ^^
- order. An initial BOM is only used as a signature -- an indication that
  ^^
- an otherwise unmarked text file is in UTF-8. Note that some recipients of
- UTF-8 encoded data do not expect a BOM. Where UTF-8 is used transparently
- in 8-bit environments, the use of a BOM will interfere with any protocol
- or file format that expects specific ASCII characters at the beginning,
- such as the use of #! of at the beginning of Unix shell scripts.

The question that was not addressed was whether you can consider UTF-8
to be little endian.  I pointed out why you cannot always make that
assumption in my previous post.

UTF-8 has no apparent endianess if you only store it as a byte stream.
It does however have a byte order.  If you store it using multibytes
(six bytes for all UTF-8 possibilites) , which is useful if you want
to have one storage container for each letter as opposed to one for
each byte(1), the bytes will still have the same order but you have
interrupted its sole existance as a byte stream and have returned it
to the underlying multibyte oriented representation.  If you attempt
any numeric or binary operations on what is now a multibyte sequence,
the processor will interpret the data using its own endian rules.

If your processor is big-endian, then you don't have any problems.
The processor will interpret the data in the order that it is stored.
If your processor is little endian, then it will effectively change the
order of the bytes for its own evaluation.

So, you can always assume a big-endian and things will work out correctly
while you cannot always make the same assumption as little endian
without potential issues.  The same holds true for any byte stream data.
That is why I say that byte streams are essentially big endian.  It is
all a matter of how you look at it.

I prefer to look at all data as endian even if it doesn't create
endian issues because it forces me to consider any endian issues that
might arise.  If none do, I haven't really lost anything.  If you simply
assume that any byte sequence cannot have endian issues you ignore the
possibility that such issues might not arise.  When an issue like the
above does, you end up with a potential bug.

(1) For unicode it is probably better to convert to characters to
UTF-32/UCS-4 for internal processing; but, creating a container large
enough to hold any length of UTF-8 character will work.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-19, Adam Skutt ask...@gmail.com wrote:
 On Jan 19, 9:00 am, Tim Harig user...@ilthio.net wrote:
 That is why I say that byte streams are essentially big endian.  It is
 all a matter of how you look at it.

 It is nothing of the sort.  Some byte streams are in fact, little
 endian: when the bytes are combined into larger objects, the least-
 significant byte in the object comes first.  A lot of industrial/
 embedded stuff has byte streams with LSB leading in the sequence, CAN
 comes to mind as an example.

You are correct.  Point well made.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-19, Antoine Pitrou solip...@pitrou.net wrote:
 On Wed, 19 Jan 2011 14:00:13 + (UTC)
 Tim Harig user...@ilthio.net wrote:
 UTF-8 has no apparent endianess if you only store it as a byte stream.
 It does however have a byte order.  If you store it using multibytes
 (six bytes for all UTF-8 possibilites) , which is useful if you want
 to have one storage container for each letter as opposed to one for
 each byte(1)

 That's a ridiculous proposition. Why would you waste so much space?

Space is only one tradeoff.  There are many others to consider.  I have
created data structures with much higher overhead than that because
they happen to make the problem easier and significantly faster for the
operations that I am performing on the data.

For many operations, it is just much faster and simpler to use a single
character based container opposed to having to process an entire byte
stream to determine individual letters from the bytes or to having
adaptive size containers to store the data.

 UTF-8 exists *precisely* so that you can save space with most scripts.

UTF-8 has many reasons for existing.  One of the biggest is that it
is compatible for tools that were designed to process ASCII and other
8bit encodings.

 If you are ready to use 4+ bytes per character, just use UTF-32 which
 has much nicer properties.

I already mentioned UTF-32/UCS-4 as a probable alternative; but, I might
not want to have to worry about converting the encodings back and forth
before and after processing them.  That said, and more importantly, many
variable length byte streams may not have alternate representations as
unicode does.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-19, Antoine Pitrou solip...@pitrou.net wrote:
 On Wed, 19 Jan 2011 16:03:11 + (UTC)
 Tim Harig user...@ilthio.net wrote:
 
 For many operations, it is just much faster and simpler to use a single
 character based container opposed to having to process an entire byte
 stream to determine individual letters from the bytes or to having
 adaptive size containers to store the data.

 You *have* to process the entire byte stream in order to determine
 boundaries of individual letters from the bytes if you want to use a
 character based container, regardless of the exact representation.

Right, but I only have to do that once.  After that, I can directly address
any piece of the stream that I choose.  If I leave the information as a
simple UTF-8 stream, I would have to walk the stream again, I would have to
walk through the the first byte of all the characters from the beginning to
make sure that I was only counting multibyte characters once until I found
the character that I actually wanted.  Converting to a fixed byte
representation (UTF-32/UCS-4) or separating all of the bytes for each
UTF-8 into 6 byte containers both make it possible to simply index the
letters by a constant size.  You will note that Python does the former.

UTF-32/UCS-4 conversion is definitly supperior if you are actually
doing any major but it adds the complexity and overhead of requiring
the bit twiddling to make the conversions (once in, once again out).
Some programs don't really care enough about what the data actually
contains to make it worth while.  They just want to be able to use the
characters as black boxes.

 Once you do that it shouldn't be very costly to compute the actual code
 points. So, much faster sounds a bit dubious to me; especially if you

You could I suppose keep a separate list of pointers to each letter so that
you could use the pointer list for indexing or keep a list of the
character sizes so that you can add them and calculate the variable width
index; but, that adds overhead as well.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-19, Antoine Pitrou solip...@pitrou.net wrote:
 On Wed, 19 Jan 2011 18:02:22 + (UTC)
 Tim Harig user...@ilthio.net wrote:
 Converting to a fixed byte
 representation (UTF-32/UCS-4) or separating all of the bytes for each
 UTF-8 into 6 byte containers both make it possible to simply index the
 letters by a constant size.  You will note that Python does the
 former.

 Indeed, Python chose the wise option. Actually, I'd be curious of any
 real-world software which successfully chose your proposed approach.

The point is basically the same.  I created an example because it
was simpler to follow for demonstration purposes then an actual UTF-8
conversion to any official multibyte format.  You obviously have no
other purpose then to be contrary, so we ended up following tangents.

As soon as you start to convert to a multibyte format the endian issues
occur.  For UTF-8 on big endian hardware, this is anti-climactic because
all of the bits are already stored in proper order.  Little endian systems
will probably convert to a native native endian format.  If you choose
to ignore that, that is your perogative.  Have a nice day.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 On Mon, 17 Jan 2011 19:41:54 +, Tim Harig wrote:

 One of the arguments for Python has always made is that you can optimize
 it by writing the most important parts in C.  Perhaps that is a crutch
 that has held the communty back from seeking higher performance
 solutions in the language itself.

 Are you aware of PyPy?

Yes I have heard of PyPy, RPython, and Cython.  If they were sufficient I
probably wouldn't have to look around.  These alternate Python projects
have made some small gains; but, they still have a long way to go.

 PyPy is now about double the speed of CPython for most things, and they 
 have set themselves the ambitious target of being faster than C.

Let me know when they reach that target.  When Java started there was hype
that the JIT would be able to run faster then native C because the JIT
could optimize at runtime based on the actual data that it encountered.
That never really panned out either.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, Stefan Behnel stefan...@behnel.de wrote:
 Tim Harig, 17.01.2011 20:41:
 One of the arguments for Python has always made is that you can optimize
 it by writing the most important parts in C.  Perhaps that is a crutch
 that has held the communty back from seeking higher performance solutions
 in the language itself.

 The good news with Cython is that you no longer need to write the most 
 important parts in C. Instead, you type them statically and compile them. 
 You don't even need to sacrifice Python source compatibility for that. So 
 you get the best of Python at the speed of C (or 'c', as some would say ;).

Well not quite.  The fact that you still need a Python interpeter should
tell you that the end result is not really just C.  Effectively, you get
the same thing that you might get from writing C modules for your
performance sensitive loops.  Cython just makes it a little easier to
generate the C modules; but, there is still a lot of Python operating under
the hood.

 I prefer a single language as opposed to a creolization of two.

 With the possible exception of Lisp, I find it hard to think of a language 
 that's still alive and not the creolisation of (at least) two other 
 languages. They all inherited from each other, sometimes right from the 
 start (lessons learned) and always during their subsequent life time.

I am not talking about language influences.  Cython effectively requires
two separate languages that interoperate.  The end result is a mix
of two code written in two separate langauges.  That is not a single
language solution.

 I certainly don't need to require anybody who
 wants to use a program I have compiled to install an interpreter.

 You will be happy to hear that you can link Cython programs against 
 libpython (even statically) to build an executable that embeds the 
 interpreter for you. Besides, on most operating systems, installing a 

Which means that you effectively end up with a py2exe style frozen binary.
While that is a nice hack for solving some problems, it is hardly an
elegant solution.  It still effectively means that I have to package a
redundant Python installation with every binary that I distribute.  I 
just don't have to write installation routines that check for, and
potentially install, a compatable version of Python.

 interpreter for you. Besides, on most operating systems, installing a 
 dependency is done automatically, and Go code doesn't run natively on 
 Windows either, without first installing and running the compiler over it.

Go works like C.  If I compile my code on Windows, then I can give the
binary to another Windows user and they can execute it without having to
install any additional software (gcgo currently compiles everything using
static binaries.  Since the linker only includes the fuctions you actually
use, you don't end up with the bloat of embedding entire libraries.
I am not sure about gccgo.).  The same holds true for Linux, BSD, OS X,
and any other supported OS.  Every go program includes a runtime componet
that provides the garbage collection; but, all of that is included as 100%
processor native code.  That is quite a bit different from having to
include an interpreter to run non-native code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, Rui Maciel rui.mac...@gmail.com wrote:
 Tim Harig wrote:

 You still don't see many
 companies doing large scale internal development using Python and you
 definately don't see any doing external developement using a language
 that gives the customers full access to the source code.

 What you refered as full access to the source code only goes as far as 
 the license which was imposed by the copyright holders lets it to go.  If 
 you distribute the source code along with the binaries but you only 
 license your code if the licencees accept that they may look at the source 
 code but they can't touch it then distributing the source code is 
 essentially meaningless.  There is a good reason why open source 
 software is not the same thing as free software.

That argument is not going to fly with where commericial interests are
concerned. The simple fact is that there is legality and then there is
reality.

People follow licenses like they follow
the speed limit: only when they think they might get caught and punished.
When people break the licenses, the only recourse is litigation; which
is expensive.  Even finding and proving license violations is difficult
where source is availible.  It is therefore in the corporate interest
to make breaking licenses as difficult, uneconomical, and tracible as
possible; and that is exactly what companies do.  Even companies that
don't really have any trade secrets to protect are protective about
their source keeping it locked out of public view.  Access to the source
generally means signing an NDA so that if the source is somehow leaked,
they know the most likely candidates for the origin [so as not to pun with
source] of the leak.

Whether or not you actually agree with that economic reality is
irrelevant.  Those who fund commerical projects do; and, any developement
tool which violates the security of the source is going to find itself
climbing an uphill battle in trying to gain market penetration with
commericial software producers.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, geremy condra debat...@gmail.com wrote:
 On Mon, Jan 17, 2011 at 4:02 PM, Tim Harig user...@ilthio.net wrote:
 I really question that you get Java anywhere even close to C performance.
 Google reports they get within the same order of magnitude as C for
 their long-lived server processes where the JIT has had time to optimize
 its results.  For shorter term processes such as desktop applications,
 Java performance stinks -- even after you discount the JVM starup time.

 I'm sorry, but you're wrong on this. Java's performance can be
 excellent, particularly when it comes to mathematical functions. I
 recall reading a pretty smug paper a few years ago describing how they
 managed to beat C on a number of numerical benchmarks.

I have no doubt that Java *can* occasionally beat *some* C for *some*
benchmarks; but, overall, Java has a terrible reputation for performance.

I worked with a company a few years ago that tried replacing a C VNC
client with a Java one so that its technical support contractor's wouldn't
need to have the VNC client installed on the agent's workstations.
Several of the contracters had to upgrade their systems in order to
use the Java version because it slowed down the machines so much that
the agents could not perform their jobs effectively; and that is pretty
typical from what I have seen with Java overall.

Java performs very well with some select tasks.  For others, it does
exceedingly poor.  That kind of hit or miss is pretty typical for JIT
compilers in general.  That isn't usually the case for fully compiled
langauges where you are pretty much guaranteed to get decent, if not
always the absolute top, performance.

 Ada is capable of C++ like performance *if* you compile it to remove *all*
 of runtime checking.  Depending what checks you enable, it can run much
 slower.

 No idea, never used it.

 in line with the work of others. You can argue that it's good enough-
 it is, for most cases- but taking a 20% performance hit rules it out
 of a lot of systems work, and the C-Go gap in many cases is much
 larger than that.

 I don't work anything that involves and absolute need for performance.

 Then don't argue about performance, it makes you look like a hack just
 eager to shill for your language.

What you don't seem to realize is there is often a performance level that
is good enough.  For many things, Python is good enough.  Many others,
where Python is insufficient, may still be acceptable to use Java.
Some things require the absolute best performance and will probably
always need C/C++ or equivilantly low level language.

 I could probably accept penalty several times that of C for higher
 level functionality; but, sometimes the penalty for Python is just
 too much.  Many of my programs are very quick lived such that even
 loading an interpeter or VM can eclipse the actual runtime.  Given less
 developmental effort required, I also find that I have more time to look
 for ways to optimize Go.  There are many things (such as using alternate
 data structures rather then the build in slices and immutable strings)
 that can be used to accelerate Go when time permits and I suspect many
 more will be found as the language matures.

 This is inconsistent with your argument about PyPy. See my earlier comment.

I can accept 2 to 3 times the overall performance of C for almost all of
the problems that I deal with.  When that multiple gets into the double
digits, it can start to cause some real headaches for some problems.  When
that number reaches the upper double digits, it is acceptable for even
fewer problems.

Python is great for those problems where performance isn't critical and I
make extensive use of it.  Different methods of mixing Python and C
(including manual, PyPy, SWIG, boost, etc) can extend Python's useful
range; but, I have not seen the kind of speed improvements that bring it to
less then an order of magnitude of C's speed overall.

Even assuming that PyPy does actually manage to reach within a magnitude
of C with the extra effort required to leverage two languages, why
would I bother when I can do it with one?  PyPy and similar methods
where great when there was no other mid level alternative that supported
Python like features.  Now it just seems like using Python as a hammer
for every problem whether or not it is the right tool for the job.

 Go is also not an ideal language for enterprise development. It
 provides no compatibility or support guarantees; in fact, the website
 describes it as 'an experiment' and recommends it for 'adventurous

 There is no doubt that it is a young project and there are a number of
 things that will need to be improved to be effective for some branches
 of programming; but, that isn't a language restriction.

 It will nevertheless preclude its use in most enterprise software
 development. That's most systems software.

So you conclude that because it is not quite ready for prime time yet that it
never will be?  I can

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, Terry Reedy tjre...@udel.edu wrote:
 On 1/18/2011 10:30 AM, Tim Harig wrote:

 Whether or not you actually agree with that economic reality is
 irrelevant.  Those who fund commerical projects do; and, any developement
 tool which violates the security of the source is going to find itself
 climbing an uphill battle in trying to gain market penetration with
 commericial software producers.

 Of course. When I submit or commit patches, I am doing it mostly for 
 hobby, educational, and scientific users, and maybe website makers (who 
 make site I can visit). If commercial users piggyback on top, ok. I do 
 not know how many developers, if any, are after such market penetration.

You kind of over-extended the intentions of my comment.  It does not apply
to open source software in general.  I agree that open source authors are
not interested in the quantitative value of market penetration.  However, I
am betting that most authors of developement tools would like to be able to
use their tools on the job.  

I am sure that more software developers would love to develop using
Python as part of their job.  For some this is a reality; but, many more
are stuck using their employer's choice of language.  One of the factors
that employers consider, when they choose a language, if they produce
retail software is that the process of compiling will sufficiently
obfiscate their code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-18, geremy condra debat...@gmail.com wrote:
 On Tue, Jan 18, 2011 at 11:05 AM, Tim Harig user...@ilthio.net wrote:
 Even assuming that PyPy does actually manage to reach within a magnitude
 of C with the extra effort required to leverage two languages, why
 would I bother when I can do it with one?  PyPy and similar methods
 where great when there was no other mid level alternative that supported
 Python like features.  Now it just seems like using Python as a hammer
 for every problem whether or not it is the right tool for the job.

 You clearly have no idea what you're talking about regarding PyPy. You
 could at least have googled it before speaking about it.

No, I have watched several such projects over the years.  Pysco, Unladen
Swallow, Cython, PyPy, Shedskin, etc.  Source to source translators, JITs,
and C language integration all just add to complexity.  You can't do this,
you can't do that, you have to learn a new way of doing something else,
ad nauseum.

So when something new that provided Python like capabilities without many
of Python's drawbacks came along, I jumped on it.  It provides a much
cleaner solution to the problem without kludges.  I will use Python for
what it does well and cleanly.  For the rest, there are now better tools.
Once again, its about the right tool for the right job.

 Again, you don't know what you're talking about WRT PyPy.

Nor do I really want to.  I have found a much simpler solution
to the problem.  I would recommend it to many others that like the
Python language but who occassionaly struggle with its implementation
constraints.

I would say that I am sorry that it doesn't work for you; but, you seem
to prefer Java and Pypy anyway so we are both happy.

 2. There is a difference in binding to a solution that is already written
        in another language so as to not reinvent a wheel and implementing
        a *new* library in another language to be used exclusively
        with Python.

 Even if that binding is done for performance reasons?

Yep, that is pretty much the summation of my dual language argument.
I don't expect a pure Python implementation of curses since there is a
perfectly good C library available to bind to.  Why reinvent the wheel.

Resorting to writing packages in another language or compiling Python
code into another source language is a kludge necessary because of
the performance characteristics of the language.  I suppose that is
an acceptable kludge when that is your only alternative.  For me it no
longer is.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-16, geremy condra debat...@gmail.com wrote:
 On Sun, Jan 16, 2011 at 3:03 AM, Tim Harig user...@ilthio.net wrote:
 On 2011-01-16, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 If the author thinks that Go is a tried and true (his words, not mine)
 language where programmers can go to look for work, I think he's
 fooling himself.

 No I wouldn't say that it has reached market penetration yet; but, it
 has more momentum then any language I am familiar with.  I wouldn't be
 at all surprised to see it becoming quite common in the next five years.

 I would be very surprised if this were the case. As you point out,
 languages typically have very long incubation times before they reach
 any kind of serious market penetration. This seems doubly true for a
 relatively narrowly targeted language that is in many ways on the
 wrong side of history.

I wouldn't say Go is narrowly targeted.  It's a systems language that can
compete in the same domain with scripting languages.  It is true that most
languages have long incubation periods; but, corporate support can change
that quite a bit.  C#, being backed by Microsoft, managed to go mainstream
pretty quickly.

 How long has it taken Python to reach its present level of market
 penetration?  And, I still don't see a huge amount of professional Python
 use outside of web developement.  Go has only been public for less then
 a year.

 Python's very widely used for scripting and related tasks, and has a
 pretty big user base in academia and the sciences.

Python has been widely used by people like us that happen to like the
language and found ways to use it in our workplaces; but, most of the
time it is an unofficial use that the company.  You still don't see many
companies doing large scale internal development using Python and you
definately don't see any doing external developement using a language
that gives the customers full access to the source code.

 Personally, I think the time is ripe for a language that bridges the
 gap between ease of use dynamic languages with the performance and
 distribution capabilities of a full systems level language.

 I agree. That does not make Go that language, and many of the choices
 made during Go's development indicate that they don't think it's that
 language either. I'm speaking specifically of its non-object model,
 lack of exceptions, etc.

1. Go has an object model.  What it lacks is an object hierarchy where all
object are decended from a single root object since it does
not support object inheritance as it is used in most languages.
In Go we simply adapt an object to meet the needs of the newer
object by adding whatever new functionality is needed.

2. Go has a similar mechanism to exceptions, defer/panic/recover.  It does
downplay

This is after all the promise the VM based languages made but never
 really fulfilled. It is also high time for a fully concurrent language fully
 capable of taking advantage of multicore processors without having to
 deal with the inherent dangers of threading.  There are several good
 choices available for both a even a few that fit both bills; but, few of
 them have the support of a company like Google that is capable of the
 push required to move the language into the mainstream.

 You might be right, but I doubt we'll know one way or the other in the
 next 5 years. Personally, I'm hoping that functional language use
 continues to grow.

I personally doubt that purely functional languages will ever make it
mainstream.  Functional programming has been around for a long time and
never really ever managed to break out of academic research.  The current
interest in functional programming stems merely because some announced
that it would be *the* way to utilize multicore computers.  Having looked
into the space somewhat, there is more hype then substantiation for
purely functional concepts.  What the hype did do was return attention
to SCP style concurrency using actors and MPI and I think that will be
the direction taken for concurrent programming into the future.

I believe functional programming will make an impact in the mainstream in
the form of functionally enabled multiparadigm but not purely functional
languages.  I think you will see code that uses more functional concepts
as guidelines to better code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages

[Tim Harig]

On 2011-01-17, Paul Rubin no.email@nospam.invalid wrote:
 geremy condra debat...@gmail.com writes:
 I agree. That does not make Go that language, and many of the choices
 made during Go's development indicate that they don't think it's that
 language either. I'm speaking specifically of its non-object model,
 lack of exceptions, etc 
 You might be right, but I doubt we'll know one way or the other in the
 next 5 years. Personally, I'm hoping that functional language use
 continues to grow.

 You know, the functional programming community seems to think of OOP as
 a 1990's thing that didn't work out.  Most things that can be done with
 OOP, can be done with higher-order functions and bounded polymorphism
 like in Haskell.

Which is rather interesting because the OOP community had traditionally
though of functional programming as a 1960's thing that didn't work out.
Functional programming has been around a long time; but, it only regained
conciousness outside of academia because of its hyped abilities to
make threading easier.  Unfortunately for functional programming,
much of that ability actually traces back to MPI and the actor model.
Using the actor model, it is possible to reap the benefits of easier
multiprocessing without using any functional programming concepts.

That isn't to say that the injection of functional programming into the
mainstream will not have some good affects overall.  I expect to see
functional programming concepts being incorporated into guidelines for
imperative language use.

 I'm not sure, but I don't think Erlang has exceptions in the sense we're
 used to.  Someone mentioned Erlang uses a VM, but I think there is a

Erlang has exceptions that look very much like Python exceptions.

 native compiler called HIPE.  Of course there is still a fairly
 substantial runtime system, but that's true of any language with a
 garbage collector and so forth.

Which is why I say Erlang doesn't compile to a native binary.  HIPE has
been available through the standard OTP for some time now.  HIPE generates
native machine OP codes where possible; but, the generated product must
still run on the BEAM runtime system.  The end result is much faster then
Python and often faster then Java with a JIT; but, still doesn't rival
C code.  It is possible to optimize things quite a bit by integrating with
C code; but, then you loose much of the safety provided by Erlang and BEAM.

This isn't such a tragedy Erlang as it is for other managed VMs because
Erlang/BEAM makes powerful usage of its VM for fault tolerance mechanisms.  I
don't know of any other VM that allows software upgrades on a running system.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-17, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 On Mon, 17 Jan 2011 09:12:04 +, Tim Harig wrote:

 Python has been widely used by people like us that happen to like the
 language and found ways to use it in our workplaces; but, most of the
 time it is an unofficial use that the company.  You still don't see many
 companies doing large scale internal development using Python and you
 definately don't see any doing external developement using a language
 that gives the customers full access to the source code.

 Careful with the FUD there *wink*

This isn't FUD, I am as enthusiastic about Python as anybody here; but, I
also have to be realistic about its actual impact in the industry.

 http://www.python.org/about/quotes/

There are always success stories; although, many of those are web based
companies which, if you will look up the thread, I already excuded
from the conversation.  However from my experience, a small percentage
of people make their primary income from writing non-web based Python
software.  It certainly hasn't displaced, or even encroached on the
primary langauge leaders.

 Sometimes giving access to the source code is a feature, not a bug. Just 
 ask Red Hat. And for those who think otherwise, you can always ship 
 the .pyc files alone. Or turn your software into a web-app.

Red Hat is the big example; but, few other companies actually make
substantial profits from FOSS.  I am not interested in an idealogy debate
about whether exposing your source is good or bad; the fact is that those
who fund software developement have a strong preference to choosing closed
source.  Any software that doesn't provide good methods for closed sourced
software is going to suffer.

1. Distributing .pyc files still requires the user to have a Python
interpreter.

2. My understanding is that Python bytecode is not guaranteed to be
compatible between releases?

3. I have already excluded web apps where Python has a large market
penetration (were most of the dynamic languages have found
their niche).  My point pertains to those areas where Python
cannot, or doesn't, compete well with systems level languages
and that even for its web niche, it didn't arrive here overnight.
There was a long period where it was less likely the Perl to be
used for server side web scripting.

You can see the recent thread about a web based IDE for what I think
about many web applications.

 In any case, most companies, and individuals, follow the crowd. They do 
 what everybody else does. There are good reasons for this, as well as bad 
 reasons, but the end result is that most companies' software development 
 is, quite frankly, crap, using the wrong language and the wrong 
 methodology for the wrong reasons. If you doubt this, then perhaps you 

People funding software developement really don't care about the quality
of the software.  They care how much money it will make them.  If you
really think quality matters, then look at Microsoft's meteoric rise
to dominance.  Early Microsoft was not known for its superior quality.
Only in the last ten years, since they have had to contend with a really
poor reputation and competition from their own products have they started
to be conserned with writing better software.

 methodology for the wrong reasons. If you doubt this, then perhaps you 
 would like to explain why most software projects fail and those that 
 don't rarely come in on time or on budget?

Yes, lots of projects fail; but, the choice of language is seldom the
primary, or even a major contributing reason, for the failure.  Python
holds the distinction of having had a book written about one such failed
project.

 It would probably be crap regardless of what language they used 
 (Sturgeon's Law), but there are degrees of crap. Being optimized for 
 rapid development, at least with Python the average company will develop 
 their crap software five times as quickly and at a third the cost than if 
 they had chosen C++ or Java.

If I didn't think Python was a good language, I wouldn't be here.
Nevertheless, it isn't a good fit for many pieces of software where a
systems language is better suited.  Reasons include ease of distribution
without an interpeter, non-necessity of distributing source with the
product, ability to leverage multiple CPU and multicore systems, and
simple sequential performance.

Given that Python does't work well in many areas, we could just give up
and accept having to write C++ for our day jobs or we can look for a
language which bridges the gap between those tasks that require C++'s
level of control and those which work well for dynamic languages.
Java attempted to do that, and has the market share to show that the
concept works, but I think that it missed the mark for many needs.  It is
still much lower level then Python for purposes of rapid developement
and too slow to be competative for non-long-lived tasks

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-17, Chris Rebert c...@rebertia.com wrote:
 On Mon, Jan 17, 2011 at 1:12 AM, Tim Harig user...@ilthio.net wrote:
 On 2011-01-16, geremy condra debat...@gmail.com wrote:
 On Sun, Jan 16, 2011 at 3:03 AM, Tim Harig user...@ilthio.net wrote:
snip
 Personally, I think the time is ripe for a language that bridges the
 gap between ease of use dynamic languages with the performance and
 distribution capabilities of a full systems level language.

 I agree. That does not make Go that language, and many of the choices
 made during Go's development indicate that they don't think it's that
 language either. I'm speaking specifically of its non-object model,
 lack of exceptions, etc.
snip

 2. Go has a similar mechanism to exceptions, defer/panic/recover.  It does
        downplay

there use for less exceptional conditions in favor of function return
values; however, there is nothing preventing you from using them as you see
fit to do so.

 Downplay what exactly? Seems your paragraph got truncated.

Sorry.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

In comp.lang.python, you wrote:
 Tim Harig, 17.01.2011 13:25:
 If I didn't think Python was a good language, I wouldn't be here.
 Nevertheless, it isn't a good fit for many pieces of software where a
 systems language is better suited.  Reasons include ease of distribution
 without an interpeter, non-necessity of distributing source with the
 product, ability to leverage multiple CPU and multicore systems, and
 simple sequential performance.

 Given that Python does't work well in many areas, we could just give up
 and accept having to write C++ for our day jobs or we can look for a
 language which bridges the gap between those tasks that require C++'s
 level of control and those which work well for dynamic languages.
 Java attempted to do that, and has the market share to show that the
 concept works, but I think that it missed the mark for many needs.  It is
 still much lower level then Python for purposes of rapid developement
 and too slow to be competative for non-long-lived tasks.

 So seriously need to take a look at Cython.

 http://cython.org

One of the arguments for Python has always made is that you can optimize
it by writing the most important parts in C.  Perhaps that is a crutch
that has held the communty back from seeking higher performance solutions
in the language itself.

I prefer a single language as opposed to a creolization of two.  Go gives
me more less complete independence from C.  I can write pretty much
anything I would like using almost pure Go and because it generates a
native binary with similar performance to C, I don't need to resort to
using another language.  I certainly don't need to require anybody who
wants to use a program I have compiled to install an interpreter.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-17, geremy condra debat...@gmail.com wrote:
 On Mon, Jan 17, 2011 at 1:12 AM, Tim Harig user...@ilthio.net wrote:
 On 2011-01-16, geremy condra debat...@gmail.com wrote:
 I wouldn't say Go is narrowly targeted.  It's a systems language that can
 compete in the same domain with scripting languages.  It is true that most
 languages have long incubation periods; but, corporate support can change
 that quite a bit.  C#, being backed by Microsoft, managed to go mainstream
 pretty quickly.

 This seems... shall we say, overly optimistic. I've seen no evidence
 at all that Go is even trying to compete with scripting languages, and
 I don't know of anyone who would actually use it where a shell script
 might do. AFAICS, it is an unabashedly concurrency-centric systems

I don't write Python where shell scripts will do either.  I have rewritten
several simple Python scripts in Go.  I will not be leaving Python
completely; but, Go will likely replace Python for most of my large scale
programming needs.

 might do. AFAICS, it is an unabashedly concurrency-centric systems
 language for people who are willing to sacrifice some speed and
 control for legible code. That's not an impossibly narrow niche (and
 it certainly has deep pockets) but it *is* narrow compared to a
 general purpose language like Java or Python.

You are writing Go into far too narrow of a niche then it actually is.  In
fact, whatever language you are commenting on doesn't sound like Go at all.

Go is every bit of a general purpose programming language.  It is useful
for the same basic applications that you would otherwise write in Java.  It
can do pretty much anything that you can do in C, with the probable
exception of writing operating systems.  I have been using C/C++ for most
of my programming career; but, everything that I have done can be done with
Go.  I am not really sure where you are seeing such narrowing limitations.

Support for concurrency is really icing on the cake.  I find it rather
supprising that so many modern languages do not already support full
concurrency constructs.

 Python's very widely used for scripting and related tasks, and has a
 pretty big user base in academia and the sciences.

 Python has been widely used by people like us that happen to like the
 language and found ways to use it in our workplaces; but, most of the
 time it is an unofficial use that the company.  You still don't see many
 companies doing large scale internal development using Python and you
 definately don't see any doing external developement using a language
 that gives the customers full access to the source code.

 Right, I mean, it's not like the company that wrote Go would *ever*
 stoop to using Python ;)

I have never said that there are not companies doing developement in
Python and certainly not for web programming; but, a handful of companies
that have adopted Python doesn't make it mainstream.  Good luck finding
many Python jobs in my area.  Java, C#, Visual Basic, PHP, *COBOL*,
RPG-IV, and, to a lesser extent, C++ jobs are available in abundance.
The few Python jobs that are available are for web based programming.

 1. Go has an object model.  What it lacks is an object hierarchy where all
        object are decended from a single root object since it does
        not support object inheritance as it is used in most languages.
        In Go we simply adapt an object to meet the needs of the newer
        object by adding whatever new functionality is needed.

 Go has structs. Its structs are not objects, principally because they
 can't do real inheritance. You can do similar things ('structural
 inheritance') to C structs, and nobody argues that C is OO.

http://www.infoq.com/interviews/johnson-armstrong-oop

Listen to the third point.  Object orientation is quite possible even
without inheritance.  Not all programming languages support the same set
of OOP features.  Not all classless OOP programming languages support
inheritance.  The basic definition of object oriented programming is
programing with self contained objects which contain their own data and
the procedures necessary to manipulate that data.

 2. Go has a similar mechanism to exceptions, defer/panic/recover.  It does
        downplay

 Defer, panic, and recover only allow you to build a recovery stack.
 That's like saying that try/except isn't needed anymore because you
 have the with statement. Do you think you could get through a rewrite
 of Django without ripping out some hair over that?

I don't know what you are referencing about Django as I don't work on or
with the project; but, if one wanted exceptions with similar sematics to
Python's, it could easily be built around Go's defer/panic/recover.  There
are actually packages availble that recreate exceptions as they are used in
Ruby and other languages.  The only think defer/panic/recover lacks is an
exception class containing more more detailed information about what
actually went wrong.

 The current

Re: [OT] Python like lanugages

[Tim Harig]

On 2011-01-17, John Nagle na...@animats.com wrote:
  That's been done once or twice.  There's what are called single
 assignment languages.  Each variable can only be assigned once.
 The result looks like an imperative language but works like a functional
 language.  Look up SISAL for an example.  This approach is good
 for concurrency and optimization, but it never caught on.

That is rather interesting; especially since most of the current hype is
about immutability and the lack of side affects of functional languages
rather then functional decomposition using lamda calculus constructs.

  Note that from a concurrency standpoint, immutability is very
 useful.  It's a lot like single assignment.  You can pass around
 immutable objects to concurrent threads without race conditions,
 provided that memory management can handle concurrency.  This works

I wonder if even that is fully necessary.  In, Ada procedure arguments
specify whether or not they can be changed within the procedure effectively
creating a gateway whereby side affects can be specified.  If you take that
a step further and require:

1. The function of a scope is completely closed.  It cannot access anything
from the scope above it except what is passed to it as arguments.

2. Arguments that can be changed must be copied on write so that they don't
directly affect the memory of the outer scope.

Then you can be sure that the functions have no side affects outside of the
values that they return.  Internally, the function can use imperative
semantics that require mutability such as iteration instead recursion
without affecting anything above it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-8 question from Dive into Python 3

[Tim Harig]

On 2011-01-17, carlo syseng...@gmail.com wrote:
 Is it true UTF-8 does not have any big-endian/little-endian issue
 because of its encoding method? And if it is true, why Mark (and
 everyone does) writes about UTF-8 with and without BOM some chapters
 later? What would be the BOM purpose then?

Yes, it is true.  The BOM simply identifies that the encoding as a UTF-8.:

http://unicode.org/faq/utf_bom.html#bom5

 2- If that were true, can you point me to some documentation about the
 math that, as Mark says, demonstrates this?

It is true because UTF-8 is essentially an 8 bit encoding that resorts
to the next bit once it exhausts the addressible space of the current
byte it moves to the next one.  Since the bytes are accessed and assessed
sequentially, they must be in big-endian order.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-17, geremy condra debat...@gmail.com wrote:
 On Mon, Jan 17, 2011 at 1:07 PM, Tim Harig user...@ilthio.net wrote:
 On 2011-01-17, geremy condra debat...@gmail.com wrote:
 On Mon, Jan 17, 2011 at 1:12 AM, Tim Harig user...@ilthio.net wrote:
 On 2011-01-16, geremy condra debat...@gmail.com wrote:
 Go is every bit of a general purpose programming language.  It is useful
 for the same basic applications that you would otherwise write in Java.  It
 can do pretty much anything that you can do in C, with the probable
 exception of writing operating systems.  I have been using C/C++ for most
 of my programming career; but, everything that I have done can be done with
 Go.  I am not really sure where you are seeing such narrowing limitations.

 Some people get very attached to one language and use it for
 everything. I'd rather use the right tool for the job.

I much agree.  You don't see me dropping python, shell, awk, etc.  I do
expect Go will enter some of the programming that I am doing in Python and
it will almost certainly replace much of the programming that I am doing in
C/C++.

 Go is not an ideal language for high-performance code. Despite the
 occasional claims of others, Go is consistently outperformed by C,
 C++, and Java on a wide variety of benchmarks. Some claim that Ada and
 Haskell do as well, and my benchmarks (CPU bound, focused on speed of

I much agree that Go doesn't beat C or C++.  I really doubt that it could
with the runtime necessary to do garbage collction.  Nevertheless, I find
that Go can be optimized to perform within tolerable limits of C given the
boost it gives in development.

I really question that you get Java anywhere even close to C performance.
Google reports they get within the same order of magnitude as C for
their long-lived server processes where the JIT has had time to optimize
its results.  For shorter term processes such as desktop applications,
Java performance stinks -- even after you discount the JVM starup time.

Ada is capable of C++ like performance *if* you compile it to remove *all*
of runtime checking.  Depending what checks you enable, it can run much
slower.

 cryptographic primitives) *seem* to confirm that for Haskell. I say
 'seem' because I'm clearly much more familiar with Haskell than with
 Go, and would be inclined to question my conclusions if they weren't

I have not worked with Haskell, so I don't really have any idea's about its
performance.  I understand that leveraging lazy evaluation and result
caching can provide some *huge* opportunities for optimizations.

 in line with the work of others. You can argue that it's good enough-
 it is, for most cases- but taking a 20% performance hit rules it out
 of a lot of systems work, and the C-Go gap in many cases is much
 larger than that.

I don't work anything that involves and absolute need for performance.
I could probably accept penalty several times that of C for higher
level functionality; but, sometimes the penalty for Python is just
too much.  Many of my programs are very quick lived such that even
loading an interpeter or VM can eclipse the actual runtime.  Given less
developmental effort required, I also find that I have more time to look
for ways to optimize Go.  There are many things (such as using alternate
data structures rather then the build in slices and immutable strings)
that can be used to accelerate Go when time permits and I suspect many
more will be found as the language matures.

 Go is also not an ideal language for enterprise development. It
 provides no compatibility or support guarantees; in fact, the website
 describes it as 'an experiment' and recommends it for 'adventurous

There is no doubt that it is a young project and there are a number of
things that will need to be improved to be effective for some branches
of programming; but, that isn't a language restriction.  Frankly, I am
rather impressed by the sheer number of third party packages that have
already become available.  No go isn't going to replace some of the other
top mainstream langauges today; but, I can easily see it starting to see
some market penetration 5 years from now.

 users'. There are no independent experts in case something goes wrong,
 and if it goes belly up a year from now you will have a one year old
 piece of legacy infrastructure on your hands. Maybe you don't think
 that this will be the case; in any event, I would not want to try
 convincing a CFO of that.

If I was trying to convince a CFO, I would ask if he really thought Google
was likely to simply drop the time and effort that Google has already
placed into the infrastructure.  Furthermore, few dying languages already
have not one, but two, open source compilers available for use.  

 As you point out, Go is also not a good language for operating systems
 work. The lack of a widely accepted GUI toolkit weakens your argument
 about application development. The lack of third party tools and
 dependence on foreign language

Re: After C++, what with Python?

[Tim Harig]

On 2011-01-16, John Nagle na...@animats.com wrote:
 On 1/15/2011 10:48 PM, Aman wrote:
 @nagle Means you are suggesting me not to proceed with Python because I've 
 had experience with C++?

No, Python is quite useful, but on the slow side.  If you're I/O
 bound, not time critical, or otherwise not performance constrained,
 it's quite useful.  The language is really quite good, but there's
 some excessive dynamism which has caused every attempt at an optimizing
 implementation to fail.

Those who are concerned about performance should check out Go.
Garbage collection, duck typing, and compiles to a native binary.
It creates a great middle ground between C++ and Python.  Any C and/or
Python programmer will feel right at home with the language.  It is
still a young language; but, I have been using it for some useful things.

http://golang.org
-- 
http://mail.python.org/mailman/listinfo/python-list

[OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-16, Paul Rubin no.email@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 Those who are concerned about performance should check out Go.
 Garbage collection, duck typing, and compiles to a native binary.
 It creates a great middle ground between C++ and Python.  Any C and/or
 Python programmer will feel right at home with the language.  It is
 still a young language; but, I have been using it for some useful things.

 Go has some nice aspects but it is much lower level than Python.  If you

It is a little lower; but, I wouldn't say much lower.  My Go code is
much more similar in concept, feel, and size to my Python code then it
is to my C code.

 want a statically typed, compiled language closer to Python's level, I
 know of some projects that have switched from Python to Ocaml.  If you

I have head good things about Ocaml; but, I have never taken the time to
learn the language myself.  It never reached a critical mass of interest
from me to consider adopting it.  One of the things that gives me hope
for Go is that it is backed by Google so I expect that it may gain some
rather rapid adoption.  It has made enough of a wake to grab one of
Eweek's 18 top languages for 2011.

http://www.eweek.com/c/a/Application-Development/Java-C-C-Top-18-Programming-Languages-for-2011-480790/

 want dynamic types, I guess there's Dylan, Lisp, or possibly Erlang.

I am a big fan of Erlang and it's ability to create fault tolerant
systems; but, it isn't really a general purpose programming language.
It also runs inside of a VM which means that it doesn't produce native
binary.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Python like lanugages [was Re: After C++, what with Python?]

[Tim Harig]

On 2011-01-16, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 On Sun, 16 Jan 2011 09:47:35 +, Tim Harig wrote:

 One of the things that gives me hope
 for Go is that it is backed by Google so I expect that it may gain some
 rather rapid adoption.  It has made enough of a wake to grab one of
 Eweek's 18 top languages for 2011.

 If the author thinks that Go is a tried and true (his words, not mine) 
 language where programmers can go to look for work, I think he's 
 fooling himself.

No I wouldn't say that it has reached market penetration yet; but, it
has more momentum then any language I am familiar with.  I wouldn't be
at all surprised to see it becoming quite common in the next five years.

How long has it taken Python to reach its present level of market
penetration?  And, I still don't see a huge amount of professional Python
use outside of web developement.  Go has only been public for less then
a year.

Personally, I think the time is ripe for a language that bridges the
gap between ease of use dynamic languages with the performance and
distribution capabilities of a full systems level language.  This is after
all the promise the VM based languages made but never really fulfilled.
It is also high time for a fully concurrent language fully capable of
taking advantage of multicore processors without having to deal with the
inherent dangers of threading.  There are several good choices available
for both a even a few that fit both bills; but, few of them have the
support of a company like Google that is capable of the push required
to move the language into the mainstream.

 When I design my new language, I will make sure I choose a name such that 
 any attempt to search for it on job sites will produce oodles and oodles 
 and oodles of false positives, all the better to ensure that simple-
 minded top language of ... surveys will give a massively inflated job 
 count.

I would agree that Go wasn't the best idea for a language name from the
search perspective.  One would have though a company like Google would have
been cognizant of those limitations...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Developing a program to make a family tree.

[Tim Harig]

On 2011-01-14, Ata Jafari a.j.romani...@gmail.com wrote:
 I'm trying to develop a program like family tree maker. I have all
 information, so there is no need to search on the net. This must be
 something like trees. Can someone help me? I'm at the beginning.

I don't know anything specific about family tree software and you don't
really specify what you want your software to do.  I can only assume
that you are interested in taking the data in some format which contains
the links between family members and creating a tree representation of
that data?

If I was going to attempt something like this, I would probably generate
the representation as a set of postscript instructions.  I would start
with a basic template for a union which could be essentially pasted
into different places in the output page.  Then generating your tree
is a simple matter of laying out the templates to match your data,
filling in the template fields for each persion within their union,
and drawing the connecting lines of the tree.

Since I was already generating postscript anyway, I would probably
implement much of the actual logic in postscript (the built in stacks
provide an exellent way to process tree like structures in all of their
nested levels). I would Python provide any user interface for manipulating
the data and to dump the data into the postscript program.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested structures question

[Tim Harig]

On 2011-01-12, Physics Python physicsandpyt...@gmail.com wrote:
 while guess != the_number:

=
 while tries  7:
 if guess  the_number:
 print Lower...
 else:
 print Higher...
 guess = int(raw_input(Take a guess: ))
 tries += 1
=

Think about what happens when this nested loop exits because tries  7?  It
returns to the outer loop whether or not the actual number was guessed
correctly.  There is no real need for this loop.
 
 print You guessed it! The number was: , the_number
 print And it only took you, tries, tries!\n

Note that the outer loop ends here without any test to see whether or not
the number was actually guested and there is *nothing* that stops this
outer loop, so it will spin forever.

 print Wow, you suck at this, you should be able to solve this in 7 attempts 
 or less

 raw_input(Press Enter to exit the program.)

This is never reached.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested structures question

[Tim Harig]

[wrapped lines to 80 characters per RFC 1855]
On 2011-01-12, Physics Python physicsandpyt...@gmail.com wrote:
 Is this an indentation problem then?

That depends how you look at it.  I was not clear from your code exactly
where you wanted to handle things.

 How do I update the sentinel within the secondary while loop. I am
 trying to avoid using breaks by the way, as I can program this example
 using breaks:

You don't need breaks.

 import random
 print \tWelcome to 'Guess my number'!:
 print \nI'm thinking of a number between 1 and 100.
 print Try to guess it in as few attempts as possible.\n

 the_number = random.randrange(1,101)

 tries = 0

 while True:

while can be used to test for more then a single condition at a time using
and/or chains.

 else:
 print You guessed it! The number was, the_number
 print And it only took you, tries, tries!\n
 break
 if  tries == 7:
 print Wow you suck! It should only take at most 7 tries!
 break

Both of these tests can be performed as part of the loop itself.  The end
results can therefore be tested and handled outside of the loop without
using breaks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested structures question

[Tim Harig]

On 2011-01-12, Jason Staudenmayer jas...@adventureaquarium.com wrote:
 Return False instead of break should work

 else:
  print You guessed it! The number was, the_number
  print And it only took you, tries, tries!\n
  return False

Since he isn't in a function, that isn't any good.  He would import sys and
use sys.exit() but that rather defeats the purpose of having a single entry
and exit point to the loop.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested structures question

[Tim Harig]

In case you still need help:

- # Set the initial values
- the_number= random.randrange(100) + 1
- tries = 0
- guess = None
-
- # Guessing loop
- while guess != the_number and tries  7:
- guess = int(raw_input(Take a guess: ))
- if guess  the_number:
- print Lower...
- elif guess  the_number:
- print Higher...
- tries += 1
- 
- # did the user guess correctly to make too many guesses?
- if guess == the_number:
- print You guessed it! The number was, the_number
- print And it only took you, tries, tries!\n
- else:
- print Wow you suck! It should only take at most 7 tries!
- 
- raw_input(Press Enter to exit the program.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Google Chart API, HTTP POST request format.

[Tim Harig]

On 2011-01-06, Chris Rebert c...@rebertia.com wrote:
 On Wed, Jan 5, 2011 at 11:21 PM, Garland Fulton stacks...@gmail.com wrote:
 On Wed, Jan 5, 2011 at 7:26 PM, Tim Harig user...@ilthio.net wrote:
    Python 3.1.2 (r312:79147, Oct  9 2010, 00:16:06)
    [GCC 4.4.4] on linux2
    Type help, copyright, credits or license for more information.
     import urllib.request, urllib.parse
     params = urllib.parse.urlencode({'cht':'lc', 'chtt':'This is | my
     chart',

Sorry I didn't notice this got accidently wrapped when I pasted it.
 params = urllib.parse.urlencode({'cht':'lc', 'chtt':'This is | my 
chart',

    ...         'chs':'600x200', 'chxt':'x,y', 'chd':'t:40,20,50,20,100'})
     chart =
 urllib.request.urlopen('https://chart.googleapis.com/chart',
    ...         data = params).read()
     chartFile = open(chart.png, 'wb')
     chartFile.write(chart)
    10782
     chartFile.close()

 Hope this isn't to stupid,
 For the
 chart = urllib.request.urlopen('https://chart.googleapis.com/chart', data =
 params).read()
 Where would I find information on why and what the ).read() part does.

For some reason, posts from from this account don't seem to be making it
through the gateway to usenet so I am only seeing what Mr. Rebert has
replied to.  If you have asked anything else, I very well may have missed
it.

 http://docs.python.org/py3k/library/urllib.request.html#urllib.request.urlopen
 Specifically: This function returns a file-like object (representing
 the stream of data received). Thus, .read() on the file-like object
 returns the actual bytes obtained from the given URL.

Mr. Rebert already answed your question; but, I will expand on that a
little bit.  One of the great things about the Python language is that it
uses what is commonly known as duck typing.  That is anything object which
provides the same attributes as another object can be used as though it is
actually the second object.  It is kind of like an implicit form of
generics that doesn't require a template or an interface.

The Python standard library makes extensive use of duck typing for file
like objects.  Any object that provides the proper method attributes can be
given to functions that expect files even though the object is given might
not be the traditional concept of a file on the filesystem.  It might be a
stringIO object, a socket file object, or something new that you have
created that supports the required method attributes.

The semantics and documentation for file like objects have changed a
little for python2 vs. python3:

python2: http://docs.python.org/library/stdtypes.html#file-objects
python3: http://docs.python.org/py3k/library/io.html#io.IOBase

but they still basically work the same way.  Much of the Python 3
documentation still refers file objects.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP: possibility of inline using of a symbol instead of import

[Tim Harig]

On 2011-01-06, dmitrey dmitrey.kros...@scipy.org wrote:
 and after several pages of code they are using somewhere, maybe only
 one time, e.g.
[SNIP]
 It makes programs less clear, you have to scroll several pages of code
 in IDE to understand what it refers to.

Python doesn't require imports to be at the top of a file.  They can be
imported at any time.

 import MyModule
 (...lots of code...)
 r = MyModule.myFunc(...)

(...lots of code...)
import MyModule
r = MyModule.myFunc(...)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP: possibility of inline using of a symbol instead of import

[Tim Harig]

On 2011-01-06, dmitrey dmitrey.kros...@scipy.org wrote:
[re-ordered]
 On Jan 6, 5:57 pm, Tim Harig user...@ilthio.net wrote:
 Python doesn't require imports to be at the top of a file.  They can be
 imported at any time.

  import MyModule
  (...lots of code...)
  r = MyModule.myFunc(...)

 (...lots of code...)
 import MyModule
 r = MyModule.myFunc(...)

 Yes, I know, still usually it is placed in file header

1. Don't top post.

2. Your so-called PEP probably clashes with Python's use of @ for
decorators.

3. Do you really expect a language holding the mantra that there should be
a single way of doing things to embrace a language bloating feature
for what is effectively already possible with the language as it
exists?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Graphing API,

[Tim Harig]

On 2011-01-05, Slie stacks...@gmail.com wrote:
 Is there a graphing API, someone suggests?

You should check the archives, variations of this question get asked
a lot.

I use GNUplot to do my graphing.  I simply pipe it commands and data
through the subprocess module; but, there are libraries available for
interacting with it.

Posts here also indicate that Google offers a web service based API for
generating graphs.  I have never actually used it; but, the documentation
seems to be clear enough to get it working without too much trouble.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Google Chart API, HTTP POST request format.

[Tim Harig]

On 2011-01-06, Slie stacks...@gmail.com wrote:
[reformated to 80 columns per RFC 1855 guidelines]
 I have read several examples on python post requests but I'm not sure
 mine needs to be that complicated.

From the HTML example on the page you posted:

form action='https://chart.googleapis.com/chart' method='POST'
input type=hidden name=cht value=lc  /
input type=hidden name=chtt value=This is | my chart  /
input type='hidden' name='chs' value='600x200' /
input type=hidden name=chxt value=x,y /
input type='hidden' name='chd' value='t:40,20,50,20,100'/
input type=submit  /
/form

you can retreive the same chart from Python:

Python 3.1.2 (r312:79147, Oct  9 2010, 00:16:06)
[GCC 4.4.4] on linux2
Type help, copyright, credits or license for more information.
 import urllib.request, urllib.parse
 params = urllib.parse.urlencode({'cht':'lc', 'chtt':'This is | my
 chart',
... 'chs':'600x200', 'chxt':'x,y', 'chd':'t:40,20,50,20,100'})
 chart = urllib.request.urlopen('https://chart.googleapis.com/chart',
... data = params).read()
 chartFile = open(chart.png, 'wb')
 chartFile.write(chart)
10782
 chartFile.close()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: User input masks - Access Style

[Tim Harig]

On 2010-12-31, flebber flebber.c...@gmail.com wrote:
 On Dec 28 2010, 12:21 am, Adam Tauno Williams awill...@whitemice.org
 wrote:
 On Sun, 2010-12-26 at 20:37 -0800, flebber wrote:
  Is there anyay to use input masks in python? Similar to the function
  found in access where a users input is limited to a type, length and
  format.

 http://faq.pygtk.org/index.py?file=faq14.022.htpreq=show

 Typically this is handled by a callback on a keypress event.

 Regarding 137 of the re module, relating to the code above.

137? I am not sure what you are referencing?

 EDIT: I just needed to use raw_input rather than input to stop this
 input error.

Sorry, I used input() because that is what you had used in your example
and it worked for my system.  Normally, I would have used window.getstr()
from the curses module, or whatever the platform equivilant is, for
getting line buffered input.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-27, Alan Meyer amey...@yahoo.com wrote:
 On 12/26/2010 3:15 PM, Tim Harig wrote:
 ...
 The problem is that XML has become such a defacto standard that it
 used automatically, without thought, even when there are much better
 alternatives available.

 I agree with you but, as you say, it has become a defacto standard.  As 
 a result, we often need to use it unless there is some strong reason to 
 use something else.

XML should be used where it makes sense to do so.  As always, use the
proper tool for the proper job.  XML became such a defacto standard, in
part, because it was abused for many uses in the first place so using it
because it is a defacto standard is just piling more and more mistakes
on top of each other.

 The same thing can be said about relational databases.  There are 
 applications for which a hierarchical database makes more sense, is more 
 efficient, and is easier to understand.  But anyone who recommends a 
 database that is not relational had better be prepared to defend his 
 choice with some powerful reasoning because his management, his 
 customers, and the other programmers on his team are probably going to 
 need a LOT of convincing.

I have no particular problem with using other database models in
theory.  In practice, at least until recently, there were few decent
implementations for alternative model databases.  That is starting to
change with the advent of the so-called NoSQL databases.  There are a few
models that I really do like; but, there are also a lot of failed models.
A large part of the problem was the push towards object databases which
is one of the failed models IMNSHO.  Its failure tended to give some of
the other datase models a bad name.

 And of course there are many applications where XML really is the best. 
   It excels at representing complex textual documents while still 
 allowing programmatic access to individual items of information.

Much agreed.  There are many things that XML does very well.  It works
great for XMP-RPC style interfaces.  I prefer it over binary formats
for documents.  It does suitibly for exporting discreet amounts of
information.

There are however a number of things that it does poorly.  I don't
condone its use for configuration files.  I don't condone its use as a
data store and when you have data approaching gigabytes, that is exaclty
how you are using it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-26, Nobody nob...@nowhere.com wrote:
 On Sun, 26 Dec 2010 01:05:53 +, Tim Harig wrote:

 XML is typically processed sequentially, so you don't need to create a
 decompressed copy of the file before you start processing it.
 
 Sometimes XML is processed sequentially.  When the markup footprint is
 large enough it must be.  Quite often, as in the case of the OP, you only
 want to extract a small piece out of the total data.  In those cases,
 being forced to read all of the data sequentially is both inconvenient and
 and a performance penalty unless there is some way to address the data you
 want directly.

 OTOH, formats designed for random access tend to be more limited in their
 utility. You can only perform random access based upon criteria which
 match the format's indexing. Once you step outside that, you often have to
 walk the entire file anyhow.

That may be true and it may not.  Even assuming that you have to walk
through a large number of top level elements there may be an advantage
to being able to directly access the next element as opposed to having
to parse through the entire current element once you have determined it
isn't one which you are looking for.  To be fair, this may be invalid
preoptimization without taking into account how the hard drive buffers;
but, I would suspect that there is a threshold where the amount of
data skipped starts to outweigh the penalty of overreaching the hard
drives buffers.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-26, Stefan Behnel stefan...@behnel.de wrote:
 Tim Harig, 26.12.2010 02:05:
 On 2010-12-25, Nobodynob...@nowhere.com  wrote:
 On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote:
 Of course, one advantage of XML is that with so much redundant text, it
 compresses well.  We typically see gzip compression ratios of 20:1.
 But, that just means you can archive them efficiently; you can't do
 anything useful until you unzip them.

 XML is typically processed sequentially, so you don't need to create a
 decompressed copy of the file before you start processing it.

 Sometimes XML is processed sequentially.  When the markup footprint is
 large enough it must be.  Quite often, as in the case of the OP, you only
 want to extract a small piece out of the total data.  In those cases, being
 forced to read all of the data sequentially is both inconvenient and and a
 performance penalty unless there is some way to address the data you want
 directly.

 So what? If you only have to do that once, it doesn't matter if you have to 
 read the whole file or just a part of it. Should make a difference of a 
 couple of minutes.

Much agreed.  I assume that the process needs to be repeated or it
probably would be simpler just to rip out what I wanted using regular
expressions with shell utilities.

 If you do it a lot, you will have to find a way to make the access 
 efficient for your specific use case. So the file format doesn't matter 
 either, because the data will most likely end up in a fast data base after 
 reading it in sequentially *once*, just as in the case above.

If the data is just going to end up in a database anyway; then why not
send it as a database to begin with and save the trouble of having to
convert it?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-26, Stefan Behnel stefan...@behnel.de wrote:
 Tim Harig, 26.12.2010 10:22:
 On 2010-12-26, Stefan Behnel wrote:
 Tim Harig, 26.12.2010 02:05:
 On 2010-12-25, Nobody wrote:
 On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote:
 Of course, one advantage of XML is that with so much redundant text, it
 compresses well.  We typically see gzip compression ratios of 20:1.
 But, that just means you can archive them efficiently; you can't do
 anything useful until you unzip them.

 XML is typically processed sequentially, so you don't need to create a
 decompressed copy of the file before you start processing it.

 Sometimes XML is processed sequentially.  When the markup footprint is
 large enough it must be.  Quite often, as in the case of the OP, you only
 want to extract a small piece out of the total data.  In those cases, being
 forced to read all of the data sequentially is both inconvenient and and a
 performance penalty unless there is some way to address the data you want
 directly.
  [...]
 If you do it a lot, you will have to find a way to make the access
 efficient for your specific use case. So the file format doesn't matter
 either, because the data will most likely end up in a fast data base after
 reading it in sequentially *once*, just as in the case above.

 If the data is just going to end up in a database anyway; then why not
 send it as a database to begin with and save the trouble of having to
 convert it?

 I don't think anyone would object to using a native format when copying 
 data from one database 1:1 to another one. But if the database formats are 
 different on both sides, it's a lot easier to map XML formatted data to a 
 given schema than to map a SQL dump, for example. Matter of use cases, not 
 of data size.

Your assumption keeps hinging on the fact that I should want to dump
the data into a database in the first place.  Very often I don't.
I just want to rip out the small portion of information that happens to
be important to me.  I may not even want to archive my little piece of
the information once I have processed it.

Even assuming that I want to dump all the data into a database,
walking through a bunch of database records to translate them into the
schema for another database is no more difficult then walking through a
bunch of XML elements.  In fact, it is even easier since I can use the
relational model to reconstruct the information in an organization that
better fits how the data is actually structured in my database instead
of being constrained by how somebody else wanted to organize their XML.
There is no need to map a[sic] SQL dump.

XML is great when the data is set is small enough that parsing the
whole tree has negligable costs.  I can choose whether I want to parse
it sequentially or use XPath/DOM/Etree etc to make it appear as though
I am making random accesses.  When the data set grows so that parsing
it is expensive I loose that choice even if my use case would otherwise
prefer a random access paradigm.  When that happens, there are better
ways of communicating that data that doesn't force me into using a high
overhead method of extracting my data.

The problem is that XML has become such a defacto standard that it
used automatically, without thought, even when there are much better
alternatives available.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: User input masks - Access Style

[Tim Harig]

On 2010-12-27, flebber flebber.c...@gmail.com wrote:
 Is there anyay to use input masks in python? Similar to the function
 found in access where a users input is limited to a type, length and
 format.

 So in my case I want to ensure that numbers are saved in a basic
 format.
 1) Currency so input limited to 000.00 eg 1.00, 2.50, 13.80 etc

Some GUIs provide this functionality or provide callbacks for validation
functions that can determine the validity of the input.  I don't know of
any modules that provide formatted input in a terminal.  Most terminal
input functions just read from stdin (in this case a buffered line)
and output that as a string.  It is easy enough to validate whether
terminal input is in the proper.

Your example time code might look like:

... import re
... import sys
... 
... # get the input
... print(Please enter time in the format 'MM:SS:HH': , end=)
... timeInput = input()
... 
... # validate the input is in the correct format (usually this would be in
... # loop that continues until the user enters acceptable data)
... if re.match(r'''^[0-9]{2}:[0-9]{2}:[0-9]{2}$''', timeInput) == None:
... print(I'm sorry, your input is improperly formated.)
... sys.exit(1)
... 
... # break the input into its componets
... componets = timeInput.split(:)
... minutes = int(componets[0])
... seconds = int(componets[1])
... microseconds = int(componets[2])
... 
... # output the time
... print(Your time is:  + %02d % minutes + : + %02d % seconds + : +
... %02d % microseconds)

Currency works the same way using validating it against:
r'''[0-9]+\.[0-9]{2}'''

 For sports times that is time duration not a system or date times
 should I assume that I would need to calculate a user input to a
 decimal number and then recalculate it to present it to user?

I am not sure what you are trying to do or asking.  Python provides time,
date, datetime, and timedelta objects that can be used for date/time
calculations, locale based formatting, etc.  What you use, if any, will
depend on what you are actually tring to accomplish.  Your example doesn't
really show you doing much with the time so it is difficult giving you any
concrete recommendations.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: User input masks - Access Style

[Tim Harig]

On 2010-12-27, Tim Harig user...@ilthio.net wrote:
 ... if re.match(r'''^[0-9]{2}:[0-9]{2}:[0-9]{2}$''', timeInput) == None:
[SNIP]
 Currency works the same way using validating it against:
 r'''[0-9]+\.[0-9]{2}'''

Sorry, you need to check to make sure that there are no trailing characters
as in the example above.  Checking the beginning is not actually necessary
with match().

r'''^[0-9]+\.[0-9]{2}$'''
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-25, Steve Holden st...@holdenweb.com wrote:
 On 12/23/2010 4:34 PM, Stefan Sonnenberg-Carstens wrote:
 For large datasets I always have huge question marks if one says xml.
 But I don't want to start a flame war.

I would agree; but, you don't always have the choice over the data format
that you have to work with.  You just have to do the best you can with what
they give you.

 I agree people abuse the spirit of XML using it to transfer gigabytes
 of data, but what else are they to use?

Something with an index so that you don't have to parse the entire file
would be nice.  SQLite comes to mind.  It is not standardized; but, the
implementation is free with bindings for most languages.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-25, Nobody nob...@nowhere.com wrote:
 On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote:
 XML works extremely well for large datasets.
 One advantage it has over many legacy formats is that there are no
 inherent 2^31/2^32 limitations. Many binary formats inherently cannot
 support files larger than 2GiB or 4Gib due to the use of 32-bit offsets in
 indices.

That is probably true of many older and binary formats; but, XML
is certainly not the the only format that supports arbitrary size.
It certainly doesn't prohibit another format with better handling of
large data sets from being developed.  XML's primary benefit is its
ubiquity.  While it is an excellent format for a number of uses, I don't
accept ubiquity as the only or preeminent metric when choosing a data
format.

 Of course, one advantage of XML is that with so much redundant text, it 
 compresses well.  We typically see gzip compression ratios of 20:1.  
 But, that just means you can archive them efficiently; you can't do 
 anything useful until you unzip them.

 XML is typically processed sequentially, so you don't need to create a
 decompressed copy of the file before you start processing it.

Sometimes XML is processed sequentially.  When the markup footprint is
large enough it must be.  Quite often, as in the case of the OP, you only
want to extract a small piece out of the total data.  In those cases, being
forced to read all of the data sequentially is both inconvenient and and a
performance penalty unless there is some way to address the data you want
directly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-25, Adam Tauno Williams awill...@whitemice.org wrote:
 On Sat, 2010-12-25 at 22:34 +, Nobody wrote:
 On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote:
 XML is typically processed sequentially, so you don't need to create a
 decompressed copy of the file before you start processing it.

 Yep.

Sometimes that is true and sometimes it isn't.  There are many situations
where you want to access the data nonsequentially or address just a small
subset of it.  Just because you never want to access data randomly doesn't
mean others might not.  Certainly the OP would be happier using something
like XPath to get just the piece of data that he is looking for.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Design Ideals Goals Python 3 - Forest for the trees

[Tim Harig]

On 2010-12-26, flebber flebber.c...@gmail.com wrote:
 I was hoping someone could shed some (articles, links) in regards
 python 3 design ideals. I was searching guido's blog which has his
 overarching view of Python from an early development perspective
 http://python-history.blogspot.com/2009/01/pythons-design-philosophy.html
 .

 I was interested in what the design goals/philosphy was of Python 3
 from a birds eye view, forest for the trees approach.

This is rather old; but you might watch:

http://www.youtube.com/watch?v=1RjtT17WbcQ

if you haven't seen it already.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Web App

[Tim Harig]

On 2010-12-23, Daniel Fetchinson fetchin...@googlemail.com wrote:
 I don't personally think the web makes a good framework for highly
 interactive applications as they must work within the constraints of the
 browser and IDEs are highly interactive applications by their very nature.
 Perhaps HTML5/CSS3 will change things; but, standard DOM manipulation,
 as I am accustomed to seeing it, cannot generate the kind of rendering
 that is available from native applications.  Attempts to do so end up being
 kludgy.

 It also cannot handle the kinds of desktop integrations that are common
 for native applications without opening up serious security trust issues.
 (Can everybody say ActiveX fiasco?)

 So, in essence, you are predicting that google's chrome OS will be a
 failure, right?

No, most people are happy using web based email interfaces and never even
know that native email clients exist.  More is the pity.

 Finally, there are difficulties in handling keystrokes without conflicting
 with the browser's native key bindings.  I seldom ever touch a mouse
 and I am a huge fan of vi, mutt, slrn, screen, ratpoison, etc. where
 the primary interface is totally accessable through the keyboard without
 having to tab through many options.

 Well, implementing vi or other text based tools in the browser is
 trivial. I mean it will function in exactly the same way as a native
 vi.

Not exactly.  I occassionally use web based terminals (Ajaxterm, Anyterm,
Shellinabox, etc) to access my systems.  This works only partially since
many of the keystrokes I use conflict with keystrokes that the browser
uses or which cause signals that the browser either does not catch or does
not pass on to be accessed by client side scripting.  The terminals must
therefore place buttons or synthetic keyboards on the screen to allow
you to simulate the keystrokes.  That kind of negates the advantages of
keystrokes in the first place.  It doesn't make fore a pleasant experience.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Web App

[Tim Harig]

On 2010-12-22, Sean secr...@gmail.com wrote:
 Anybody know where I can find a Python Development Environment in the
 form of a web app for use with Chrome OS. I have been looking for a
 few days and all i have been able to find is some old discussions with
 python developers talking about they will want one for the OS to be a
 success with them.

Personally, I think a web app based IDE would be ghastly; but, you might
have a look at Mozilla Skywriter (formerly Bespin):

https://mozillalabs.com/skywriter/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Web App

[Tim Harig]

[Reordered to preserve context in bottom posting]
On 2010-12-23, Hidura hid...@gmail.com wrote:
 2010/12/22, Tim Harig user...@ilthio.net:
 On 2010-12-22, Sean secr...@gmail.com wrote:
 Anybody know where I can find a Python Development Environment in the
 form of a web app for use with Chrome OS. I have been looking for a
 few days and all i have been able to find is some old discussions with
 python developers talking about they will want one for the OS to be a
 success with them.

 Personally, I think a web app based IDE would be ghastly; but, you might
 have a look at Mozilla Skywriter (formerly Bespin):

 Why grashtly?

I don't personally think the web makes a good framework for highly
interactive applications as they must work within the constraints of the
browser and IDEs are highly interactive applications by their very nature.
Perhaps HTML5/CSS3 will change things; but, standard DOM manipulation,
as I am accustomed to seeing it, cannot generate the kind of rendering
that is available from native applications.  Attempts to do so end up being
kludgy.

It also cannot handle the kinds of desktop integrations that are common
for native applications without opening up serious security trust issues.
(Can everybody say ActiveX fiasco?)

Finally, there are difficulties in handling keystrokes without conflicting
with the browser's native key bindings.  I seldom ever touch a mouse
and I am a huge fan of vi, mutt, slrn, screen, ratpoison, etc. where
the primary interface is totally accessable through the keyboard without
having to tab through many options.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Web App

[Tim Harig]

On 2010-12-23, Hidura hid...@gmail.com wrote:
 Ok, but you are comparing a web-based framework with a native-based
 framework that use the components of the system to make all the things
 that need, a web-based framewok use the resourses of the browser to

Right.  That is exactly what I am comparing.

 make it all, so the developer that use a framework on the web can't
 expect get the same results, in my case i beleive that a web-based

Which is exactly the problem with web apps that are highly interactive.  My
suggestion, is not to develope a web based IDE or use one.  It just isn't
something that the web was designed to do well.

 expect get the same results, in my case i beleive that a web-based
 framework adjust better to the needs if you'll make a web-app,

Most IDEs that are targeted at web developement have a built in web browser
or strong integration with one to run the web app as you are developing it.
I don't see any advantage or the necessity of actually running the IDE code
itself in the browser.

 otherwise use eclipse or netbeans.

I would; but then, I wouldn't purchase an operating system that is entirely
based on a web browser.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

[Wrapped to meet RFC1855 Netiquette Guidelines]
On 2010-12-20, spaceman-spiff ashish.mak...@gmail.com wrote:
 This is a rather long post, but i wanted to include all the details 
 everything i have tried so far myself, so please bear with me  read
 the entire boringly long post.
 
 I am trying to parse a ginormous ( ~ 1gb) xml file.
[SNIP]
 4. I then investigated some streaming libraries, but am confused - there
 is SAX[http://en.wikipedia.org/wiki/Simple_API_for_XML] , the iterparse
 interface[http://effbot.org/zone/element-iterparse.htm]

I have made extensive use of SAX and it will certainly work for low
memory parsing of XML.  I have never used iterparse; so, I cannot make
an informed comparison between them.

 Which one is the best for my situation ?

Your posed was long but it failed to tell us the most important piece
of information:  What does your data look like and what are you trying
to do with it?

SAX is a low level API that provides a callback interface allowing you to
processes various elements as they are encountered.  You can therefore
do anything you want to the information, as you encounter it, including
outputing and discarding small chunks as you processes it; ignoring
most of it and saving only what you want to memory data structures;
or saving all of it to a more random access database or on disk data
structure that you can load and process as required.

What you need to do will depend on what you are actually trying to
accomplish.  Without knowing that, I can only affirm that SAX will work
for your needs without providing any information about how you should
be using it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-20, spaceman-spiff ashish.mak...@gmail.com wrote:
 0. Goal :I am looking for a specific element..there are several 10s/100s
 occurrences of that element in the 1gb xml file.  The contents of the xml,
 is just a dump of config parameters from a packet switch( although imho,
 the contents of the xml dont matter)

Then you need:
1. To detect whenever you move inside of the type element you are
seeking and whenever you move out of it.  As long as these
elements cannot be nested inside of each other, this is an
easy binary task.  If they can be nested, then you will
need to maintain some kind of level count or recursively
decompose each level.

2. Once you have obtained a complete element (from its start tag to
its end tag) you will need to test whether you have the
single correct element that you are looking for.

Something like this (untested) will work if the target tag cannot be nested
in another target tag:

- import xml.sax
- class tagSearcher(xml.sax.ContentHandler):
- 
- def startDocument():
- self.inTarget = False
- 
- def startElement(name, attrs):
- if name == targetName:
- self.inTarget = True
- elif inTarget = True:
- # save element information
- 
- def endElement(name):
- if name == targetName:
- self.inTarget = False
- # test the saved information to see if you have the
- # one you want:
- #
- #if its the peice you are looking for, then
- #you can process the information
- #you have saved
- #
- #if not, disgard the accumulated
- # information and move on
- 
- def characters(content):
- if self.inTarget == True:
- # save the content
- 
- yourHandler = tagSearcher()
- yourParser = xml.sax.make_parser()
- yourParser.parse(inputXML, yourHandler)

Then you just walk through the document picking up and discarding each
target element type until you have the one that you are looking for.

 I need to detect them  then for each 1, i need to copy all the content
 b/w the element's start  end tags  create a smaller xml file.

Easy enough; but, with SAX you will have to recreate the tags from
the information that they contain because they will be skipped by the
character() events; so you will need to save the information from each tag
as you come across it.  This could probably be done more automatically
using saxutils.XMLGenerator; but, I haven't actually worked with it
before. xml.dom.pulldom also looks interesting
 
 1. Can you point me to some examples/samples of using SAX, especially ,
 ones dealing with really large XML files.

There is nothing special about large files with SAX.  Sax is very simple.
It walks through the document and calls the the functions that you
give it for each event as it reaches varius elements.  Your callback
functions (methods of a handler) do everthing with the information.
SAX does nothing more then call your functions.  There are events for
reaching a  starting tag, an end tag, and characters between tags;
as well as some for beginning and ending a document.

 2.This brings me to another q. which i forgot to ask in my OP(original
 post).  Is simply opening the file,  using reg ex to look for the element
 i need, a *good* approach ?  While researching my problem, some article
 seemed to advise against this, especially since its known apriori, that
 the file is an xml  since regex code gets complicated very quickly 
 is not very readable.

 But is that just a style/elegance issue,  for my particular problem
 (detecting a certain element,  then creating(writing) a smaller xml
 file corresponding to, each pair of start  end tags of said element),
 is the open file  regex approach, something you would recommend ?

It isn't an invalid approach if it works for your situatuation.  I have
used it before for very simple problems.  The thing is, XML is a context
free data format which makes it difficult to generate precise regular
expressions, especially where where tags of the same type can be nested.

It can be very error prone.  Its really easy to have a regex work for
your tests and fail, either by matching too much or failing to match,
because you didn't anticipate a given piece of data.  I wouldn't consider
it a robust solution.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyUNO [Was: Read / Write OpenOffice SpreadSheet ?]

[Tim Harig]

On 2010-12-17, Adam Tauno Williams awill...@whitemice.org wrote:
 I would strongly recommend against floundering about in OOo's very
 complex XML files - it is trivially easy to render a document unusable.

I do it all the time and have never had a problem.  I don't generate the
documents from scratch; I generate a template that contains everything
that don't need to dynamically generate.  Then I use one of two methods
to to update the content.xml:

1. In the simplest cases, I only need to change a single data field.  I
replace the literal data in the content.xml file with:

replace field=variable_name/

Then, using a DOM implementation, I can use getElementsByTagName()
to get all of the replace tags and send the variable name to a
distpach that generates the text used to replace the tag.

2. For collections of data (spreadsheet cells, table cells/rows, etc,
I leave one piece of sample data in place.  I then clone the DOM
element that I can use as a template and delete the origional.
Entering the data is then a simple matter of cloning the template
element, updating the information that it contains, and adding
it to the childNodes of the parent.  Since tags all come from
the file that OO.org/LibreOffice generated, the resulting markup
will be valid.

Once the content.xml file has been updated, I simply run jar as a
subprocess to update the content.xml file in the ods/odt file.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: If/then style question

[Tim Harig]

On 2010-12-16, John Gordon gor...@panix.com wrote:
 I like this style more, mostly because it eliminates a lot of indentation.

 However I recall one of my college CS courses stating that one entry,
 one exit was a good way to write code, and this style has lots of exits.

So, take the good intentation from one and the single exit from the other:

   def myMethod(self, arg1, arg2):

 if some_bad_condition:
   exitCode = bad1

 elif some_other_bad_condition:
   exitCode = bad2

 elif yet_another_bad_condition:
   exitCode = bad3

 else:
   exitCode = do_some_useful_stuff()

 # possible common cleanup code here

 return exitCode

Or, raise an exception on bad condtions rather then passing an error code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Read / Write OpenOffice SpreadSheet ?

[Tim Harig]

On 2010-12-17, Torsten Mohr tm...@s.netic.de wrote:
 i search for a possibility to access OpenOffoce SpreadSheets from Python 
 with a reasonably new version of Python.

 Can anybody point me to a package that can do this?

There is no package needed to read or write the new open document files.
The files are merely a jar archive containing XML files.  You can open
and update them using jar as a subprocess and manipulate the XML files
using your favorite XML libraries DOM/SAX/XPath/Etree/etc.

If that doesn't suit you, you can manipulate them using OO.org through its
UNO interface; but, I find that much more involved then simply accessing
the files directly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching user switching and getting current active user from root on linux

[Tim Harig]

On 2010-12-13, mpnordland mpnordl...@gmail.com wrote:
 I think I do understand multiuser systems, although I'm sorry I did
 not make my self clear. Yes, I understand that there can be multiple
 people logged in, and yes if you really wanted to, you could login as

Apparantly you do not.  There is nothing that prevents me from downloading
from the web when I am not even at the computer.  The point people are
making is that there is no such thing as a single active user.  Any user
that has a running process is active.  Somebody is not going to appreciate
getting fired because somebody else scheduled a porn download at a time
when they happened to be flagged as the current user.

 yourself as many times as you want. This is not a server based
 program, it is intended for use in a normal environment. The people

There is nothing definitive about a so-called normal enviroment and
since you didn't really specify anything, we had go make assumtpions
for ourselves.

Note also that a server is a piece of software, it need not be running
on server class hardware.  There are probably several local servers
running on your system right now.

As is commonly the case, the reason that you are having trouble with this
problem is that you are trying to solve it wrong in the first place.
You told us how you thought you should solve it which meant that we
couldn't help you because we didn't know what problem you were trying to
solve in the first place.  For future reference, always give a 10,000ft
explanation of the problem that you are actually trying to solve rather
then just the microcosm of the problem that you feel is relevant.

 or someone else. The problem I have is I want monitor what sites the
 user visits, I need to know what sites were accessed by which user.
 So, a way to find which user is requesting what url is what I want.

As Carl has already pointed out, an authenticated proxy is the proper
way to get the individualized logging that you are looking for.
It is capable of logging each connection (including software from
which you might not have anticipated) with its own identified user of
origin. Even when several users are accessing the network at once, the
network connections can be tied directly to the person who initated
the connection.  Users cannot use somebody elses connection without
their authentication credentials.

With a little bit of scripting, the browser can be configured to
automatically use the user's credentials whenever the user account is
created so that the user never needs to enter their credentials manually
after logging in to their account.  All connections that attempt to
bypass the proxy should be firewalled.

 Have I passed the Turing test?

You still don't seem to understand how to use threads.  Heres a clue:
find a decent usenet client that actually displays them by default.  Or,
if you can't manage to get rid of that junky interface that is Google
Groups, at least select the option to view as a tree.  Then you might
be able to actually post your replies in the proper thread rather then
just appending on the the person who last posted.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reading by positions plain text files

[Tim Harig]

On 2010-12-12, javivd javiervan...@gmail.com wrote:
 On Dec 1, 7:15 am, Tim Harig user...@ilthio.net wrote:
 On 2010-12-01, javivd javiervan...@gmail.com wrote:
  On Nov 30, 11:43 pm, Tim Harig user...@ilthio.net wrote:
  encodings and how you mark line endings.  Frankly, the use of the
  world columns in the header suggests that the data *is* separated by
  line endings rather then absolute position and the position refers to
  the line number. In which case, you can use splitlines() to break up
  the data and then address the proper line by index.  Nevertheless,

^^
Note that I specifically questioned the use of absolute file position vs.
postion within a column.  These are two different things.  You use
different methods to extract each.

  I work in a survey research firm. the data im talking about has a lot
  of 0-1 variables, meaning yes or no of a lot of questions. so only one
  position of a character is needed (not byte), explaining the 123-123
  kind of positions of a lot of variables.

 Thenfile.seek() is what you are looking for; but, you need to be aware of
 line endings and encodings as indicated.  Make sure that you open thefile
 using whatever encoding was used when it was generated or you could have
 problems with multibyte characters affecting the offsets.

 f = open(r'c:c:\somefile.txt', 'w')

I suspect you don't need to use the c: twice.

 f.write('0123456789\n0123456789\n0123456789')

Note that the file you a writing contains three lines.  Is the data that
you are looking for located at an absolute position in the file or on a
position within a individual line?  If the latter, not that line endings
may be composed of more then a single character.

 f.write('0123456789\n0123456789\n0123456789')
  ^ postion 3 using fseek()

 for line in f:

Perhaps you meant:
for character in f.read():
or
for line in f.read().splitlines()

 f.seek(3,0)

This will always take you back to the exact fourth position in the file
(indicated above).

 I used .seek() in this manner, but is not working.

It is working the way it is supposed to.

If you want the absolution position 3 in a file then:

f = open('somefile.txt', 'r')
f.seek(3)
variable = f.read(1)

If you want the absolute position in a column:
f = open('somefile.txt', 'r').read().splitlines()
for column in f:
variable = column[3]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reading by positions plain text files

[Tim Harig]

On 2010-12-12, Tim Harig user...@ilthio.net wrote:
 I used .seek() in this manner, but is not working.

 It is working the way it is supposed to.
 If you want the absolute position in a column:

   f = open('somefile.txt', 'r').read().splitlines()
   for column in f:
   variable = column[3]

or:
f = open('somefile.txt', 'r')
for column in f.readlines():
variable = column[3]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching user switching and getting current active user from root on linux

[Tim Harig]

Mr. Chase, I really wouldn't even bother wasting my time on this one.
He asked an incomplete question to start with; so, the replies that
he received were insufficient to solve his problem.  He still has not
provided enough information to know how to answer his question propery.
He doesn't understand a sacastic reply when he hears one, he doesn't
understand the concept of a multi-user operating system, and he doesn't
understand the concept of how usenet threads work.  Until he demonstrates
some intelligence, I would say that he has flunked the Turing test.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-05, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 The fact that I bothered to create classes for the dice and roles, rather
 then simply iterating over a list of numbers,  should tell you that I
 produced was of a far more flexible nature; including the support for
 roles with dice having different numbers of sides.

 from itertools import product
 def n_sided_die(n): return xrange(1, n+1)

 # make one 3-sided die, one 4-sided die, and one 5-sided die
 dice = (n_sided_die(3), n_sided_die(4), n_sided_die(5))
 for roll in product(*dice):
 print roll

Notice that you had to change the structure of your program to accomodate
the new possiblity; and, if I throw further requirements at you, you
will have to change them again.  I didn't want that.  What the dice
returned may or may not have returned an easy to compute sequence.
In fact, for all of the rest of the logic cared, the dice could have
computed their value from a previous role of another dice.  All of the
logic of about what the dice may have returned when asked for their
value and how they derived, was encapsilated in the dice themselves.
It did not need to be known anywhere else in the program logic.

The DSL effectively provided a way do define how the dice decided how
to increment themselves, how to choose the value that they returned for
their face, and how to know when they could no longer be incremented.
The DSL parser generated the dice set from the description given.
Creating new dice objects was much easier then attempting to change the
logic of how they were rolled.

 I merely posted a simplied description of the dice-role objects
 because I thought that it demonstrated how exceptions can provide
 eligance of control for situations that don't involve what would
 traditionally be defined as an error.

 Exceptions (though sometimes necessary) are messy and I'm having a hard
 time trying to envision that code being cleaner with them than without
 them.  If you post the actual code maybe that will help me understand.

Let me get this straight, the same person that was trying to tell me
setjmp/longjmp weren't messy thinks exceptions are messy?  I have used
both.  I much prefer the exceptions.  I not have to code here to post.

The cleanliness of using the exception and calling the dice increments
recursively, was that there was no need to figure out which dice needed
to be incremented whenever the first die needed to be reset.  When a dice
needed to be reset, it would raise an exception.  This exception would
rise through the recursion stack, and thus through the dice, resetting
each along the way until it found the one which needed to be incremented
or raised past the top call indicating that all of the combinations has
been exhausted.  There, once the reset condition for the previous dice
had been effectively handled, it would be supprested. 

Had this been done using in band data:

1. The roll object would have needed logic to determine when
a reset condition needed to take place, effectively
leaking some of the logic from the dice object to the
role object.

2. The roll object would have needed logic to determine how to
follow the dice which needed to be reset until it found
which one needed incrementing.  Once again, this logic
was better left to the dice walking the resets was
automatically handled by the progression of the exception.

Even if it wasn't an error, the resets were effectively a exceptional
condition from the normal flow of the role object (the primary flow simply
being to increment the first die).  By using exceptions, I effectively
isolated each into its own separate independent flow; and, because they
where called separatly, neither needed to have control conditions to detect
which was needed.  They simply allowed the dice object to decide.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-05, Tim Harig user...@ilthio.net wrote:
 On 2010-12-05, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 The fact that I bothered to create classes for the dice and roles, rather
 then simply iterating over a list of numbers,  should tell you that I
 produced was of a far more flexible nature; including the support for
 roles with dice having different numbers of sides.

 from itertools import product
 def n_sided_die(n): return xrange(1, n+1)

 # make one 3-sided die, one 4-sided die, and one 5-sided die
 dice = (n_sided_die(3), n_sided_die(4), n_sided_die(5))
 for roll in product(*dice):
 print roll

 Notice that you had to change the structure of your program to accomodate
 the new possiblity; and, if I throw further requirements at you, you
 will have to change them again.  I didn't want that.  What the dice
 returned may or may not have returned an easy to compute sequence.
 In fact, for all of the rest of the logic cared, the dice could have
 computed their value from a previous role of another dice.  All of the
 logic of about what the dice may have returned when asked for their
 value and how they derived, was encapsilated in the dice themselves.
 It did not need to be known anywhere else in the program logic.

 The DSL effectively provided a way do define how the dice decided how
 to increment themselves, how to choose the value that they returned for
 their face, and how to know when they could no longer be incremented.
 The DSL parser generated the dice set from the description given.
 Creating new dice objects was much easier then attempting to change the
 logic of how they were rolled.

 I merely posted a simplied description of the dice-role objects
 because I thought that it demonstrated how exceptions can provide
 eligance of control for situations that don't involve what would
 traditionally be defined as an error.

 Exceptions (though sometimes necessary) are messy and I'm having a hard
 time trying to envision that code being cleaner with them than without
 them.  If you post the actual code maybe that will help me understand.

 Let me get this straight, the same person that was trying to tell me
 setjmp/longjmp weren't messy thinks exceptions are messy?  I have used
 both.  I much prefer the exceptions.  I not have to code here to post.

 The cleanliness of using the exception and calling the dice increments
 recursively, was that there was no need to figure out which dice needed
 to be incremented whenever the first die needed to be reset.  When a dice
 needed to be reset, it would raise an exception.  This exception would
 rise through the recursion stack, and thus through the dice, resetting
 each along the way until it found the one which needed to be incremented
 or raised past the top call indicating that all of the combinations has
 been exhausted.  There, once the reset condition for the previous dice
 had been effectively handled, it would be supprested. 

 Had this been done using in band data:

   1. The roll object would have needed logic to determine when
   a reset condition needed to take place, effectively
   leaking some of the logic from the dice object to the
   role object.

   2. The roll object would have needed logic to determine how to
   follow the dice which needed to be reset until it found
   which one needed incrementing.  Once again, this logic
   was better left to the dice walking the resets was
   automatically handled by the progression of the exception.

 Even if it wasn't an error, the resets were effectively a exceptional
 condition from the normal flow of the role object (the primary flow simply
 being to increment the first die).  By using exceptions, I effectively
 isolated each into its own separate independent flow; and, because they
 where called separatly, neither needed to have control conditions to detect
 which was needed.  They simply allowed the dice object to decide.

Okay, it occures to me that you don't really need to see much to know what was
going on, here is the basic idea of how the role function of the object would
have looked like:

def role(self, dice):

try:
self.role(dice.previous())
except diceReset:
dice.increment()
except endOfDice:
raise diceReset
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP8 compliance and exception messages ?

[Tim Harig]

On 2010-12-06, Andreas Waldenburger use...@geekmail.invalid wrote:
 On Sun, 5 Dec 2010 19:52:54 -0800 Chris Rebert c...@rebertia.com wrote:

 On Sun, Dec 5, 2010 at 7:40 PM, shearichard shearich...@gmail.com
 wrote:
  Hi - PEP8 says lines should not exceed 79 characters in length
  ( http://www.python.org/dev/peps/pep-0008/ ).
 
  So if you've got some code that looks like this :
 
  raise fooMod.fooException(Some message which is quite long)
 
  ... and assuming a certain amount of indenting you're going to break
  that guideline.
 
  [etc.]
 
 Use implicit string literal concatenation:
 
 [...]
 
 But isn't explicit string literal concatenation better than implicit
 string literal concatenation?

So add the +, it really doesn't change it much.

 Alternatively, you could disregard PEP 8 on this point on the grounds
 that the 79/80 characters per line limit is outdated.
 
 Maybe, but it's not outmoded.

I would say that it is not even outdated.  Just because you happen to enjoy
longer lines doesn't mean that everybody does.  Not all progammers have
20/10 vision and even those who have hardware to handled it don't
necessarily like having a single piece of code take up their entire display
just to be readable.  Many of us like the extra ability that wider screen
technology gives us to actually be able to view more of the file at once
by splitting the display into a couple of columns.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-05, Harishankar v.harishan...@gmail.com wrote:
 Or consider this code:
 
 if y != 0:
 result = x/y
 else:
 handle_division_by_zero()
 
 
 This is also unsafe unless you know the type of y. Suppose y is an
 interval quantity that straddles zero, then division by y may fail even
 though y != 0.

 Of course in each of these cases the in-built exceptions are used to 
 verify the result of certain system level or lower level operations. My 
 object was not to deprecate the system-level or other low level 
 exceptions thrown by Python, but to consider whether such a mechanism 
 would be a preferable method of handling your own programs error-
 conditions. 

Whether you happen to like the exception mechanism and syntax or not, it is
the idiomatic way of handling errors in Python.  Using two different
conventions in your code will lead to confusion.  I come from a long C
background as well.  I have come to appreciate the power the Python's
exception handling provides.  It does everything that you need to do with
passing values in C and more.

 The issue to be considered by every programmer is to define what can be 
 defined as the exceptional condition and what is a condition that merits 
 merely different treatment without causing disruption of the normal flow 
 of the program.

That is an issue much harder to define.  Anything it is an obvious
error *should* throw an exception.  Invalid input is an error.
Unusable hardware states are errors.  Any invalid request to an object,
is an error.  Essentially anything that deviates from a normal flow of
a program, to handle an exceptional condition, is an error

Where it becomes less obvious is when you start using exceptions as
part normal control flow.  An example is a try it and see methodology.
You might for instance have a group of file objects which might or might
not support a particular method attribute.  You might have a preference for
using the attribute; but, have a fallback plan if it does not.  One way to
handle this is to try to use the attribute and catch the exception raised
if it is not present to execute your backup method.  I have found this
*essential* in some marsaling enviroments where you might not have access to
the meta-data of the object that you are working with.

Another, questionable but useful use, is to ignore the complex accounting
of your position inside of a complex data structure.  You can continue
moving through the structure until an exception is raised indicating
that you have reached a boundary of the structure.

Whether you accept uses of exceptions like these is more of a personal
quesion.  Like many good tools, they can be useful in ways that they were
never really designed to be and I would hate to proclude some of these
really useful features.

This can, of course, be easily abused.  I was once writing code, involving
complex object marshaling like I described above, with a partner who
wasn't totally familiar with Python.  We came to a situation where it
was impossible to know ahead of time what kind of object (one of two
possiblities) we would receive from another marshalled object and had no
meta-data to be able to figure out before attempting to access the object.
I used a try/except clause to resolve the problem.  The next day, I
found several poorly conceived try/except blocks in the codebase that
my partner had used for control structures using dictionaries because
he didn't know of dict.has_key().  I was not so pleased.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-05, Tim Harig user...@ilthio.net wrote:
 Another, questionable but useful use, is to ignore the complex accounting
 of your position inside of a complex data structure.  You can continue
 moving through the structure until an exception is raised indicating
 that you have reached a boundary of the structure.

Here is another example in this vein.  A friend was trying to derive a
mathematical formula for determining the possibly distribution of results
from rolling arbitrariy numbers of m n-sided dice and needed several sets
of data in different directions from which to draw conclusions.

I created objects for dice and roles which contained and manipulated
multiple dice.  To generate a listing of all (non-uniq) possible roles,
I would call the first dices increment method read and read the dice
faces into a log until the first dice threw an exception that it could
not be further incremented.  Then I would call reset() on the first dice
and increment the second and so on much like the odometer of a car.

By using exceptions rather then checking the return value of increment,
the state information of the dice was completely isolated to the dice
and did not polute into the role structure; the logic for incrementing
the dice, logging the role state, and rolling over the dice where
all completely seperated and independent of any state; and therefore
reseting multiple previous dice as the higher values on the odometer were
incremented functioned automatically as each dice threw its own exception
recursively rather then requiring logic to handle these multiple rollovers.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-05, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 A friend was trying to derive a mathematical formula for determining
 the possibly distribution of results from rolling arbitrariy numbers
 of m n-sided dice

 http://en.wikipedia.org/wiki/Multinomial_distribution

I sure he rediscovered much of that.  Working that out for himeself was
probably far more educational then simply reading an article on the
solution.

 To generate a listing of all (non-uniq) possible roles, I would call
 the first dices increment method read and read the dice faces into a
 log until the first dice threw an exception that it could not be
 further incremented.  Then I would call reset() on the first dice and
 increment the second and so on much like the odometer of a car.

 from itertools import product
 m, n = 5, 2
 for roll in product(*(xrange(1,m+1) for i in xrange(n))):
print roll

The fact that I bothered to create classes for the dice and roles, rather
then simply iterating over a list of numbers,  should tell you that I
produced was of a far more flexible nature; including the support for
roles with dice having different numbers of sides.  I basically created
a DSL that he could use to generate and automatically calculate the
properties of series of roles defined by one or more varying property.

I merely posted a simplied description of the dice-role objects because I
thought that it demonstrated how exceptions can provide eligance of control
for situations that don't involve what would traditionally be defined as an
error.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-03, Paul Rubin no.em...@nospam.invalid wrote:
 Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
 There are better ways to handle errors than Python's exception system.
 I'm curious -- what ways would they be?
 I'm aware of three general exception handling techniques: ...
 What else is there?

 The Erlang approach is to chop the application into a lot of very
 lightweight processes, and let any process that encounters an error
 simply crash.  A monitoring process notices the crashed process and
 restarts it.  There is a supervision tree of uber-monitor processes
 that restart crashed monitor proceses.  I haven't programmed in that
 style myself and I'm not persuaded that it's better than what Python
 does, but I think it's different from the stuff on your list, which is

Erlang also offers an exception syntax almost identical to Python's for use
within a single process.

What makes Erlang's supervisor mode of error handling superior is that it
works for more then just the current vm.  If a process throws an exception,
the supervisor catches and handles it.  If a vm dies, a supervisor from
another vm takes over.  If an entire computer dies, a supervisor on another
computer takes over.  OTP provides some extremely advanced support for
supervisory structures.

 an answer to your what else is there.  I do know that they write some
 complex, very high reliability systems (phone switches) in Erlang.

Erlang isn't what I would call a very general purpose programming language
like Python; but, if you want to build highly scalable and/or highly
available systemes, there really isn't anything else that comes close
to it.  I am not really a huge fan of the purely functional nature of
the language; but, light weight processes using the actor model is the
way to go for concurrent processing.

The BEAM virtual machine is also a powerful system with its ability to
patch systems on the fly.  It has start to become the target for other
languages.  I know of two that are in current developement.  I wouldn't
mind seeing a version of Python that could leverage the power of the
BEAM vm.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-04, alex23 wuwe...@gmail.com wrote:
 On Dec 3, 2:12 am, Tim Harig user...@ilthio.net wrote:
 Actually, I thought that debate was resolved years ago.  I cannot think of
 a single recently developed programming language that does not provide
 exception handling mechanisms because they have been proven more reliable.

 Google's Go lacks exceptions and I believe that was a deliberate
 design choice.

1. The debate that I was referring to was between simple function checking
vs. everything else.  I didn't mean to automatically proclude any
newer methodologies of which I might not even be aware.

2.  I would consider the defer/panic/recovery mechanism functionally similar 
to exceptions in most ways.  It allows the error handling
code to be placed at a higher level and panics tranverse the stack
until they are handled by a recovery.  This is basically equivilent
to how exceptions work using different names.  The change is basically 
the defer
function which solves the problem of any cleanup work that the
function needs to do before the panic is raised.  I like it, its
nice.  It formalizes the pattern of cleaning up within an exception
block and re-raising the exception.

I do have to wonder what patterns will emerge in the object given
to panic().  Since it takes anything, and since Go doesn't have an
object hierarchy, much less an exception hierarchy, the panic value
raised may or may not contain the kind of detailed information that
can be obtained about the error that we are able to get from the
Exception objects in Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-03, Harishankar v.harishan...@gmail.com wrote:
 On Fri, 03 Dec 2010 14:31:43 +, Mark Wooding wrote:
 In general, recovering from an exceptional condition requires three
 activities:
 
   * doing something about the condition so that the program can continue
 running;
 
   * identifying some way of rejoining the program's main (unexceptional)
 flow of control; and
 
   * actually performing that transfer, ensuring that any necessary
 invariants are restored.

 my fault. Seems that exception handling requires a lot of forethought 
 since the control of program execution breaks at the point of exception 
 with no obvious way to rejoin it seamlessly whereas with an error, a 
 simple if condition could handle the error state and resume execution 
 from that point forward. This is the main reason why I think I used 
 simple error codes to handle certain recoverable conditions and avoided 
 exceptions.   

If you are returning an error code to the above function, then there is
nothing that you cannot do with with the exception.  Basically, you resolve
the issue in your except block just as you would in the block of your if
statement after returning the error code.  If you try and fail to handle
the exception or just needed to do some cleanup before allowing the
exception to continue, then you just re-raise the exception. 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Harishankar v.harishan...@gmail.com wrote:
 There are some reasons why I hate exceptions but that is a different 
 topic. However, in short I can say that personally:

 1. I hate try blocks which add complexity to the code when none is 
 needed. Try blocks make code much more unreadable in my view and I use it 
 only for the built-in exceptions when absolutely needed.

Actually, exceptions remove a ton of complexity and almost universally
remove a ton of redundant error checking code.  Second, they aleviate a
ton of design complexity about designing a clean and unified method of
error handling throughout the program.  Using exceptions, the only real
questions are where to handle various errors.  It is a godsend for having
to clean up intermediary results when an error occurs as part of a complex
set of dependant sequential operations.

 2. I prefer the less irksome True or False to do error checking. 
 Exceptions seem too heavyweight for simple problems.

Error handling in C huge source of bugs.  First, error handling code is
scattered throughout the codebase and second it is hard to test the error
handling code for a number of failures.  Being able to raise exceptions
within your test code makes it much easier to write tests capable of
detecting error handling bugs.

 3. Philosophically I think exception handling is the wrong approach to 
 error management. I have never grown up programming with exceptions in C 
 and I couldn't pick up the habit with python either. Did I mention that I 
 detest try blocks? try blocks seem ugly and destroy code clarity at least 
 in my view. And enclosing single statements under separate try blocks 
 seem to add a lot of clutter. 

Perhaps you should take a look at how Erlang appoaches exception handling.
Being message passing and concurrency oriented, Erlang encourages ignoring
error conditions within worker processes.  Errors instead cause the worker
processes to be killed and a supervisory process is notified, by message,
so that it can handle the error and respawn the worker process.  Since it
doesn't use try/exept blocks, maybe that will be more to your liking.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Harishankar v.harishan...@gmail.com wrote:
 I am also wary of using larger catch-all try blocks or try blocks with 
 multiple exception exits (which seem to make tracking subtle bugs 
 harder). I prefer the philosophy of dealing with errors immediately as 

If you are using exceptions to try to catch bug then you are using them
improperly.  Exceptions (with the exception (no pun intended) of
AssertionError) are designed to catch error conditions, not bugs.

 harder). I prefer the philosophy of dealing with errors immediately as 
 they arise, rather than delegate them to exception mechanism. Of course, 
 I could wrap single statements in try blocks, but that makes the code 
 even messier without any significant benefits.

Actually, finer grained error handling commonly covers up bugs.  If you
want to find bugs, you want to make the program prone to crashing if
a bug is present.  It is all too easy to accidently mistake the return
value of a function as error condition and handle it rather the letting
the program crash.  By separating the results from the transmission of
error conditions (in effect taking error conditions out of band) then
you make it much harder to make such a mistake because you have to
explicity indicate which error conditions your code is capable of
generating.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Harishankar v.harishan...@gmail.com wrote:
 I understand that the error vs exception debate is quite a big one in the 
 programming community as a whole and I don't consider myself very 

Actually, I thought that debate was resolved years ago.  I cannot think of
a single recently developed programming language that does not provide
exception handling mechanisms because they have been proven more reliable.

 Python. I do understand both sides of the issue. Exceptions seem to be 
 generally more reliable but I feel they add a lot of complexity 
 particular when a lot of code is placed in a try block. 

Lines of code is one measure of complexity.  Each line of code has a small
chance of containing a bug.  The more lines of code that you have, then the
more likely that one of them contains a bug.  Exceptions, by placing error
handling code in fewer places, requires much fewer lines of code then
requiring error handling code after each call that might produce an error
condition.

The operations of propogating an error up to the higher level logic of
the program is another measure.  In languages without exception handling,
careful planning is needed to pass error conditions up through the call
stack until they reach a high enough level in the logic that decisions
can be made about how to handle them.  Even further planning must be
taken so that when the error condition reaches level where it needs to
be handled, that enough information about the error is present to know
exactly what went wrong so that it can figure out what to do about it.
This usually involves using globals like errorno to pass out of band
information about the error.  Sometimes you even need to know about how
the error affect intermediate levels, did the intermediate code attempt
to handle the condtion and fail?  The Openssl error handling system,
that creates an error logging chain is an example of just how complex
this can become.  You gain all of this functionality automatically
through exception mechanisms; without all of the complexity.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Harishankar v.harishan...@gmail.com wrote:
 Actually, finer grained error handling commonly covers up bugs.  If you
 want to find bugs, you want to make the program prone to crashing if a
 bug is present.  It is all too easy to accidently mistake the return
 value of a function as error condition and handle it rather the letting
 the program crash.  By separating the results from the transmission of
 error conditions (in effect taking error conditions out of band) then
 you make it much harder to make such a mistake because you have to
 explicity indicate which error conditions your code is capable of
 generating.

 Doesn't the same finer grained exception mechanism make it prone to the 
 same problems? 

Exception handling is a move away from fine grained handling.  If in
IOError is raised by a parser, I don't need to know at exactly which
line the error occured.  All I need to know is that the file is somehow
unreadable; and that is why the parser failed.  Therefore, none of
the parser code should be handling IOError because it is being handled
higher up in the stack.  Since I expected that the parser might have
problemes reading files, for problems that have nothing to do with my
code, I explicity catch that error if it occurs.  I am not expecting an
IndexError from the parser and I don't bother to catch it.  If the parser
code does raise an IndexError, then my program crashes and I know that I
have a bug in the parsing code.  The call trace will tell me where that
error occurs.  I can watch that section of code in debugger to find out
exactly what went wrong.

 Actually return values of functions which I write myself can be as 
 specific and to the point. I could make it as fuzzy or as precise as I 
 like. This is no doubt, something that I could emulate with an exception 
 as well, but only make it slightly more complex with no obvious benefit.

You seem to be making it complex because you are still trying to be too
fine grained in handling each exception where it occurs as opposed to
handing where the logic makes sense that it should be handled and because
you are trying to code too defensively against your own code.  Exception
handling does require a different focus from handling errors from return
values alone.

 As I said before, the way exceptions are caught seem to me to be the most 
 confusing bit. Non-atomic operations always worry me. What if my function 
 which is wrapped inside a try block has two different statements that 
 raised the same exception but for different reasons? With error handling 
 I could probably handle it right below the statement which was called and 
 thus reduce the problem???

If you are having that issue, then you are likely placing the try blocks
at too low of a level in your code.  In general you will find that
most systems have a gateway function as an entry point to the system.
If there is not one already, then create such a function in you code.
The parse function in my code above would be an example of such a
gateway function.  Beneath that function, you don't need to know exactly
where the error occured, you just need to know the nature of the error and
have general error handling procedures for each kind of error that you
expect might occur.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Paul Rubin no.em...@nospam.invalid wrote:
 MRAB pyt...@mrabarnett.plus.com writes:
 When writing the C code for the new regex module I thought that it
 would've been easier if I could've used exceptions to propagate errors
 and unwind the stack, instead of having to return an error code which
 had to be checked by the caller, and then have the caller explicitly
 return an error code to /its/ caller.

 That's called longjmp.

The problem is that you might have partially allocated data structures that
you need to free before you can go anywhere.  Jumping up several levels,
without letting the intermediate levels do their cleanup could easily lead
to a memory leak or worse.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 That's called longjmp.

 The problem is that you might have partially allocated data structures
 that you need to free before you can go anywhere.

 Alloca can help with that since the stack stuff gets released by the
 longjmp.  Alternatively you can have an auxiliary stack of cleanup

alloca() only helps if you actually *want* the data stored on the stack.
There are many reasons one might prefer or need that the data in the heap.

 longjmp.  Alternatively you can have an auxiliary stack of cleanup
 records that the longjmp handler walks through.  Of course if you do

Only if you already have pointers to *all* of the data structures at
the point where you put your setjmp().  This approach is error prone.

 records that the longjmp handler walks through.  Of course if you do
 that, you're halfway towards reinventing exceptions.

Exactly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 longjmp.  Alternatively you can have an auxiliary stack of cleanup
 records that the longjmp handler walks through.  Of course if you do

 Only if you already have pointers to *all* of the data structures at
 the point where you put your setjmp().

 The setjmp point only has to know where the aux stack is and its depth
 when the longjmp happens.  The cleanup records contain any necessary
 pointers to data structures that need freeing.  That is basically how
 try/finally would do it too.  This is pretty standard stuff.

I am not talking about what setjmp() has to do, I am talking about what
*you* have to do after setjmp() returns.  If you have allocated memory in
intermediate functions and you don't have a reference to them outside of
the functions that longjmp() bypasses from returning properly (and thus
either not clearning data structures or returning a reference to those data
structures as it normally would) then you have potential memory leaks,
dangling pointers, etc.

I am not saying that this cannot be done.  What I am saying is that it
is inherently error prone.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, MRAB pyt...@mrabarnett.plus.com wrote:
 On 02/12/2010 19:15, Tim Harig wrote:
 On 2010-12-02, Paul Rubinno.em...@nospam.invalid  wrote:
 Tim Hariguser...@ilthio.net  writes:
 longjmp.  Alternatively you can have an auxiliary stack of cleanup
 records that the longjmp handler walks through.  Of course if you do

 Only if you already have pointers to *all* of the data structures at
 the point where you put your setjmp().

 The setjmp point only has to know where the aux stack is and its depth
 when the longjmp happens.  The cleanup records contain any necessary
 pointers to data structures that need freeing.  That is basically how
 try/finally would do it too.  This is pretty standard stuff.

 I am not talking about what setjmp() has to do, I am talking about what
 *you* have to do after setjmp() returns.  If you have allocated memory in
 intermediate functions and you don't have a reference to them outside of
 the functions that longjmp() bypasses from returning properly (and thus
 either not clearning data structures or returning a reference to those data
 structures as it normally would) then you have potential memory leaks,
 dangling pointers, etc.

 I am not saying that this cannot be done.  What I am saying is that it
 is inherently error prone.

 Automatic garbage collection is nice to have when using exceptions
 precisely because it's automatic, so unwinding the stack is much safer.

Indeed.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-02, Paul Rubin no.em...@nospam.invalid wrote:
 Tim Harig user...@ilthio.net writes:
 I am not talking about what setjmp() has to do, I am talking about what
 *you* have to do after setjmp() returns.  If you have allocated memory in
 intermediate functions and you don't have a reference to them outside of
 the functions that longjmp() bypasses from returning properly (and thus
 either not clearning data structures or returning a reference to those data
 structures as it normally would) then you have potential memory leaks,
 dangling pointers, etc.

 Sure, that's what the aux stack is for--you put any such references into
 it, for the setjmp handler to find later.  You do that BEFORE setjmp
 returns, of course.

If you miss something, you are in trouble.

There is a concept of variable life that is measured by how many lines
separate the use of variable from its first use to its last.  By using
setjmp/longjmp, you effectively extend the life of these variables,
potentially through several files, to at least as long as the jump.  If
there are several function calls in depth, there may be quite a lot of
space that you have to check to make sure that you have not missed
anything.

 I am not saying that this cannot be done.  What I am saying is that it
 is inherently error prone.

 I suppose so, but so is everything else in C.  On the overall scale of
 C-related hazards, this particular one isn't so bad if you code in a
 consistent style and are disciplined about recording the cleanups.

 You could also use something like an obstack, which is a stack allocated
 on the heap, so it persists after the control stack returns, but you can
 release the whole thing in one operation.

By working the error back up through the call stack, you can keep track of
the variables and allocations in each function isolated to that function.
The smaller each function is, the easier and less error prone it will be
to theck it is to check.  That makes it much easier to make sure that
you have not missed anything.  Essentially, you can validate that each
function correctly handles is allocations rather then having to validate
the setjmp/longjmp structure as a whole.  To use Joe Armstrong's phrase,
it makes the impossible merely difficult.

Back to the topic, by using Python with its exceptions and garbage
collection, all of this is a moot point.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: three column dataset - additions and deletions

[Tim Harig]

On 2010-12-02, draeath draeath.spamt...@gmail.com wrote:
 The idea is that this script will run periodically, pulling the table, 
 and comparing the data gathered at that run to that stored by the 
 previous, acting on changes made, and storing the current data back (to 
 be referenced against in the next invocation)

So, basically, you want to store a local copy of the data and sync it to
the original.

 I figure it will be easy enough to determine changed hashes for a given 
 key. What I'm unclear on is what the best type of structure to keep this 
 data in, given that I need to modify the data after it comes in 
 (replacing that long string with, say, an MD5 from hashlib) and both need 
 to act on new rows (rows that don't exist in the 'old' data) and 
 deleted rows (rows that only exist in the 'old' data).

You need to differentiate between the in memory data model and the storage
model.  Since this data comes from a database in the first place, I would
dump it to an sqlite3 database from the beginning.  You can use this to
store, modify, and change the values as you receive them from the database.

If you are looking for in-memory structures, then you haven't really
provided us with enough information on the significance and organization of
the data.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: three column dataset - additions and deletions

[Tim Harig]

On 2010-12-03, draeath draeath.spamt...@gmail.com wrote:
 On Thu, 02 Dec 2010 22:55:53 +, Tim Harig wrote:

 Thanks for taking the time to check in on this, Tim!

 So, basically, you want to store a local copy of the data and sync it to
 the original.
 In a way. I only need to store one copy of the data, and make note of 
 changes between it and the current data.

Perhaps I am missing a sublty that makes those statements different.

 You need to differentiate between the in memory data model and the
 storage model.  Since this data comes from a database in the first
 place, I would dump it to an sqlite3 database from the beginning.  You
 can use this to store, modify, and change the values as you receive them
 from the database.
 I thought of doing that, but given that I only need to store a single 
 instance of the data, a simple pickle will do the job nicely (am I 
 correct in reading that it can save/load any python object?)

So, you start by dumping the data from the remote server into an sqlite3
database table.  What you end up with is a record=record copy of the
original query (plus any other meta data that you want to add).  Then,
when the data changes, you apply those same changes to your local table
(or just regenerate it since you seem to be pulling all of the information
from the server anyway.  The result is a *single instance of the data*.

Why you would want to replace this with a pickle of a nested set up tuples
or a homebrew on disk data structure is beyond me.  Using sqlite3 is almost
certainly faster and more functional then anything you are going to create
without some serious work.

 If you are looking for in-memory structures, then you haven't really
 provided us with enough information on the significance and organization
 of the data.
 The data columns:
 Long Int, String (under 30 chars), String (over 100 chars)
 The rows can scale up to hundreds, perhaps thousands.

Then those are the columns that you create for your local table.

 The integer is the database key, the shorter string is a user name, and 
 the longer string is an access control definition. The whole idea of this 
 script is to check, daily, for any added or removed users - or any 
 altered access control definition.

The question is how are you going to use this information once you have
mirroed it locally.  Most likely, from you description, you just need to
access it as a local read only data store.  Now compare the differences in
how you would acces the data:

pickle method:
1. You have to load the entire pickle into memory.
2. Unless you add some kind of tree or indexing mechanism, you will have to
walk through an average of 1/2 of the records to find the matching
id.  If you do use an advanced mechanism you have to create the
code that inserts and locates the data.

sqlite3 method:
1. You open the file using the sqlite3 connector which does not have to
read all of the data into memory.
2. You use a select query to get just the record for the id that you are
looking for.  sqlite3 has already provided you with optimized
lookup and indexing capability, as well as modification
operations, etc (most likely written in C).

As an added bonus, you don't have to worry about locking issues, to keep
the clients from accesses the datastore and receiving an inconsistant
copy, while you are making your periodic updates to the database

Summary: the pickle method is reinventing the wheel.  You can do what has
already been done for you with the sqlite3 module (and library)
that has already been written for you; but, getting anything
near the same functionality is going to require considerable effort
on your part; and you are probably going to have to write C to get
the equivilant performance.

Which seems like a better option to you?

 I realize this could likely all be done from inside the database itself - 
 but altering the DB itself is not an option (as the product vendor is 
 very touchy about that, and altering it can null our support agreement)

Altering the remote database is not an option; but, I am talking about
modifying only your local copy.  If you can rewrite your pickle file,
then you can modify the sqlite3 file.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: three column dataset - additions and deletions

[Tim Harig]

On 2010-12-03, MRAB pyt...@mrabarnett.plus.com wrote:
 On 03/12/2010 01:42, Tim Harig wrote:
 On 2010-12-03, draeathdraeath.spamt...@gmail.com  wrote:
 On Thu, 02 Dec 2010 22:55:53 +, Tim Harig wrote:

 Thanks for taking the time to check in on this, Tim!

 So, basically, you want to store a local copy of the data and sync it to
 the original.
 In a way. I only need to store one copy of the data, and make note of
 changes between it and the current data.

 Perhaps I am missing a sublty that makes those statements different.

 [snip]
 I take the difference to be this:

 The first statement says there are 2 copies, the local one and the
 original one, with the local one kept in synch with the original one.

 The second statement says there's 1 copy, plus the changes which have
 been made to it.

Okay, so you keep one local copy of the data pristine with what is on the
server, then you want to keep what is effectively a set of local changes
from the what is on the server like a patch so that when you want to access
the data locally, you can generate return a local copy by applying the
patch to the pristine server data.  Then you want to be able to pull down
and incorporate changes from the server, something like what you would do
with update on an SCM.  The informated is still changed on a record by
record basis.

Does that sound correct?

In that case, I would use the same basic scheme that I suggested before
but adding a row in the table to track the changes for each field that
might be locally modified.  Null would of course mark fields that have not
changed.  You could store the changes literaly or using some kind of diff
style delta that could be applied against the pristine copy.  Updating from
the server works as before; but, you need to decide what happens if there
is a conflict between what changes on the server and local changes.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison with False - something I don't understand

[Tim Harig]

On 2010-12-03, Harishankar v.harishan...@gmail.com wrote:
 On Thu, 02 Dec 2010 16:52:57 +, Tim Harig wrote:

 If you are having that issue, then you are likely placing the try blocks
 at too low of a level in your code.  In general you will find that most
 systems have a gateway function as an entry point to the system. If
 there is not one already, then create such a function in you code. The
 parse function in my code above would be an example of such a gateway
 function.  Beneath that function, you don't need to know exactly where
 the error occured, you just need to know the nature of the error and
 have general error handling procedures for each kind of error that you
 expect might occur.

 I think I might very well by using try blocks rather defensively rather 
 than letting the code reach its logical conclusion in normal 
 circumstances. This is why I think I find it a bit clunky. 

That was the conclusion I was coming to.

 I think I understand the general trend of what you're saying. It 
 definitely requires a mindset change. I still feel that user-defined 
 exception classes might not be the way, but maybe I should allow the 
 built-in exceptions which are thrown by library functions to follow its 
 natural path upwards till it reaches the top level where it could be 
 handled. 

Look at it this way, in C you were constrained to place your error
handling code around ever function that might fail.  Now you are free
to place the error handling code wherever it makes sense to do so.
As a general rule, if, in C, your function would handle the error by
passing an error return value to the calling function, then the error
handling code should be higher up.

 Maybe I should handle the error only at the highest level (UI level) 
 rather than returning False to flag errors. 

I don't write many UIs; but, I normally consider the UI code to be yet
another subsystem.  In general, I find the best place to place error
handling code in the high level business logic of your application (which
might be what you actually meant by UI, to me the UI code is the code that
actually draws the interface), in the high level logic of the systems,
and in the bounderies between subsystems.  The exceptions caught at each
level depend on where the logic to handle the error is best applied.

 One of the reasons why I feared to do this is because I need to know each 
 and every exception that might be thrown by the function and litter my 
 top-level code with too many exception handlers.

Each exception has a place where it is better handled.  Wherever you
find boundaries between subsystems, think about what error conditions
that subsystem might encounter.  Subsystems dealing with are likely
to encounter io related errors, network subsystems network errors,
parsers validation errors etc.  Logic exceptions that indicate errors
in your code should be left alone entirely so that they may be easily
found. Look at the exceptions pertaining to these subsystems.

For each error reaching the boundery, think about whether you have enough
information within the module to handle the error in a constructive
manner or whether the error handling would benefit from information
further up in the program.  If you have all of the information that you
need then handle it in the main logic of that subsystem.  If not, pass
it up to the error handlers on top of the boundry.  When you get there,
make the same decision.

In general you only need to catch a handful of exceptions at each level.
The easy excpetions will be handled at lower levels.  The most difficult
exceptions will rise towards the top of the program until only the terminal
exceptions, that cannot be resolved are left with the inevitable result that
you should notify the user and exit, will remain.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python's equivalent to Main calling program and subprograms

[Tim Harig]

On 2010-12-01, goldtech goldt...@worldpost.com wrote:
 Start
 Main
 Global Var
 Subprogram1
 Subprogram2
 Subprogram3
 End of Main
 End

module_wide_var = value

def Subprogram1:
# code

def Subprogram2:
# code

def Subprogram3:
# code

def main:
Subprogram1()
Subprogram2()
Subprogram3()

if __name__ == __main__:
main()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python's equivalent to Main calling program and subprograms

[Tim Harig]

 On Wed, Dec 1, 2010 at 9:08 AM, m b sn...@hotmail.se wrote:
  if __name__ == __main__:
  main()

 What does this mean?

It is a Python idiom and a good practice.  Strictly speaking it is
unnecessary.  Python doesn't recognize any functional initialization
vector other then the start of the file.  When Python loads a module,
it executes anything it finds in the module scope (Anything not in the
body of a class or function declaration).  Using a main function is just
a convention.  You could just place all of your main level code in the
module scope:

def Subprogram1():
# code
def Subprogram2():
# code
def Subprogram3():
# code

# main code

or equivilantly, always execute main():

def Subprogram1():
# code
def Subprogram2():
# code
def Subprogram3():
# code
def main():
# main code
main()

Both are valid from Python's point of view.

The 'if __name__ == __main__:' idiom is used, because it allows the
module to be loaded without running main().  This is useful if you wanted
to use Subprogram2() from another program.  Even if you don't forsee using
any of the subprograms (functions to Python), this can be useful when
writing test code as you can import the program as a module to test its
classes or functions separately.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reading by positions plain text files

[Tim Harig]

On 2010-11-30, javivd javiervan...@gmail.com wrote:
 I have a case now in wich another file has been provided (besides the
 database) that tells me in wich column of the file is every variable,
 because there isn't any blank or tab character that separates the
 variables, they are stick together. This second file specify the
 variable name and his position:

 VARIABLE NAME POSITION (COLUMN) IN FILE
 var_name_1123-123
 var_name_2124-125
 var_name_3126-126
 ..
 ..
 var_name_N512-513 (last positions)

I am unclear on the format of these positions.  They do not look like
what I would expect from absolute references in the data.  For instance,
123-123 may only contain one byte??? which could change for different
encodings and how you mark line endings.  Frankly, the use of the
world columns in the header suggests that the data *is* separated by
line endings rather then absolute position and the position refers to
the line number. In which case, you can use splitlines() to break up
the data and then address the proper line by index.  Nevertheless,
you can use file.seek() to move to an absolute offset in the file,
if that really is what you are looking for.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching user switching and getting current active user from root on linux

[Tim Harig]

On 2010-11-30, mpnordland mpnordl...@gmail.com wrote:
 I have situation where I need to be able to get the current active
 user, and catch user switching eg user1 locks screen, leaves computer,
 user2 comes, and logs on.
 basically, when there is any type of user switch my script needs to
 know.

Well you could use inotify to trigger on any changes to /var/log/wtmp.
When a change is detected, you could check of deltas in the output of who
-a to figure out what has changed since the last time wtmp triggered.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reading by positions plain text files

[Tim Harig]

On 2010-12-01, javivd javiervan...@gmail.com wrote:
 On Nov 30, 11:43 pm, Tim Harig user...@ilthio.net wrote:
 On 2010-11-30, javivd javiervan...@gmail.com wrote:

  I have a case now in wich another file has been provided (besides the
  database) that tells me in wich column of the file is every variable,
  because there isn't any blank or tab character that separates the
  variables, they are stick together. This second file specify the
  variable name and his position:

  VARIABLE NAME      POSITION (COLUMN) IN FILE
  var_name_1                 123-123
  var_name_2                 124-125
  var_name_3                 126-126
  ..
  ..
  var_name_N                 512-513 (last positions)

 I am unclear on the format of these positions.  They do not look like
 what I would expect from absolute references in the data.  For instance,
 123-123 may only contain one byte??? which could change for different
 encodings and how you mark line endings.  Frankly, the use of the
 world columns in the header suggests that the data *is* separated by
 line endings rather then absolute position and the position refers to
 the line number. In which case, you can use splitlines() to break up
 the data and then address the proper line by index.  Nevertheless,
 you can use file.seek() to move to an absolute offset in the file,
 if that really is what you are looking for.

 I work in a survey research firm. the data im talking about has a lot
 of 0-1 variables, meaning yes or no of a lot of questions. so only one
 position of a character is needed (not byte), explaining the 123-123
 kind of positions of a lot of variables.

Then file.seek() is what you are looking for; but, you need to be aware of
line endings and encodings as indicated.  Make sure that you open the file
using whatever encoding was used when it was generated or you could have
problems with multibyte characters affecting the offsets.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite autoincrement of primary key

[Tim Harig]

On 2010-11-29, tinauser tinau...@libero.it wrote:
 '''
 INSERT INTO 'foo' VALUES (?,?)
 '''
 ,('NULL','yyy'))

s/'NULL'/None/

 I get a datatype mismatch error.

The sqlite module is smart enough to convert between Python types and
Sqlite types.  If you pass it 'NULL' it thinks you are passing it a string.
Python uses None in much the same way that databases use NULL, so the
module converts None to 'NULL' and vise versa.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite autoincrement of primary key

[Tim Harig]

On 2010-11-29, D'Arcy J.M. Cain da...@druid.net wrote:
 On Mon, 29 Nov 2010 13:19:19 -0500
 Mel mwil...@the-wire.com wrote:
 tinauser wrote:
 '''INSERT INTO foo VALUES (NULL, ?)'''

 Does this work in SQLite:

   INSERT INTO foo (name) VALUES ('xxx')

 That's the standard SQL way.

Yes, it works; but, the OP asked specifically to be able to enter all of
the field values, including the autoincrement field.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite autoincrement of primary key

[Tim Harig]

On 2010-11-29, D'Arcy J.M. Cain da...@druid.net wrote:
 On Mon, 29 Nov 2010 19:11:18 + (UTC)
 Tim Harig user...@ilthio.net wrote:
INSERT INTO foo (name) VALUES ('xxx')
 
  That's the standard SQL way.
 
 Yes, it works; but, the OP asked specifically to be able to enter all of
 the field values, including the autoincrement field.

 You're right, I missed that.  However reading the OP's message I am
 still confused.  How does removing the field name and adding a
 positional NULL or None improve readability.  I now wonder if it was
 more of an assignment requirement rather than a real one.

That, I don't know.  I would agree that it seems like a mis-guided
approach; but, it is what he asked for.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regular expression help

[Tim Harig]

On 2010-11-30, goldtech goldt...@worldpost.com wrote:
 Hi,

 say:
 import re
 m=cccvlvlvlvnnnflfllffccclfnnnooo
 re.compile(r'ccc.*nnn')
 rtt=.sub(||,m)
 rtt
 '||ooo'

 The regex is eating up too much. What I want is every non-overlapping
 occurrence I think.

 so rtt would be:

 '||flfllff||ooo'

Python 3.1.2 (r312:79147, Oct  9 2010, 00:16:06)
[GCC 4.4.4] on linux2
Type help, copyright, credits or license for more information.
 import re
 m=cccvlvlvlvnnnflfllffccclfnnnooo
 pattern = re.compile(r'ccc[^n]*nnn')
 pattern.sub(||, m)
'||flfllff||ooo'

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: regular expression help

[Tim Harig]

Python 3.1.2 (r312:79147, Oct  9 2010, 00:16:06)
[GCC 4.4.4] on linux2
Type help, copyright, credits or license for more information.
 import re
 m=cccvlvlvlvnnnflfllffccclfnnnooo
 pattern = re.compile(r'ccc[^n]*nnn')
 pattern.sub(||, m)
'||flfllff||ooo'
 # or, assuming that the middle sequence might contain singular or
 # double 'n's
 pattern = re.compile(r'ccc.*?nnn')
 pattern.sub(||, m)
'||flfllff||ooo'

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: remote control firefox with python

[Tim Harig]

On 2010-11-28, News123 news1...@free.fr wrote:
 Thanks in advance for any pointers ideas.

google XPCOM
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I get the URL of the active tab in Firefox/IE/Chrome?

[Tim Harig]

On 2010-11-28, He Jibo hej...@gmail.com wrote:
 I did a lot of Googling, and get the following code. The following
 code can get the url of the first tab in internet explorer. My
 question is, how can I get the url of the current active tab? Thanks.

It would be beneficial to know what your ultimate goal is.  The
InternetExplorer.Application automation object doesn't contain any way to
manipulate tabs directly; but, there are likely less direct methods of
achieving whatever you are trying to accomplish if you let us know what
that is.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I get the URL of the active tab in Firefox/IE/Chrome?

[Tim Harig]

On 2010-11-29, Michel Claveau - MVP enleverlesx_xx...@xmclavxeaux.com.invalid 
wrote:
 Hello!

 The InternetExplorer.Application automation object doesn't contain 
 any way to manipulate tabs directly

 False. Try this example:
   import win32com.client
   for instance in 
 win32com.client.Dispatch('{9BA05972-F6A8-11CF-A442-00A0C90A8F39}'):
   print instance,  URL :,instance.LocationURL

A Shell collection object != a InternetExplorer.Application object.
The problem with using a ShellWindows object is that you could easily
connect to the wrong instance, especially since the OP doesn't know the
LocationURL of the instance that he is looking for.  That is why I asked
for clarification.  Once we know what the OP is trying to do, we can
make sure that he has a reliable method to connect to the proper instance.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie subprocess question

[Tim Harig]

On 2010-11-25, Hugo Léveillé hu...@fastmail.net wrote:
 I'm starting various application using subprocess.Popen without any
 problem. The problem is with application inside Program Files. It
 looks like subprocess is stopping the application string after
 Program. I tried puting the programe name inside double quote like
 'Program File*...'. No luck.

Not sure you you are having issues.

subprocess.Popen(C:\\Program Files\\Vim\\vim73\\gvim.exe)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to run an EXE, with argument, capture output value

[Tim Harig]

C:\Documents and Settings\Tim Harig\My Documents\autoCalcdir
 Volume in drive C has no label.
 Volume Serial Number is 30D9-35E0

 Directory of C:\Documents and Settings\Tim Harig\My Documents\autoCalc

11/19/2010  12:20 PMDIR  .
11/19/2010  12:20 PMDIR  ..
11/19/2010  12:19 PM   686 autoCalc.pys
   1 File(s)686 bytes
   2 Dir(s)  16,343,552,000 bytes free

C:\Documents and Settings\Tim Harig\My Documents\autoCalctype autoCalc.pys
# autoCalc.pys: The pys extension indicates that it should be run under
# Windows Script Host

# perform the calculation using Windows calculator
keySequence = ['2', '{+}', '2', '=', '^c', '%{F4}']
WshShell = WScript.CreateObject(WScript.Shell)
calculator = WshShell.Run(calc)
WshShell.AppActivate(calc)
WScript.Sleep(1000)
for currentKey in keySequence:
WshShell.SendKeys(currentKey)
WScript.Sleep(100)

# write the results to notepad and same as demo.txt
keySequence = ['result: ', '^v', '^s',
'c:\\Documents and Settings\\Tim Harig\\My 
Documents\\autoCalc\\demo.txt',
'~', '%{F4}']
notepad = WshShell.Run(notepad)
WshShell.AppActivate(notepad)
WScript.Sleep(1000)
for currentKey in keySequence:
WshShell.SendKeys(currentKey)
WScript.Sleep(100)

C:\Documents and Settings\Tim Harig\My Documents\autoCalccscript.exe 
autoCalc.pys
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

Debugging extensions (axdebug) module does not exist - debugging is disabled..

C:\Documents and Settings\Tim Harig\My Documents\autoCalctype demo.txt
result: 4
C:\Documents and Settings\Tim Harig\My Documents\autoCalc
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to run an EXE, with argument, capture output value

[Tim Harig]

On 2010-11-18, noydb jenn.du...@gmail.com wrote:
 import subprocess
 pig = subprocess.Popen([C:\Halls\hallbig2.exe],
 stdin=subprocess.PIPE, stdout=subprocess.PIPE)
 result = pig.communicate(input='C:\Halls\Input\Ea39j.txt')[-1] #I need
 to capture the, what I think is the, last output

From the subprocess documentation:

   [62]communicate() returns a tuple (stdoutdata, stderrdata).
   Note that if you want to send data to the process's stdin,
   you need to create the Popen object with stdin=PIPE. Similarly,
   to get anything other than None in the result tuple, you need
   to give stdout=PIPE and/or stderr=PIPE too.

By using index [-1] you are accessing the processes stderr stream.  I am
not really sure why you changed it.  It doesn't give you the last output.
Index 0 gives you *all* of stdout and index 1 gives you *all* of stderr,
period.  If you wish to further disect the output to get say the last line,
then you will need to parse it separately.

 print result
 print pig.returncode
 None
 0

 So the tuple is empty. ??  The exe executes fine and returns output in
 th exe tool itself.  The python script seems to execute fine, no
 errors, '...returned exit code 0'.  Any ideas/suggestions?

No the tuple contains two items (stdout, stderr).  The first is what the
program printed to its stdout stream (which is most likely the output you
see if you run the command at a terminal/console).  The second is what it 
printed to its
stderr stream which is a channel used for out of band data such as error or
status messages.  In this case, it is None, because you did open stderr as a
subprocess.PIPE.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to run an EXE, with argument, capture output value

[Tim Harig]

On 2010-11-18, Jean-Michel Pichavant jeanmic...@sequans.com wrote:
 Tim Harig wrote:
 If you are not already, I would highly suggest using Python3 with the

 subprocess module:
 Suggesting subprocess is a good idea, *highly* suggesting python3 is 
 questionable. The above code works in python 2. Many libraries (those 
 included batteries) have not been ported yet to python 3.
 Py3 is a better core language than py2, but for now, less featured.

I didn't, and I don't, recommend Python3 over Python2 for just any
purpose.I recommended Python3's subprocess module over the Python2's
subprocess module if that is indeed possible.

I happen to be one of those that feels this transition was terribly
mis-handled and that there should have been provisions to allow both
versions to either be maintained together or to allow modules from both
versions to work together.  I don't really have a huge preference for
either version; but, having to deal with both of them has given the
project a black eye.
-- 
http://mail.python.org/mailman/listinfo/python-list