Re: Ruby parens-free function calls [was Re: Accessing parent objects]

[Stephen Hansen]

On Mon, Mar 26, 2018, at 2:19 PM, Rick Johnson wrote:
>Sure, the behavior that Steven
> uncovered is odd, but it could be that Maz harbors a strong
> disliking for undisciplined pupils, and thus, he designed
> and placed this little trap in the hopes the pain it induced
> might encourage the petulant little meat-heads to follow
> some sensible styling rules.

My god, I've been away from this list for quite awhile, but we're still 
entertaining this fool? 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for / while else doesn't make sense

[Stephen Hansen]

On Thu, May 19, 2016, at 07:02 PM, gst wrote:
> Python 4.0 ? My son will thank us !

No, he won't, because while Python 4.0 will happen, likely after Python
3.9, it will not be a major backwards compatible breaking point.

Some people infer that because 3.0 was, 4.0, 5.0, 6.0 are open to it.
They aren't.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Steve D'Aprano, you're the "master". What's wrong with this concatenation statement?

[Stephen Hansen]

On Tue, May 10, 2016, at 09:16 AM, DFS wrote: 
> But no, "master", your answers are incorrect.  What's wrong with that 
> concat statement is that += concatenation is frowned upon by python's 
> creator, and is not recommended (in PEP8):
> 
> 
> Code should be written in a way that does not disadvantage other 
> implementations of Python (PyPy, Jython, IronPython, Cython, Psyco, and 
> such).
> 
> For example, do not rely on CPython's efficient implementation of 
> in-place string concatenation for statements in the form a += b or a = a 
> + b . This optimization is fragile even in CPython (it only works for 
> some types) and isn't present at all in implementations that don't use 
> refcounting. In performance sensitive parts of the library, the 
> ''.join() form should be used instead. This will ensure that 
> concatenation occurs in linear time across various implementations.
> 

You once again misread PEP8. 

Not one of Steven's answers used string concatenation, except for the
implicit literal concatenation, which all other implementations support.

Note:

> sSQL = """line 1
> line 2
> line 3"""

No concatenation. One string literal. Works in all implementations.

> sSQL = "line 1\nline 2\nline 3"

No concatenation. One string literal. Works in all implementations.

>
> sSQL = ("line 1\n"
> "line 2\n"
> "line 3")

Concatenation occurs at compile time, implicitly. Works in all
implementations.

The PEP says when building a string dynamically (that is, adding
together two strings that exist and are separate), use "".join() in
performance sensitive places. It says don't combine two strings with the
addition operator -- because while that's sometimes efficient in
CPython, the language doesn't guarantee it.

You'll notice that the one time Steven combined two strings, it was
implicitly at compile time.

In every Python implementation, it is guaranteed that:

>>> a = "a" "a"
>>> b = "aa"

Are the same. Two+ string literals are implicitly combined into one at
compile time.

> 'master craftswoman' my ass...

Yes, you're being that. Please stop.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should a decorator do if an attribute already exists?

[Stephen Hansen]

On Tue, May 10, 2016, at 08:45 AM, Steven D'Aprano wrote:
> I have a decorator that adds an attribute to the decorated function:
> [...]
> My question is, what should I do if the decorated function already has an
> instrument attribute?
> 
> 1. raise an exception?

This. Your decorator should, IMHO, treat the attribute as private data,
and if something else is using the same thing, something has clearly
gone wrong and raising the error early and clearly is right.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 02:46 PM, DFS wrote:
> On 5/8/2016 5:38 PM, Stephen Hansen wrote:
> > On Sun, May 8, 2016, at 02:16 PM, DFS wrote:
> >> I was surprised to see the PEP8 guide approve of:
> >>
> >> "Yes: if x == 4: print x, y; x, y = y, x"
> >>
> >> https://www.python.org/dev/peps/pep-0008/#pet-peeves
> >
> > That is not approving of that line of code as something to mimic, its
> > speaking *only* about *whitespace*.
> >
> > ALL its saying is, "don't put spaces before commas, colons or
> > semicolons". You can infer nothing else about it.
> 
> I can infer that it's 100% approved of, since he used it as an example.

Not if you don't want to be a fool.

And at this point I'm signing out from helping you.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 02:16 PM, DFS wrote:
> I was surprised to see the PEP8 guide approve of:
> 
> "Yes: if x == 4: print x, y; x, y = y, x"
> 
> https://www.python.org/dev/peps/pep-0008/#pet-peeves

That is not approving of that line of code as something to mimic, its
speaking *only* about *whitespace*.

ALL its saying is, "don't put spaces before commas, colons or
semicolons". You can infer nothing else about it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 08:06 AM, DFS wrote:
> On 5/8/2016 10:36 AM, Chris Angelico wrote:
> > ... and then you just commit???!?
> >
> 
> That's what commit() does.
> 

I assure you, he knows what commit does :)

The point is, you don't usually commit after an error happens. You
rollback. Or correct the data. Since the data didn't go in, there should
(in theory) be nothing TO commit if an error happens. Or, there should
be partial data in that needs a rollback before you decide to do
something else.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 07:25 AM, DFS wrote:
> for nm,street,city,state,zipcd in zip(nms,street,city,state,zipcd):

> > for vals in zip(nms,street,city,state,zipcd):
> > nm,street,city,state,zipcd = vals
> > cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
> 
> 
> I like the first one better.  python is awesome, but too many options 
> for doing the same thing also makes it difficult.  For me, anyway.

Eeh, Now you're just making trouble for yourself.

for name, street, city, state, zipcd in zip(names, streets, cities,
states, zipcds):


may be sorta vaguely long, but its not that long. Just do it and move
on. Get over whatever makes you not like it. 


-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 08:28 PM, DFS wrote:
> >> +-++
> >> |superfluous-parens   |3   | I like to surround 'or'
> >>  statments with parens
> >
> > I would need examples to comment
> 
> 
> if ("Please choose a state" in str(matches)):
> if (var == "val" or var2 == "val2"):

Gah, don't do that. You're adding meaningless noise. 

Especially in the first case.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 08:04 PM, DFS wrote:
> The lists I actually use are:
> 
> for j in range(len(nms)):
>   cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
>   vals = nms[j],street[j],city[j],state[j],zipcd[j]
> 
> 
> The enumerated version would be:
> 
> ziplists = zip(nms,street,city,state,zipcd)
> for nm,street,city,state,zipcd in ziplists:
>   cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
>   vals = nm,street,city,state,zipcd
> 
> 
> I guess the enumeration() is a little nicer to look at.  Why do you 
> think it's more maintainable?

Code is read more then its written.

That which is nicer to look at, therefore, is easier to read.

That which is easier to read is easier to maintain.

Beyond that, its simpler, and more clearly articulates in the local
space what's going on. 

> Aside: I haven't tried, but is 'names' a bad idea or illegal for the 
> name of a python list or variable?

Nothing wrong with names. Or 'name', for that matter. Try to avoid
abbreviations.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 06:16 PM, DFS wrote:

> Why is it better to zip() them up and use:
> 
> for item1, item2, item3 in zip(list1, list2, list3):
>   do something with the items
> 
> than
> 
> for j in range(len(list1)):
> do something with list1[j], list2[j], list3[j], etc.

Although Chris has a perfectly good and valid answer why conceptually
the zip is better, let me put forth: the zip is simply clearer, more
readable and more maintainable.

This is a question of style and to a certain degree aesthetics, so is
somewhat subjective, but range(len(list1)) and list1[j] are all
indirection, when item1 is clearly (if given a better name then 'item1')
something distinct you're working on.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Sat, May 7, 2016, at 12:17 PM, Christopher Reimer wrote:
> On 5/5/2016 6:37 PM, Stephen Hansen wrote:
> > On Thu, May 5, 2016, at 06:26 PM, Christopher Reimer wrote:
> >> Which is one is correct (Pythonic)? Or does it matter?
> > First, pylint is somewhat opinionated, and its default options shouldn't
> > be taken as gospel. There's no correct: filter is fine.
> 
> Since the code I'm working on is resume fodder (i.e., "Yes, I code in 
> Python! Check out my chess engine code on GitHub!"), I want it to be as 
> Pythonic and PEP8-compliant as possible. That includes scoring 10/10 
> with pylint. Never know when an asshat hiring manager would reject my 
> resume out of hand because my code fell short with pylint.
> 
> For my purposes, I'm using the list comprehension over filter to keep 
> pylint happy.

Bear in mind, when I say, "Pylint is opinionated", I mean the tool --
especially in its default configuration -- has its own opinion of what
is good style, and *I think its wrong on a number of points*. 

Its fine to use, but I'd read over PEP8 (the document, not the tool) and
apply style guide recommendations thoughtfully, not mechanically. 

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 11:52 AM, Christopher Reimer wrote:
> You can do better.  You should strive for 10/10 whenever possible, 
> figure out why you fall short and ask for help on the parts that don't 
> make sense.

I think this is giving far too much weight to pylint's opinion on what
is "good" or "bad" programming habits.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

Pylint is very opinionated. Feel free to adjust its configuration to
suit your opinions of style.

In particular, several of these might be related to PEP8 style issues.

On Sat, May 7, 2016, at 09:51 AM, DFS wrote:
>DFS comments
> +-++ ---
> |message id   |occurrences |
> +=++
> |mixed-indentation|186 | I always use tab

And yet, it appears there's some space indentation in there. In
Notepad++ enable View->Show Symbol->Show White Space and Tab and Show
Indent Guide.

> +-++
> |invalid-name |82  | every single variable name?!

It probably defaults to PEP8 names, which are variables_like_this, not
variablesLikeThis.

> +-++
> |bad-whitespace   |65  | mostly because I line up =
>   signs:
>   var1  = value
>   var10 = value

Yeah and PEP8 says don't do that. Adjust the configuration of pylint if
you want.

> +-++
> |multiple-statements  |23  | do this to save lines.
>   Will continue doing it.

This you really shouldn't do, imho. Saving lines is not a virtue,
readability is -- dense code is by definition less readable.

> +-++
> |no-member|5   |
> 
> "Module 'pyodbc' has no 'connect' member"   Yes it does.
> "Module 'pyodbc' has no 'Error' member" Yes it does.
> 
> Issue with pylint, or pyodbc?

Pylint. 

> +-++
> |line-too-long|5   | meh

I'm largely meh on this too. But again its a PEP8 thing.

> +-++
> |wrong-import-order   |4   | does it matter?

Its useful to have a standard so you can glance and tell what's what and
from where, but what that standard is, is debatable.

> +-++
> |missing-docstring|4   | what's the difference between
>   a docstring and a # comment?

A docstring is a docstring, a comment is a comment. Google python
docstrings :) Python prefers files to have a docstring on top, and
functions beneath their definition. Comments should be used as little as
possible, as they must be maintained: an incorrect comment is worse then
no comment.

Go for clear code that doesn't *need* commenting.

> +-++
> |superfluous-parens   |3   | I like to surround 'or'
>   statments with parens

Why?

> +-++
> |multiple-imports |2   | doesn't everyone?

I don't actually know what its complaining at.

> +-++
> |bad-builtin  |2   | warning because I used filter?

Don't know what its complaining at about here either.

> +-++
> |missing-final-newline|1   | I'm using Notepad++, with
>   EOL Conversion set to
>   'Windows Format'.  How
>   or should I fix this?

Doesn't have anything to do with it. Just scroll to the bottom and press
enter. It wants to end on a newline, not code.

> Global evaluation
> -
> Your code has been rated at -7.64/10
> 
> I assume -7.64 is really bad?
> 
> Has anyone ever in history gotten 10/10 from pylint for a non-trivial 
> program?

No clue, I don't use pylint at all.

> [1]
> pylint says "Consider using enumerate instead of iterating with range 
> and len"
> 
> the offending code is:
> for j in range(len(list1)):
>do something with list1[j], list2[j], list3[j], etc.
> 
> enumeration would be:
> for j,item in enumerate(list1):
>do something with list1[j], list2[j], list3[j], etc.
> 
> Is there an advantage to using enumerate() here?

Its cleaner, easier to read. In Python 2 where range() returns a list,
its faster. (In python2, xrange returns a lazy evaluating range)

Use the tools Python gives you. Why reinvent enumerate when its built
in?

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python is an Equal Opportunity Programming Language

[Stephen Hansen]

On Fri, May 6, 2016, at 11:43 PM, Gregory Ewing wrote:
> Steven D'Aprano wrote:
> > Who is setting and enforcing this quota, and given that only about 1 in 20
> > Python programmers is a woman, do you think men are seriously missing out
> > on any opportunities?
> 
> Suppose there are 100 people wanting to ask questions, and
> there is only time to answer 10 questions. If the 1 in 20
> ratio holds, then 5 of those people are women and the other
> 95 are men.
> 
> Alternating between men and women means that all of the
> women get their questions answered, and only 5/95 of the
> men. So in this example, if you're a woman you have a 100%
> chance of getting answered, and if you're a man you only
> have a 5.26% chance.
> 
> Whether you think this is a good strategy or not,
> beliavsky is right that it's not "equal".

This is a pedantically and nonsensical definition of "equal", that
ignores the many, many reasons why there are 1 in 20 women in that
conference. Its looking at the end effect and ignoring everything that
leads up to it, and deciding its instead special rights -- this is the
great argument against minorities getting a voice, that their requests
for equal *opportunity* are instead *special rights* that diminish the
established majority's entrenched power. 

Those women are dealing with suppression, discrimination and dismissal
on multiple levels that leave them in a disenfranchised position. 

Recognizing those faults and taking corrective action is fundamentally
an act in the name of equality. 

Correcting for inequalities can not, itself, be a purely "equal" task
done in pure blindness of the contextual reality of what is going on in
the world. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do these statements evaluate the way they do?

[Stephen Hansen]

On Fri, May 6, 2016, at 11:36 PM, Anthony Papillion wrote:
> I'm trying to figure out why the following statements evaluate the way
> they do and I'm not grasping it for some reason. I'm hoping someone can
> help me.
> 
> 40+2 is 42 #evaluates to True
> But
> 2**32 is 2**32 #evaluates to False
> 
> This is an example taken from a Microsoft blog on the topic. They say the
> reason is because the return is based on identity and not value but, to
> me, these statements are fairly equal.

Yes, those statements are fairly *equal*. But you are not testing
*equality*. The "is" operator is testing *object identity*.

>>> a = 123456
>>> b = 123456
>>> a is b
False
>>> a == b
True

Equality means, "do these objects equal the same", identity means, "are
these specific objects the exact same objects".

In the above, a and b are different objects. There's two objects that
contain a value of 12345. They are equal (see the == test), but they are
different objects. 

If you write on a piece of paper "12345", and I write on a piece of
paper "12345", are those two pieces of paper identical? Of course not.
Your paper is in your hand, mine is in mine. These are different pieces
of paper. Just look at them: they're clearly not the same thing. That
said, if you look at the number they contain, they're the same value.

Our two pieces of paper are equivalent, but they're different objects.

This might get confusing for you because:

>>> a = 123
>>> b = 123
>>> a is b
True
>>> a == b 
True

This happens because Python caches small integers, so everytime you
write '123', it re-uses that same '123' object everyone else is using.
It shares those objects as an optimization.

The long and short of it is: you should almost never use 'is' for
comparing integers (or strings). It doesn't mean what you think it does
and isn't useful to you. Compare equality.

In general, the only things you should use 'is' for is when comparing to
singletons like None, True or False (And consider strongly not comparing
against False/True with is, but instead just 'if thing' and if its True,
it passes).

Otherwise, 'is' should only be used when you're comparing *object
identity*. You don't need to do that usually. Only do it when it matters
to you that an object with value A might be equal to an object of value
B, but you care that they're really different objects.


-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A fun python CLI program for all to enjoy!

[Stephen Hansen]

On Fri, May 6, 2016, at 04:58 PM, DFS wrote:
> Improper f.close didn't seem to affect any of the files my program wrote 
> - and I checked a lot of them when I was writing the code.

To be clear, its not an "improper" f.close. That command is simply not
closing the file. Period. "f.close" is how you get the 'close' function
from the 'f' object, and then... you do nothing with it.

If you removed "f.close" entirely, you'd get the exact same behavior as
you have now. The "f.close" does nothing.

That said, in CPython semantics, closing a file explicitly is often not
required. CPython is reference-counted. Once the references to an object
reaches 0, CPython deletes the object. This is an implementation detail
of the CPython and not a guarantee of the Python language itself, which
is why explicit close calls are preferred.

So while 'f.close' does nothing, CPython might be closing the file
*anyways*, and it might work... but that 'might' is hard to reason about
without a deeper understanding, so using explicit closing mechanics
(either via f.close() or with or something else) is strongly
recommended. 

For example, if you were to do:

for item in sequence:
f = open(item, 'wb')
f.write("blah")

It probably works fine. The first time through, 'f' is bound to a file
object, and you write to it. The second time through, 'f' is bound to a
*new file object*, and the original file object now has 0 references, so
is automatically deleted. 

The last sequence through, f is not closed: the 'for loop' is not a
scope which deletes its internal name bindings when its done. So that
'f' will likely remain open until the very end of the current function,
which may be an issue for you.

Implicit closing actually works in a large number of situations in
CPython, but it isn't a good thing to rely on. It only works in simple
operations where you aren't accidentally storing a reference somewhere
else. You have to keep track of the references in your head to make sure
things will get closed at proper times.

The 'with' statement clearly defines when resources should be closed, so
its preferred (As I see you've adopted from other responses). But its
also needed in other Python implementations which might not follow
CPython's reference counting scheme.

I'm not giving further feedback because MRAB caught everything I thought
was an issue.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Thu, May 5, 2016, at 07:46 PM, Dan Sommers wrote:
> On Thu, 05 May 2016 18:37:11 -0700, Stephen Hansen wrote:
> 
> > ''.join(x for x in string if x.isupper())
> 
> > The difference is, both filter and your list comprehension *build a
> > list* which is not needed, and wasteful. The above skips building a
> > list, instead returning a generator ...
> 
> filter used to build a list, but now it doesn't (where "used to" means
> Python 2.7 and "now" means Python 3.5; I'm too lazy to track down the
> exact point(s) at which it changed):

Oh, didn't know that. Then again the OP was converting the output of
filter *into* a list, which wasted a list either way.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Thu, May 5, 2016, at 06:26 PM, Christopher Reimer wrote:
> Which is one is correct (Pythonic)? Or does it matter?

First, pylint is somewhat opinionated, and its default options shouldn't
be taken as gospel. There's no correct: filter is fine.

That said, the general consensus is, I believe, that list comprehensions
are good, and using them is great.

In your case, though, I would not use a list comprehension. I'd use a
generator comprehension. It looks almost identical:

''.join(x for x in string if x.isupper())

The difference is, both filter and your list comprehension *build a
list* which is not needed, and wasteful. The above skips building a
list, instead returning a generator, and join pulls items out of it one
at a time as it uses them. No needlessly creating a list only to use it
and discard it.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 11:03 AM, Steven D'Aprano wrote:
> - Nobody could possibly want to support non-ASCII text. (Apart from the
> approximately 6.5 billion people in the world that don't speak English of
> course, an utterly insignificant majority.)

Oh, I'd absolutely want to support non-ASCII text. If I have unicode
input, though, I unfortunately have to rely on
https://pypi.python.org/pypi/regex as 're' doesn't support matching on
character properties. 

I keep hoping it'll replace "re", then we could do:

pattern = regex.compile(ru"^\p{Lu}\s&]+$")

where \p{property} matches against character properties in the unicode
database.

> - Data validity doesn't matter, because there's no possible way that you
> might accidentally scrape data from the wrong part of a HTML file and end
> up with junk input.

Um, no one said that. I was arguing that the *regular expression*
doesn't need to be responsible for validation.

> - Even if you do somehow end up with junk, there couldn't possibly be any
> real consequences to that.

No one said that either...

> - It doesn't matter if you match too much, or to little, that just means
> the
> specs are too pedantic.

Or that...

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 05:31 AM, DFS wrote:
> You are out of your mind.

Whoa, now. I might disagree with Steven D'Aprano about how to approach
this problem, but there's no need to be rude. Everyone's trying to help
you, after all.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 10:43 AM, Steven D'Aprano wrote:
> On Thu, 5 May 2016 11:32 pm, Stephen Hansen wrote:
> 
> > On Thu, May 5, 2016, at 12:36 AM, Steven D'Aprano wrote:
> >> Oh, a further thought...
> >> 
> >> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> >> > I don't even care about faster: Its overly complicated. Sometimes a
> >> > regular expression really is the clearest way to solve a problem.
> >> 
> >> Putting non-ASCII letters aside for the moment, how would you match these
> >> specs as a regular expression?
> > 
> > I don't know, but mostly because I wouldn't even try. 
> 
> Really? Peter Otten seems to have found a solution, and Random832 almost
> found it too.
> 
> 
> > The requirements 
> > are over-specified. If you look at the OP's data (and based on previous
> > conversation), he's doing web scraping and trying to pull out good data.
> 
> I'm not talking about the OP's data. I'm talking about *my* requirements.
> 
> I thought that this was a friendly discussion about regexes, but perhaps
> I
> was mistaken. Because I sure am feeling a lot of hostility to the ideas
> that regexes are not necessarily the only way to solve this, and that
> data
> validation is a good thing.

Umm, what? Hostility? I have no idea where you're getting that.

I didn't say that regexs are the only way to solve problems; in fact
they're something I avoid using in most cases. In the OP's case, though,
I did say I thought was a natural fit. Usually, I'd go for
startswith/endswith, "in", slicing and such string primitives before I
go for a regular expression.

"Find all upper cased phrases that may have &'s in them" is something
just specific enough that the built in string primitives are awkward
tools.

In my experience, most of the problems with regexes is people think
they're the hammer and every problem is a nail: and then they get into
ever more convoluted expressions that become brittle.  More specific in
a regular expression is not, necessarily, a virtue. In fact its exactly
the opposite a lot of times.

> > There's no absolutely perfect way to do that because the system he's
> > scraping isn't meant for data processing. The data isn't cleanly
> > articulated.
> 
> Right. Which makes it *more*, not less, important to be sure that your
> regex
> doesn't match too much, because your data is likely to be contaminated by
> junk strings that don't belong in the data and shouldn't be accepted.
> I've
> done enough web scraping to realise just how easy it is to start grabbing
> data from the wrong part of the file.

I have nothing against data validation: I don't think it belongs in
regular expressions, though. That can be a step done afterwards.

> > Instead, he wants a heuristic to pull out what look like section titles.
> 
> Good for him. I asked a different question. Does my question not count?

Sure it counts, but I don't want to engage in your theoretical exercise.
That's not being hostile, that's me not wanting to think about a complex
set of constraints for a regular expression for purely intellectual
reasons.

> I was trying to teach DFS a generic programming technique, not solve his
> stupid web scraping problem for him. What happens next time when he's
> trying to filter a list of floats, or Widgets? Should he convert them to
> strings so he can use a regex to match them, or should he learn about
> general filtering techniques?

Come on. This is a bit presumptuous, don't you think?

> > This translates naturally into a simple regular expression: an uppercase
> > string with spaces and &'s. Now, that expression doesn't 100% encode
> > every detail of that rule-- it allows both Q and Q & A-- but on my own
> > looking at the data, I suspect its good enough. The titles are clearly
> > separate from the other data scraped by their being upper cased. We just
> > need to expand our allowed character range into spaces and &'s.
> > 
> > Nothing in the OP's request demands the kind of rigorous matching that
> > your scenario does. Its a practical problem with a simple, practical
> > answer.
> 
> Yes, and that practical answer needs to reject:
> 
> - the empty string, because it is easy to mistakenly get empty strings
> when
> scraping data, especially if you post-process the data;
> 
> - strings that are all spaces, because "   " cannot possibly be a
> title;
> 
> - strings that are all ampersands, because "&&&&&" is not a title, and it
> almost surely indicates that your scraping has gone wrong and you're
> reading junk from somewhere;
> 
> - even leading and trailing spaces are suspe

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 12:36 AM, Steven D'Aprano wrote:
> Oh, a further thought...
> 
> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> > I don't even care about faster: Its overly complicated. Sometimes a
> > regular expression really is the clearest way to solve a problem.
> 
> Putting non-ASCII letters aside for the moment, how would you match these 
> specs as a regular expression?

I don't know, but mostly because I wouldn't even try. The requirements
are over-specified. If you look at the OP's data (and based on previous
conversation), he's doing web scraping and trying to pull out good data.
There's no absolutely perfect way to do that because the system he's
scraping isn't meant for data processing. The data isn't cleanly
articulated.

Instead, he wants a heuristic to pull out what look like section titles. 

The OP looked at the data and came up with a simple set of rules that
identify these section titles:

>> Want to keep all elements containing only upper case letters or upper 
case letters and ampersand (where ampersand is surrounded by spaces)

This translates naturally into a simple regular expression: an uppercase
string with spaces and &'s. Now, that expression doesn't 100% encode
every detail of that rule-- it allows both Q and Q & A-- but on my own
looking at the data, I suspect its good enough. The titles are clearly
separate from the other data scraped by their being upper cased. We just
need to expand our allowed character range into spaces and &'s.

Nothing in the OP's request demands the kind of rigorous matching that
your scenario does. Its a practical problem with a simple, practical
answer.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 12:04 AM, Steven D'Aprano wrote:
> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> > > On Wed, May 4, 2016, at 11:04 PM, Steven D'Aprano wrote:
> >> Start by writing a function or a regex that will distinguish strings that
> >> match your conditions from those that don't. A regex might be faster, but
> >> here's a function version.
> >> ... snip ...
> > 
> > Yikes. I'm all for the idea that one shouldn't go to regex when Python's
> > powerful string type can answer the problem more clearly, but this seems
> > to go out of its way to do otherwise.
> > 
> > I don't even care about faster: Its overly complicated. Sometimes a
> > regular expression really is the clearest way to solve a problem.
> 
> You're probably right, but I find it easier to reason about matching in 
> Python rather than the overly terse, cryptic regular expression mini-
> language.
> 
> I haven't tested my function version, but I'm 95% sure that it is
> correct. 
> It trickiest part of it is the logic about splitting around ampersands.
> And 
> I'll cheerfully admit that it isn't easy to extend to (say) "ampersand,
> or 
> at signs". But your regex solution:
> 
> r"^[A-Z\s&]+$"
> 
> is much smaller and more compact, but *wrong*. For instance, your regex 
> wrongly accepts both "&&&&&" and "  " as valid strings, and wrongly 
> rejects "ΔΣΘΛ". Your Greek customers will be sad...

Meh. You have a pedantic definition of wrong. Given the inputs, it
produced right output. Very often that's enough. Perfect is the enemy of
good, it's said. 

There's no situation where "&&&&&" and " " will exist in the given
dataset, and recognizing that is important. You don't have to account
for every bit of nonsense. 

If the OP needs a unicode-aware solution that redefines "A-Z" as perhaps
"\w" with an isupper call. Its still far simpler then you're suggesting.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Wed, May 4, 2016, at 11:04 PM, Steven D'Aprano wrote:
> Start by writing a function or a regex that will distinguish strings that 
> match your conditions from those that don't. A regex might be faster, but 
> here's a function version.
> ... snip ...

Yikes. I'm all for the idea that one shouldn't go to regex when Python's
powerful string type can answer the problem more clearly, but this seems
to go out of its way to do otherwise.

I don't even care about faster: Its overly complicated. Sometimes a
regular expression really is the clearest way to solve a problem.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Wed, May 4, 2016, at 09:58 PM, DFS wrote:
> Want to whittle a list like this:
> 
> [u'Espa\xf1ol', 'Health & Fitness Clubs (36)', 'Health Clubs & 
> Gymnasiums (42)', 'Health Fitness Clubs', 'Name', 'Atlanta city guide', 
> 'edit address', 'Tweet', 'PHYSICAL FITNESS CONSULTANTS & TRAINERS', 
> 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 
> 'www.custombuiltpt.com/', 'RACQUETBALL COURTS PRIVATE', 
> 'www.lafitness.com', 'GYMNASIUMS', 'HEALTH & FITNESS CLUBS', 
> 'www.lafitness.com', 'HEALTH & FITNESS CLUBS', 'www.lafitness.com', 
> 'PERSONAL FITNESS TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & 
> PHYSICAL FITNESS PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & 
> GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'PERSONAL FITNESS TRAINERS', 
> '5', '4', '3', '2', '1', 'Yellow Pages', 'About Us', 'Contact Us', 
> 'Support', 'Terms of Use', 'Privacy Policy', 'Advertise With Us', 
> 'Add/Update Listing', 'Business Profile Login', 'F.A.Q.']
> 
> down to
> 
> ['PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 
> 'HEALTH CLUBS & GYMNASIUMS', 'RACQUETBALL COURTS PRIVATE', 'GYMNASIUMS', 
> 'HEALTH & FITNESS CLUBS', 'HEALTH & FITNESS CLUBS',  'PERSONAL FITNESS 
> TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS 
> PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS 
> & GYMNASIUMS', 'PERSONAL FITNESS TRAINERS']

Sometimes regular expressions are the tool to do the job:

Given:

>>> input = [u'Espa\xf1ol', 'Health & Fitness Clubs (36)', 'Health Clubs & 
>>> Gymnasiums (42)', 'Health Fitness Clubs', 'Name', 'Atlanta city guide', 
>>> 'edit address', 'Tweet', 'PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH 
>>> CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'www.custombuiltpt.com/', 
>>> 'RACQUETBALL COURTS PRIVATE', 'www.lafitness.com', 'GYMNASIUMS', 'HEALTH & 
>>> FITNESS CLUBS', 'www.lafitness.com', 'HEALTH & FITNESS CLUBS', 
>>> 'www.lafitness.com', 'PERSONAL FITNESS TRAINERS', 'HEALTH CLUBS & 
>>> GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS PROGRAMS', 'FITNESS CENTERS', 
>>> 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'PERSONAL FITNESS 
>>> TRAINERS', '5', '4', '3', '2', '1', 'Yellow Pages', 'About Us', 'Contact 
>>> Us', 'Support', 'Terms of Use', 'Privacy Policy', 'Advertise With Us', 
>>> 'Add/Update Listing', 'Business Profile Login', 'F.A.Q.']

Then:

>>> pattern = re.compile(r"^[A-Z\s&]+$")
>>> output = [x for x in list if pattern.match(x)]
>>> output
['PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH CLUBS & GYMNASIUMS',
'HEALTH CLUBS & GYMNASIUMS', 'RACQUETBALL COURTS PRIVATE', 'GYMNASIUMS',
'HEALTH & FITNESS CLUBS', 'HEALTH & FITNESS CLUBS', 'PERSONAL FITNESS
TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS
PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS
& GYMNASIUMS', 'PERSONAL FITNESS TRAINERS']

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No SQLite newsgroup, so I'll ask here about SQLite, python and MS Access

[Stephen Hansen]

On Wed, May 4, 2016, at 03:46 PM, DFS wrote:
> I can't find anything on the web.

Have you tried: 
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
If you really must access it over a newsgroup, you can use the Gmane
mirror:
http://gmane.org/info.php?group=gmane.comp.db.sqlite.general

> Any ideas?

Sorry, I don't use Access.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Conditionals And Control Flows

[Stephen Hansen]

On Wed, May 4, 2016, at 07:41 AM, Cai Gengyang wrote:
> I am trying to understand the boolean operator "and" in Python. It is
> supposed to return "True" when the expression on both sides of "and" are
> true 

The thing is, its kinda dubious to think of 'and' as a 'boolean
operator', because once you go down that road, some people start wanting
it to be a *pure* boolean operator. Something that always returns True
or False. Instead, 'and' and 'or' return something that is true, or
something that is false. Notice the lower case. (I know the docs call
them Boolean Operations, but I still think saying 'boolean' is
unhelpful)

Python defines false things as False, None, 0 (of any numeric type), an
empty container (lists, tuples, mappings, something else that defines
__len__ and it returns 0), and instances of classes that define
__nonzero__ that return 0 or False.

Everything else is a true thing.

If you see "x and y", the rule is: if x is a false thing, it'll return
something false. As it happens, it has x handy, and since its decided x
is false, it'll return that. Therefore, "x and y" is false. If x is
true, though, it'll return y. In this case, "x and y" will be a true
thing if y is a true thing, and a false thing if y is a false thing.

As you can see, all of this logic happens without ever using True or
False. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Mon, May 2, 2016, at 08:57 PM, jf...@ms4.hinet.net wrote:
> Stephen Hansen at 2016/5/3 11:49:22AM wrote:
> > On Mon, May 2, 2016, at 08:27 PM, jf...@ms4.hinet.net wrote:
> > > But when I try to get this forum page, it does get a html file but can't
> > > be viewed normally.
> > 
> > What does that mean?
> > 
> > -- 
> > Stephen Hansen
> >   m e @ i x o k a i . i o
> 
> The page we are looking at:-)
> https://groups.google.com/forum/#!topic/comp.lang.python/jFl3GJbmR7A

Try scraping gmane. Google Groups is one big javascript application.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Saving Consol outputs in a python script

[Stephen Hansen]

On Tue, May 3, 2016, at 05:14 AM, drewes@gmail.com wrote:
> What I need are the 2 values for the 2 classes saved in a variable in the
> .py script, so that I can write them into a text file.
> 
> Would be super nice if someone could help me!

You shouldn't use the call() convienence function, but instead create a
process using the Popen constructor, passing PIPE to stdout. Then use
communicate() to get the output.

This should get you started:

process = subprocess.Popen(["commandline"], stdout=subprocess.PIPE)
output, error = process.communicate()

Output will be a string, string has a splitlines method, etc.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Mon, May 2, 2016, at 08:27 PM, jf...@ms4.hinet.net wrote:
> But when I try to get this forum page, it does get a html file but can't
> be viewed normally.

What does that mean?

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to clean up list items?

[Stephen Hansen]

On Mon, May 2, 2016, at 11:09 AM, DFS wrote:
> I'd prefer to get clean data in the first place, but I don't know a 
> better way to extract it from the HTML.

Ah, right. I didn't know you were scraping HTML. Scraping HTML is rarely
clean so you have to do a lot of cleanup.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python3 html scraper that supports javascript

[Stephen Hansen]

On Mon, May 2, 2016, at 08:33 AM, zljubi...@gmail.com wrote:
> I tried to use the following code:
> 
> from bs4 import BeautifulSoup
> from selenium import webdriver
> 
> PHANTOMJS_PATH =
> 'C:\\Users\\Zoran\\Downloads\\Obrisi\\phantomjs-2.1.1-windows\\bin\\phantomjs.exe'
> 
> url =
> 'https://hrti.hrt.hr/#/video/show/2203605/trebizat-prica-o-jednoj-vodi-i-jednom-narodu-dokumentarni-film'
> 
> browser = webdriver.PhantomJS(PHANTOMJS_PATH)
> browser.get(url)
> 
> soup = BeautifulSoup(browser.page_source, "html.parser")
> 
> x = soup.prettify()
> 
> print(x)
> 
> When I print x variable, I would expect to see something like this:
>  src="mediasource:https://hrti.hrt.hr/2e9e9c45-aa23-4d08-9055-cd2d7f2c4d58;
> id="vjs_video_3_html5_api" class="vjs-tech" preload="none"> type="application/x-mpegURL"
> src="https://prd-hrt.spectar.tv/player/get_smil/id/2203605/video_id/2203605/token/Cny6ga5VEQSJ2uZaD2G8pg/token_expiration/1462043309/asset_type/Movie/playlist_template/nginx/channel_name/trebiat__pria_o_jednoj_vodi_i_jednom_narodu_dokumentarni_film/playlist.m3u8?foo=bar;>
> 
> 
> but I can't come to that point.

Why? As important as it is to show code, you need to show what actually
happens and what error message is produced.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to clean up list items?

[Stephen Hansen]

On Mon, May 2, 2016, at 09:33 AM, DFS wrote:
> Have: list1 = ['\r\n   Item 1  ','  Item 2  ','\r\n  ']

I'm curious how you got to this point, it seems like you can solve the
problem in how this is generated.

> Want: list1 = ['Item 1','Item 2']

That said:

list1 = [t.strip() for t in list1 if t and not t.isspace()]

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Mon, May 2, 2016, at 12:37 AM, DFS wrote:
> On 5/2/2016 2:27 AM, Stephen Hansen wrote:
> > I'm again going back to the point of: its fast enough. When comparing
> > two small numbers, "twice as slow" is meaningless.
> 
> Speed is always meaningful.
> 
> I know python is relatively slow, but it's a cool, concise, powerful 
> language.  I'm extremely impressed by how tight the code can get.

I'm sorry, but no. Speed is not always meaningful. 

It's not even usually meaningful, because you can't quantify what "speed
is". In context, you're claiming this is twice as slow (even though my
tests show dramatically better performance), but what details are
different?

You're ignoring the fact that Python might have a constant overhead --
meaning, for a 1k download, it might have X speed cost. For a 1meg
download, it might still have the exact same X cost.

Looking narrowly, that overhead looks like "twice as slow", but that's
not meaningful at all. Looking larger, that overhead is a pittance.

You aren't measuring that.

> > You have an assumption you haven't answered, that downloading a 10 meg
> > file will be twice as slow as downloading this tiny file. You haven't
> > proven that at all.
> 
> True.  And it has been my assumption - tho not with 10MB file.

And that assumption is completely invalid.

> I noticed urllib and curl returned the html as is, but urllib2 and 
> requests added enhancements that should make the data easier to parse. 
> Based on speed and functionality and documentation, I believe I'll be 
> using the requests HTTP library (I will actually be doing a small amount 
> of web scraping).

The requests library's added-value is ease-of-use, and its overhead is
likely tiny: so using it means you spend less effort making a thing
happen. I recommend you embrace this. 

> VBScript
> 1st run: 7.70 seconds
> 2nd run: 5.38
> 3rd run: 7.71
> 
> So python matches or beats VBScript at this much larger file.  Kewl.

This is what I'm talking about: Python might have a constant overhead,
but looking at larger operations, its probably comparable. Not fast,
mind you. Python isn't the fastest language out there. But in real world
work, its usually fast enough.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:59 PM, DFS wrote:
> startTime = time.clock()
> for i in range(loops):
>   r = urllib2.urlopen(webpage)
>   f = open(webfile,"w")
>   f.write(r.read())
>   f.close
> endTime = time.clock()  
> print "Finished urllib2 in %.2g seconds" %(endTime-startTime)

Yeah on my system I get 1.8 out of this, amounting to 0.18s. 

I'm again going back to the point of: its fast enough. When comparing
two small numbers, "twice as slow" is meaningless.

You have an assumption you haven't answered, that downloading a 10 meg
file will be twice as slow as downloading this tiny file. You haven't
proven that at all. 

I suspect you have a constant overhead of X, and in this toy example,
that makes it seem twice as slow. But when downloading a file of size,
you'll have the same constant factor, at which point the difference is
irrelevant. 

If you believe otherwise, demonstrate it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 10:23 PM, DFS wrote:
> Trying the rawstring thing (say it fast 3x):
> 
> webpage = "http://econpy.pythonanywhere.com/ex/001.html;
> 
> 
> webfile = "D:\\econpy001.html"
> urllib.urlretrieve(webpage,webfile) WORKS
> 
> webfile = "rD:\econpy001.html"

The r is *outside* the string.

Its: r"D:\econpy001.html"

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Code Opinion - Enumerate

[Stephen Hansen]

On Sun, May 1, 2016, at 08:17 PM, Sayth Renshaw wrote:
> Just looking for your opinion on style would you write it like this
> continually calling range or would you use enumerate instead, or neither
> (something far better) ?

I can't comment on your specific code because there's too much noise to
it, but in general:

Using enumerate increases readability, and I use it whenever the idiom:

for index, item in enumerate(thing):
...

is used.

Enumerate is your friend. Hug it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 10:08 PM, DFS wrote:
> On 5/2/2016 1:02 AM, Stephen Hansen wrote:
> >> I actually use "D:\\file.html" in my code.
> >
> > Or you can do that. But the whole point of raw strings is not having to
> > escape slashes :)
> 
> 
> Nice.  Where/how else is 'r' used?

Raw strings are primarily used A) for windows paths, and more
universally, B) for regular expressions. 

But in theory they're useful anywhere you have static/literal data that
might include backslashes where you don't actually intend to use any
escape characters.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:04 PM, DFS wrote:
> And two small numbers turn into bigger numbers when the webpage is big, 
> and soon the download time differences are measured in minutes, not half 
> a second.

Are you sure of that? Have you determined that the time is not a
constant overhead verses that the time is directly relational to the
size of the page? If so, how have you determined that?

You aren't showing how you're testing. 0.4s difference is meaningless to
me, if its a constant overhead. If its twice as slow for a 1 meg file,
then you might have an issue. Maybe. You haven't shown that.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:00 PM, DFS wrote:
> I tried the 10-loop test several times with all versions.

Also how, _exactly_, are you testing this?

C:\Python27>python -m timeit "filename='C:\\test.txt';
webpage='http://econpy.pythonanywhere.com/ex/001.html'; import urllib2;
r = urllib2.urlopen(webpage); f = open(filename, 'w');
f.write(r.read()); f.close();"
10 loops, best of 3: 175 msec per loop

That's a whole lot less the 0.88secs.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 09:50 PM, DFS wrote:
> On 5/2/2016 12:40 AM, Chris Angelico wrote:
> > On Mon, May 2, 2016 at 2:34 PM, Stephen Hansen <me+pyt...@ixokai.io> wrote:
> >> On Sun, May 1, 2016, at 09:06 PM, DFS wrote:
> >>> Then I tested them in loops - the VBScript is MUCH faster: 0.44 for 10
> >>> iterations, vs 0.88 for python.
> >> ...
> >>> I know it's asking a lot, but is there a really fast AND really short
> >>> python solution for this simple thing?
> >>
> >> 0.88 is not fast enough for you? That's less then a second.
> >
> > Also, this is timings of network and disk operations. Unless something
> > pathological is happening, the language used won't make any
> > difference.
> >
> > ChrisA
> 
> 
> Unfortunately, the VBScript is twice as fast as any python method.

And 0.2 is twice as fast as 0.1. When you have two small numbers, 'twice
as fast' isn't particularly meaningful as a metric. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 09:51 PM, DFS wrote:
> On 5/2/2016 12:31 AM, Stephen Hansen wrote:
> > On Sun, May 1, 2016, at 08:39 PM, DFS wrote:
> >> To save a webpage to a file:
> >> -
> >> 1. import urllib
> >> 2. urllib.urlretrieve("http://econpy.pythonanywhere.com
> >>  /ex/001.html","D:\file.html")
> >> -
> >
> > Note, for paths on windows you really want to use a rawstring. Ie,
> > r"D:\file.html".
> > 
> Thanks.
> 
> I actually use "D:\\file.html" in my code.

Or you can do that. But the whole point of raw strings is not having to
escape slashes :) 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 09:06 PM, DFS wrote:
> Then I tested them in loops - the VBScript is MUCH faster: 0.44 for 10 
> iterations, vs 0.88 for python.
...
> I know it's asking a lot, but is there a really fast AND really short 
> python solution for this simple thing?

0.88 is not fast enough for you? That's less then a second.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 08:39 PM, DFS wrote:
> To save a webpage to a file:
> -
> 1. import urllib
> 2. urllib.urlretrieve("http://econpy.pythonanywhere.com
>  /ex/001.html","D:\file.html")
> -

Note, for paths on windows you really want to use a rawstring. Ie,
r"D:\file.html".

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Not x.islower() has different output than x.isupper() in list output...

[Stephen Hansen]

On Sat, Apr 30, 2016, at 09:48 AM, Christopher Reimer wrote:
> On 4/29/2016 11:43 PM, Stephen Hansen wrote:
> > The official documentation is accurate. 
> 
> That may be true on a technical level. But the identically worded text 
> in the documentation implies otherwise. 

That's the thing -- no it doesn't. The documentation is very specific:
if all cased characters in the string are lowercase, and there's at
least one cased character.

It only seems to because you're packing a loop and test on substrings
into an operation.

   list(filter((lambda x: not x.islower()), string))

Let's unpack that. This is basically what you're doing:

result = []
for ch in string:
if not ch.islower():
result.append(ch)

You're thinking of the whole "string", but you're operating on
single-character substrings, and when " ".islower() is run, its false.
Because the two-pronged test, a) if all cased characters are lowercase
and b) there is at least one cased character. b) is failing. Ergo,
you're getting the underscores.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Not x.islower() has different output than x.isupper() in list output...

[Stephen Hansen]

On Fri, Apr 29, 2016, at 06:55 PM, Christopher Reimer wrote:
> On 4/29/2016 6:29 PM, Stephen Hansen wrote:
> > If isupper/islower were perfect opposites of each-other, there'd be no 
> > need for both. But since characters can be upper, lower, or *neither*, 
> > you run into this situation.
> 
> Based upon the official documentation, I was expecting perfect opposites.
> 
> str.islower(): "Return true if all cased characters [4] in the string 
> are lowercase and there is at least one cased character, false
> otherwise."

The thing is, your use of filter is passing a string of 1 character long
to your function, and so when it comes up against a string " ", there
are no cased characters. Therefore, false.

The documentation holds true. 

Your use of filter is breaking "Whiskey Tango Foxtrot" into ["W", "h",
"i", ... "o" t"] and calling not x.islower() vs x.isupper() on each.

When it comes up against " ", it doesn't pass the documented definition
of what islower defines. 

The official documentation is accurate.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Not x.islower() has different output than x.isupper() in list output...

[Stephen Hansen]

On Fri, Apr 29, 2016, at 06:17 PM, Christopher Reimer wrote:
> Greetings,
> 
> I was playing around with a piece of code to remove lowercase letters 
> and leave behind uppercase letters from a string when I got unexpected 
> results.
> 
>  string = 'Whiskey Tango Foxtrot'
> 
>  list(filter((lambda x: not x.islower()), string))
> 
>  ['W', ' ', 'T', ' ', 'F']
> 
> Note the space characters in the list.
> 
>  list(filter((lambda x: x.isupper()), string))
> 
>  ['W', 'T', 'F']
> 
> Note that there are no space characters in the list.
> 

> Shouldn't the results between 'not x.islower()' and 'x.isupper()' be 
> identical?

Absolutely not. There are characters which are neither uppercase or
lowercase: specifically, whitespace in your situation. 

"x.islower()" is true for everything that is lowercase, and false for
everything else. (Including things which have no case). If you invert
with not, then its false for everything that is lowercase, but true for
everything else: including things which have no case.

"x.isupper()" is true for everything that is uppercase, and false for
everything else. This is therefore a narrower test then "not
x.islower()"

x.isupper() is a narrower, more specific test then "not x.islower()". If
you used "not x.isupper()" you'd get the whitespace in it too (along
with all the lower case characters). 

If isupper/islower were perfect opposites of each-other, there'd be no
need for both. But since characters can be upper, lower, or *neither*,
you run into this situation. 

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Madlibs.py code and error message

[Stephen Hansen]

On Wed, Apr 27, 2016, at 11:55 PM, Ben Finney wrote:
> Stephen Hansen <m...@ixokai.io> writes:
> 
> > On Wed, Apr 27, 2016, at 10:32 PM, Ben Finney wrote:
> > > Better: when you have many semantically-different values, use named
> > > (not positional) parameters in the format string. […]
> > > 
> > > <URL:https://docs.python.org/3/library/string.html#formatstrings>
> >
> > Except the poster is not using Python 3, so all of this is for naught. 
> 
> Everything I described above works fine in Python 2. Any still-supported
> version has ‘str.format’.

This response is completely unhelpful. The OP is using Python 2, and
using %-formatting, and so you give a series of examples of using
str.format, to, what? Confuse matters?

You can show using non-positional values using the format the user is
using -- "%(name)s" formatting. You even reference a link to the Python
3 docs, even though the OP is running code which isn't 3.x compatible.
Confusion.

Wishing Python2 away isn't helpful. 

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Madlibs.py code and error message

[Stephen Hansen]

On Wed, Apr 27, 2016, at 10:32 PM, Ben Finney wrote:
> Stephen Hansen <me+pyt...@ixokai.io> writes:
> 
> > The error message means there's a mismatch between the number of
> > formatting instructions (ie, %s) and arguments passed to formatting. I
> > leave it to you to count and find what's missing or extra, because I'm
> > seriously not going to do that :)
> 
> Better: when you have many semantically-different values, use named (not
> positional) parameters in the format string.
> 
> Some simple format string examples:
> 
> "First, thou shalt count to {0}" # References first positional
> argument
> "Bring me a {}"  # Implicitly references the first
> positional argument
> "From {} to {}"  # Same as "From {0} to {1}"
> "My quest is {name}" # References keyword argument 'name'
> […]
> 
> <URL:https://docs.python.org/3/library/string.html#formatstrings>

Except the poster is not using Python 3, so all of this is for naught. 

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Madlibs.py code and error message

[Stephen Hansen]

On Wed, Apr 27, 2016, at 10:01 PM, Cai Gengyang wrote:
> I changed it to all lowercase, this time I get a different error message
> though (a TypeError message) 

The error message means there's a mismatch between the number of
formatting instructions (ie, %s) and arguments passed to formatting. I
leave it to you to count and find what's missing or extra, because I'm
seriously not going to do that :)

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Madlibs.py code and error message

[Stephen Hansen]

On Wed, Apr 27, 2016, at 09:37 PM, Cai Gengyang wrote:
> print STORY % (Adjective1, name, Verb1, Adjective2, Noun1, Noun2, animal,
> food, Verb2, Noun3, fruit, Adjective3, name, Verb3, number, name ,
> superhero_name, superhero_name, name, country, name, dessert, name, year,
> Noun4)

Python is case-sensitive. "Adjective1" and "adjective1" are separate
things. In your code you're reading into "adjective1".

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python path and append

[Stephen Hansen]

On Tue, Apr 26, 2016, at 07:56 PM, Seymore4Head wrote:
> On Tue, 26 Apr 2016 11:53:57 +1000, Steven D'Aprano
> <st...@pearwood.info> wrote:
> 
> >On Tue, 26 Apr 2016 08:04 am, Seymore4Head wrote:
> >
> >> BTW I was trying to use a line like yours that used an output file
> >> that didn't exist and was getting an error.  I assume that import os
> >> fixes that.
> >
> >
> >Why would you assume that?
> >
> >
> >"Doctor, I have a problem with my arm, but I won't tell you what. I assume
> >that if I take cough drops that will fix it."
> 
> OK.  Dumb question acknowledged.
> 
> I got an error when I tried to write to a non existent file.  I
> "incorrectly" assumed wrong.  That is going to be a common theme.

Oh, he wasn't saying it was a dumb question. He was complaining you said
you were "getting an error". That's not a dumb question, that's a
useless report of a problem. If you don't say what exactly the error is,
we can't help you.

If you want help, the two best things to do are:
  1) Show actual code. 
  2) Show actual, complete tracebacks. 

Having a nice description of what you expect to happen is often nice
too, especially if its doing something "wrong" and not giving an obvious
traceback. Seeing specifically what the wrong behavior is, and you
explaining why you think its wrong, can be invaluable. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python(x,y) 64 bit

[Stephen Hansen]

On Wed, Apr 27, 2016, at 04:25 PM, Pierre wrote:
> I did check and it looks like the Python(x,y) 64 distribution I
> downloaded uses a 32 bit Python.
> The question is if there is ANY Python(x,y) 64 distribution that uses the
> 64 bit python version.
> I looked it up online and could not find anything related to this issue

I don't know anything about Python(x,y), but from their website it looks
like it doesn't offer 64-bit python. 

Note, Python(x,y) is not affiliated with Python itself.

That said, perhaps you should check out
https://www.continuum.io/downloads which is the Anaconda scientific
distribution, which I know does offer 64-bit Python support.

---
Stephen Hansen
  m e @ i x o k a i . i o

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How much sanity checking is required for function inputs?

[Stephen Hansen]

On Thu, Apr 21, 2016, at 08:33 PM, Christopher Reimer wrote:
> On 4/21/2016 7:20 PM, Stephen Hansen wrote:
> > I... that... what... I'd forget that link and pretend you never went
> > there. Its not helpful.
> 
> I found it on the Internet, so it must be true -- and Pythonic at that!

My advice is to not look at that site further. I can't count the number
of things that are just... not useful or helpful. 

Directly translating the Gang of Four Design Pattern book to Python
doesn't generally result in useful ideas, except in certain abstractions
like the visitor pattern when you're designing big systems. 



> 
> > What's the contents of this big dictionary that has everything in it 
> > for some reason?
> 
> Keep in mind that I'm coming from a Java background (not by choice) with 
> a smattering of C programming. I initially had ALL THESE CONSTANTS in 
> different parts of my code and couldn't easily use them across different 
> modules. I didn't like all these constants, created a singleton class 
> that uses ConfigParser, dumped everything into a .ini file, and accessed 
> from in each module. As I learn to write Pythonic code, the constants 
> may go away. Switching from my original code to the factory method 
> eliminated a half-dozen constants.

I'm not criticizing, I'm *asking* in the hopes you'll explain what
you're doing better, so I can understand and give advice that's more
relevant: I don't understand what it is you're doing with the VARS or
const dictionary. Even with your explanation here, I don't get it. 

Why do you need to read VARS["COLOR_BLACK"] or the new
const["COLOR_BLACK"} from a configuration file? How is this stuff
configurable? 

Why not, 'color in ("black", "white")'?

The only reason I can think of why you'd want to read what is "black" or
"white" out of a configuration file is internationalization, and if
that's your aim, there's a couple really good projects I can recommend
that solve that issue better. 

I can't think of any reason why you'd want a structure like:
struct = {}
struct["COLOR_BLACK"] = "black"
struct["COLOR_WHITE"] = "white"

"black" is a perfectly good way to express "black", you have this
indirection/abstraction for a reason I don't understand. Granted, maybe
this is a heavy javaism, and I'm utterly ignorant of Java, but my C/C++
mind can't think of a reason for this either.

That said, if you're wanting to share constants across different parts
of your code, use a module.

Create a module, say, things.py, that itself doesn't import other
things, and it says:

black = "Black"
white = "White"

(Bear in mind I still don't know why you want or need this indirection)

Then you "import things" and refer to those constants as things.black,
things.white.

You can then "if color in (things.black, things.white)". Maybe even "if
color in things.colors" if in your things file you have things = (black,
white).

Modules are great for sharing information. The admonition of GLOBALS R
BAD is not as strong in Python as you find in some other languages.
(Don't take that to mean GLOBALS R GUD, granted. Passing state is best
when it makes sense).

--S
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How much sanity checking is required for function inputs?

[Stephen Hansen]

On Thu, Apr 21, 2016, at 06:34 PM, Christopher Reimer wrote:
> class PieceFactory(object):
> 
>  def factory(color, piece, position):
>  if piece == 'Bishop':
>  return Bishop(color, position)
>  if piece == 'King':
>  return King(color, position)
>  if piece == 'Knight':
>  return Knight(color, position)
>  if piece == 'Pawn':
>  return Pawn(color, position)
>  if piece == 'Queen':
>  return Queen(color, position)
>  if piece == 'Rook':
>  return Rook(color, position)

This whole section is begging for a dictionary. Like...

_PIECE_TYPES= {"Bishop": Bishop, "King": King, ...]

class PieceFactory(object):
def factory(color, piece, position):
klass = __PIECE_TYPES.get(piece)
if klass is None:
raise PieceException("...")

return klass(color, position)

Or something like that.

That said, I'm not sure why its not just a function that does the same
thing. Why is it in a class that only does one thing? You never even
instantiate it.

> def generate_set(color, pieces, positions):
>  for piece, position in zip(pieces, positions):
>  yield getattr(PieceFactory, 'factory')(color, piece, position)

Whyyy are you using getattr? Something wrong with
PieceFactory.factory(color, piece, position)? (Or, better yet, yield
piece_factory(color, piece, position) where piece_factory is just a
function)

> I got the factory method from here: 
> http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Factory.html

I... that... what... I'd forget that link and pretend you never went
there. Its not helpful.

> Finally, VARS['VARIABLE_NAME'] got change to const['variable_name']. 
> Should smell better.

I don't know that this changes anything about the small at all. What's
the contents of this big dictionary that has everything in it for some
reason?

That said, dear god, 'piece' doesn't look like an english word to me
anymore. I've never suffered semantic satiation from text before.

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: Re: Error 0*80070570

[Stephen Hansen]

On Thu, Apr 21, 2016, at 10:46 AM, Allan Leo wrote:
> I need help with this setup error.
> -- Forwarded message --
> From: "Allan Leo" 
> Date: Apr 21, 2016 10:06 AM
> Subject: Re: Error 0*80070570
> To: 
> Cc:
> 
> When running the  setup for your 3.5.1(32-bit version), the setup
> experiences error 0*80070570 and tells me to check the log file. What
> could
> be the problem and whats the solution.
> On Apr 21, 2016 7:05 AM, "Allan Leo"  wrote:
> 
> > When running the setup for your 3.5.1(32-bit version) the setup
> > experiences  error 0*80070570 and tells me to checkout the log file. What
> > could be the problem and whats the resolution.

What version of windows?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running lpr on windows from python

[Stephen Hansen]

On Wed, Apr 20, 2016, at 06:57 AM, loial wrote:
> process = subprocess.Popen(commandline, shell=True,
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> 
> where command line is
> C:/windows/system32/lpr.exe -S 172.28.84.38 -P RAW C:/john/myfile

Try making command line:
commandline = r"C:\windows\system32\lpr.exe -S 172.28.84.38 -P RAW
C:\john\myfile"

The r in front of the string makes it a raw string so you don't have to
double up the slashes.

---
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How much sanity checking is required for function inputs?

[Stephen Hansen]

On Tue, Apr 19, 2016, at 11:09 PM, Ethan Furman wrote:
> On 04/19/2016 10:51 PM, Stephen Hansen wrote:
> > I use 1) more to be less 'nicer' and more, er, 'more specific'. Since I
> > don't like exceptions to rise to the user level where niceness is
> > needed.
> 
> Yeah, that's a better phrasing for (1); I meant more appropriate or 
> informative, such as swapping an internal error (such as KeyError) for a 
> more meaningful FieldNotFound error (or whatever) -- largely library 
> code concerns.

Yeah, and what the OP is doing is going the exact opposite-- from a
more-specific exception (KeyError) to a more generic one (Exception).

To me that's (usually) an anti-pattern. 

--S
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How much sanity checking is required for function inputs?

[Stephen Hansen]

On Sun, Apr 17, 2016, at 12:34 PM, Christopher Reimer wrote:
>  if color not in [VARS['COLOR_BLACK'], VARS['COLOR_WHITE']]:
>  raise Exception("Require \'{}\' or \'{}\' for input value, got 
> \'{}\' instead.".format(VARS['COLOR_BLACK'], VARS['COLOR_WHITE'], color))

Do you think it likely you'll receive a 'color' value not in the list of
black and white? Are you accepting colors from users? If you are, I'd
sanity check *at user input*. Users are crazy, sanitize and check the
heck out of their stuff.

Your code, though? Your time and effort is better spent putting in a
docstring documenting what's valid in color, and if some other coder
puts in red, why, they'll get a KeyError, and it'll point to precisely
what line is wrong, and be able to figure it out.

Unit tests are where you try feeding invalid data into functions and see
how they react-- and the correct reaction to bad data is usually
exceptions. In this case, a KeyError thrown by [VARS['COLOR_BLACK'],
VARS['COLOR_WHITE']][color] is more descriptive then your verbose
Exception.

(What's with the VARS business? Something smells bad there. You're doing
something weird there)

> How much sanity checking is too much in Python?

IMHO, you've got too much.

But that's a fuzzy question, there's no solid and clear answer. Did you
see Ethan's response? I largely agree with his trinity:

On Sun, Apr 17, 2016, at 10:26 PM, Ethan Furman wrote:
> I sanity check for three reasons:
> 
> 1) raise a nicer error message
> 
> 2) keep the point of failure close to the error
> 
> 3) the consequences of bad data are Bad Bad Things (tm)

With a 4)th that exceptions aren't for users, only programmers. But
ultimately, I'd rather a user see an exception if something weird goes
wrong because they can send it to me and I can diagnose it and push an
update. So I also:

4) Attempt to make sure all user errors result in user error messages,
not exceptions.

Note, 1) doesn't mean I always raise a nicer message, it means if
"KeyError" is ambiguious or odd, I raise a better and more informative
one. But you're getting nothing swapping out KeyError for
Exception(lotsofwords). 

I use 1) more to be less 'nicer' and more, er, 'more specific'. Since I
don't like exceptions to rise to the user level where niceness is
needed.

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2.7

[Stephen Hansen]

On Fri, Apr 15, 2016, at 03:48 PM, a3a95797 wrote:
> 
> Sirs(s)
> 
> I wish to have python 2.7 on a computer.  I have not been able to get a
> working copy to work on my machine.  I am prepared to follow instructions
> or to pay someone to install Python on my computer.  Either the Debian or
> the Windows operating system is satisfactory.  I am prepared to pay a
> company or person to reach my goal of programming with Python.

Define "not been ablke to get a working copy on my machine".

For windows, just go to python.org and download it. For debian, it
should come pre-installed.

---
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Guido sees the light: PEP 8 updated

[Stephen Hansen]

On Sat, Apr 16, 2016, at 01:51 AM, Marko Rauhamaa wrote:
> Chris Angelico <ros...@gmail.com>:
> 
> > On Sat, Apr 16, 2016 at 6:06 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> >> It doesn't really matter one way or another. The true WTF is that it's
> >> been changed.
> >
> > Why? Was PEP 8 inscribed on stone tablets carried down from a mountain?
> 
> In a way, yes.
> 
> I don't follow PEP 8 to the tee; probably nobody does. However, I don't
> see the point of turning truckloads of exemplary Python code into
> truckloads of substandard Python code.

This attitude is part of the problem: not following PEP8 does not make
code "substandard". PEP8 was never meant to be an authoritative metric
of 'good'. Its a set of guidelines that are subject to change over time
(this isn't even KINDA the first change!) and represent the core devs
taste and particular needs, and it goes out of its way to say that it is
only a suggestion and other concerns (especially local consistency)
override its advice.

---
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sys.exit(1) vs raise SystemExit vs raise

[Stephen Hansen]

> * You can use named constants from ‘os’ for the purpose of specifying
>   exit status numbers.

Only on *nix. 

Even then it varies from platform to platform which constants you can
use. I'd prefer to document the return status and use numbers/my own
constants directly, that way supporting any platform (even windows,
where its %ERRORLEVEL%

--S
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to XOR a byte output?

[Stephen Hansen]

On Wed, Apr 13, 2016, at 06:51 AM, durgadevi1 wrote:
> I would like to check with you whether using binascii.hexlify() to
> convert the series of bytes into alphabets and integers is correct.

To be clear, they already are integers.The \x notation is how you
naively represent a byte out of the printable range in ASCII. A byte is
a number from 0 to 255, which can also be thought of as 0x00 to 0xFF..
The 'printable range' is those bytes which represent normal characters
instead of control codes and such.

Computers like showing raw byte data in hex \x (which shouldn't be
confused with binascii.hexify) because then each byte concisely fills up
exactly 2 (well, 4, counting the \x) characters, instead of some bytes
being only one character (1), some being two (10), and some being three
(100).

You can see the integer value, consider:

>>> data = b'$//W?\xc0\x829\xa2\xb9\x13\x8c\xd5{\\'
>>> print data[0]
36
>>> print data[10]
19
>>> list(data)
[36, 47, 47, 87, 63, 192, 130, 57, 162, 185, 19, 140, 213, 123, 92]

binascii is almost certainly not what you want: that converts arbitrary
bytes into an ASCII encoded string, at which point its no longer bytes
(and before you did something to it besides displaying it, you'd want to
decode it back to bytes again, probably).

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: hourly weather forecast data

[Stephen Hansen]

On Tue, Apr 12, 2016, at 08:36 PM, kamaraju kusumanchi wrote:
> Is there a way to get hourly weather forecast data (temperature,
> chance of precipitation) from the command line in Debian Linux?

Personally, the last time I wanted to do something like this, I used the
requests library to fetch a RSS feed from Wunderground. But that was
awhile ago and I don't see the obvious RSS links banymore.

Did you see: https://www.wunderground.com/weather/api/d/docs

--
Stephen Hansen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The reason I uninstalled Python 3.5.1 (32-bit) for Windows

[Stephen Hansen]

On Tue, Apr 12, 2016, at 07:57 PM, Jason Honeycutt wrote:
> I am providing feedback as to why I just uninstalled Python. I could not
> use pip. My command line would not recognize pip.exe as a file, even
> though
> I could see the file listed when I type "dir" in the Scripts folder.

If you can't use pip while in the same directory as pip.exe, I don't
even know what is wrong.

That said, you can access pip via 'python -m pip args' instead of using
the pip executable.

--
Stephen Hansen
  m e @ i x o k a i  . i o

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Most probably a stupid question, but I still want to ask

[Stephen Hansen]

On Sun, Apr 10, 2016, at 10:18 PM, Rustom Mody wrote:
> On Monday, April 11, 2016 at 10:17:13 AM UTC+5:30, Stephen Hansen wrote:
> > On Sun, Apr 10, 2016, at 09:03 PM, Fillmore wrote:
> > > and the (almost always to be avoided) use of eval()
> > 
> > FWIW, there's ast.literal_eval which is safe and there's no reason to
> > avoid it.
> 
> Its error reporting is clunky:
> 
> >>> from ast import literal_eval as le
> >>> le("(1)")
> 1
> >>> le("(1,)")
> (1,)
> >>> le("1")
> 1
> >>> le("{1:x}")
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/usr/lib/python2.7/ast.py", line 80, in literal_eval
> return _convert(node_or_string)
>   File "/usr/lib/python2.7/ast.py", line 63, in _convert
> in zip(node.keys, node.values))
>   File "/usr/lib/python2.7/ast.py", line 62, in 
> return dict((_convert(k), _convert(v)) for k, v
>   File "/usr/lib/python2.7/ast.py", line 79, in _convert
> raise ValueError('malformed string')
> ValueError: malformed string

So? Worst case scenario, someone puts invalid data into the file and it
throws an exception. Could it be more specific? Maybe, but I don't see
why it needs to be. If your input isn't reliably formatted, catch
ValueError and log it and fix (either what wrote it or change how you
read it).

--S
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: one-element tuples

[Stephen Hansen]

On Sun, Apr 10, 2016, at 09:43 PM, Fillmore wrote:
> I thought I had made the point clear with the REPL session below. 

Considering how many people expressed repeatedly they didn't know what
was surprising, it wasn't clear at all. 

In general you need to explain these things with your words: code is
good, show code, don't get me wrong, but you need to express your
expectations and how the difference between what happened and what you
expected surprised you.

Both parts, the code and the expression of your thoughts, are really
important to getting help around here :) 

---
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Most probably a stupid question, but I still want to ask

[Stephen Hansen]

On Sun, Apr 10, 2016, at 09:03 PM, Fillmore wrote:
> and the (almost always to be avoided) use of eval()

FWIW, there's ast.literal_eval which is safe and there's no reason to
avoid it. You'll still have to deal with the fact that a single string
on a line will return a string while multiples will return a tuple, but
how you're doing that (checking the type of the result) is fine.

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: one-element tuples

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:48 PM, Fillmore wrote:
> On 04/10/2016 08:31 PM, Ben Finney wrote:
> > Can you describe explicitly what that “discontinuation point” is? I'm
> > not seeing it.
> 
> Here you go:
> 
>  >>> a = '"string1"'

Here, "a" is a string that contains a double quoted string. So if you
evaluate it, you get a string.

>  >>> b = '"string1","string2"'

Here, "b" is a string that contains two double quoted strings separated
by a comma. So if you evaluate it, you get a tuple of two strings.

>  >>> c = '"string1","string2","string3"'

This is as above, but with three items.

With that in mind:
>  >>> ea = eval(a)
>  >>> eb = eval(b)
>  >>> ec = eval(c)
>  >>> type(ea)
>  
>  >>> type(eb)
> 
>  >>> type(ec)
> 

This should all be expected. The commas, when you evaluate them, are in
B making tuples. There's only a single string in A, so you get a
string. If you wanted a one item tuple, you would have written:

>>> a = '"string1",'

Note the trailing comma.

> I can tell you that it exists because it bit me in the butt today...
> 
> and mind you, I am not saying that this is wrong. I'm just saying that it
> surprised me.

If the above doesn't explain it, then I still don't understand what
you're finding surprising and what you'd expect otherwise.

---Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Parens do create a tuple (was: one-element tuples [Was: Most probably a stupid question, but I still want to ask])

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:45 PM, Ben Finney wrote:
> So, let's please stop saying “parens don't create a tuple”. They do, and
> because of that I've stopped saying that false over-simplification.

I stand by "parens don't make a tuple", with the caveat that I should
have mentioned the empty tuple exception that proves the rule. The only
time you need parens is to resolve ambiguity.

To suggest that parens do make tuples confuses the issue, given things
like this:
>>> a = 1,2,3
>>> b = (1, 2, 3)

--
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Most probably a stupid question, but I still want to ask

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:17 PM, Fillmore wrote:
> On 04/10/2016 07:30 PM, Stephen Hansen wrote:
> 
> > There's nothing inconsistent or surprising going on besides you doing
> > something vaguely weird and not really expressing what you find
> > surprising.
> 
> well, I was getting some surprising results for some of my data, so I can
> guarantee that I was surprised!

The point is you are not saying *what* is surprising. Nothing in your
example code looks the least bit surprising to me, but I've been using
Python for ages. If you're surprised by something, say *what* surprises
you at the very least.

But to repeat what I said that you didn't quote: the thing you need to
understand is that parentheses do not create tuples, commas do.

Parentheses only group things together.

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: one-element tuples [Was: Most probably a stupid question, but I still want to ask]

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:22 PM, Fillmore wrote:
> Hold on a sec! it turns up that there is such thing as single-element
> tuples in python:
> 
>  >>> c = ('hello',)
>  >>> c
> ('hello',)
>  >>> c[0]
> 'hello'
>  >>> c[1]
> Traceback (most recent call last):
>File "", line 1, in 
> IndexError: tuple index out of range
>  >>>
> 
> So, my original question makes sense. Why was a discontinuation point
> introduced by the language designer?

What discontinuation point? 

You keep masterfully managing to *not explain what you're asking*. What
is surprising to you? 

--S
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: one-element tuples [Was: Most probably a stupid question, but I still want to ask]

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:18 PM, Stephen Hansen wrote:
> The parens are optional, I always put them in because:
> >>> b = "hello",

Ahem, "because its easy to miss the trailing comma" is what I meant to
say here.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: one-element tuples [Was: Most probably a stupid question, but I still want to ask]

[Stephen Hansen]

On Sun, Apr 10, 2016, at 05:13 PM, Fillmore wrote:
> I guess that the answer to my question is: there is no such thing as a
> one-element tuple,
> and Python will automatically convert a one-element tuple to a string...
> hence the
> behavior I observed is explained...
> 
>  >>> a = ('hello','bonjour')
>  >>> b = ('hello')
>  >>> b
> 'hello'
>  >>> a
> ('hello', 'bonjour')
>  >>>
> 
> 
> Did I get this right this time?

No, you didn't. Your mistake is again -- parens don't make tuples,
commas do.

A one element tuple is:
>>> b = ("hello,)

The parens are optional, I always put them in because:
>>> b = "hello",

The parens group an expression, they don't make a type.

--
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Most probably a stupid question, but I still want to ask

[Stephen Hansen]

On Sun, Apr 10, 2016, at 03:51 PM, Fillmore wrote:
> 
> let's look at this:
> 
> $ python3.4
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> line1 = '"String1" | bla'
>  >>> parts1 = line1.split(" | ")
>  >>> parts1
> ['"String1"', 'bla']
>  >>> tokens1 = eval(parts1[0])
>  >>> tokens1
> 'String1'
>  >>> tokens1[0]
> 'S'
> 
> and now this
> 
>  >>> line2 = '"String1","String2" | bla'
>  >>> parts2 = line2.split(" | ")
>  >>> tokens2 = eval(parts2[0])

I *THINK* what you're asking is why this returns a tuple, where in the
first eval you got a string. The answer is because commas create tuples
(not parens), so:

"String1", "String2"

is a tuple expression. Whereas:

"String1"

is a string expression. 

> the question is: at which point did the language designers decide to
> betray the
> "path of least surprise" principle and create a 'discontinuity' in the
> language?

There's nothing inconsistent or surprising going on besides you doing
something vaguely weird and not really expressing what you find
surprising.

--Stephen
m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode normalisation [was Re: [beginner] What's wrong?]

[Stephen Hansen]

On Sat, Apr 9, 2016, at 12:25 PM, Mark Lawrence via Python-list wrote:
> Again, where is the relevance to Python in this discussion, as we're on 
> the main Python mailing list?  Please can the moderators take this stuff 
> out, it is getting beyond the pale.

You need to come to grip with the fact that python-list is only
moderated in the vaguest sense of the word. 

Quote:

https://www.python.org/community/lists/
"Pretty much anything Python-related is fair game for discussion, and
the group is even fairly tolerant of off-topic digressions; there have
been entertaining discussions of topics such as floating point, good
software design, and other programming languages such as Lisp and
Forth."

If you don't like it, sorry. We all have our burdens to bear.

--S
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange range

[Stephen Hansen]

On Sat, Apr 2, 2016, at 02:40 PM, Marko Rauhamaa wrote:
> That's why I was looking for counterexamples in the standard library

This entire bent of an argument seems flawed to me.

The standard library has never been a beacon for best practices or
idiomatic uses of Python. That a use exists in the standard library, or
that one does not, doesn't really tell you anything meaningful about
Python itself or good practices with the language. The standard library
is under uniquely conservative constraints that enshrine compatibility
and reliability from one point release to another over any kind of
innovation. 

That code exists in the standard library is, itself, an incredibly
strong reason why it should stay as IS: changes for style, idiom, best
practices, modern techniques, those are all valid but *weak* arguments
to change the standard library. 

The stdlib works and its continuing to work tomorrow is its most
important burden. Just look at how much of the stdlib is not PEP8
compliant. Changing it to be PEP8 compliant is seen as a worse thing to
do then the possibility of introducing bugs by doing such a sweeping
change in the interest of good practices and style.

The stdlib exists as a bastion of stability above all else. Its
standards aren't a reason to make a change (or, not to make a change,
either). That doesn't mean its not useful to look at the standard
library, but you should not enshrine it as the example of good or
idiomatic code to measure decisions against. Most code exists outside
the stdlib. 

---
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Array

[Stephen Hansen]

On Wed, Mar 30, 2016, at 10:34 PM, tdspe...@gmail.com wrote:
> as you can see the option element was added third but is the first one
> displayed.
> 
> Is this just standard - I am using Python 3.5

The order of items in dictionaries is based on the hash value -- which
while stable, should be considered random and not reliable. If you want
to maintain insertion order, consider using an OrderedDict:

https://docs.python.org/3.5/library/collections.html#collections.OrderedDict

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python.exe has stopped working

[Stephen Hansen]

On Fri, Jun 5, 2015, at 02:03 AM, Alexis Dubois wrote:
 Anyone else for an idea on that?

Sorry, I have no idea.

Have you tried asking on the PyQT mailing list where there is likely
more of a concentration of PyQT expertise?
http://www.riverbankcomputing.com/mailman/listinfo/pyqt

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trees

[Stephen Hansen]

On Tue, Jan 20, 2015 at 1:45 AM, Marko Rauhamaa ma...@pacujo.net wrote:

 Terry Reedy tjre...@udel.edu:

  Others have answered as to why other special-purpose
  constrained-structure trees have not been added to the stdlib.

 Ordered O(log n) mappings are not special-purpose data structures. I'd
 say strings and floats are much more special-purpose than ordered
 mappings, and yet Python has direct support for those.


Your anecdote is strong, sir.

However, I use strings thousands of times, floats maybe a hundred of times,
and order mappings a few times.

My anecdote counters yours.

A tree structure is special purpose because there is a lot of options with
different characteristics that make certain implementations ideal in some
cases and not in others.

A float is a float, there's a standard (IEEE 754?), its not special at all.

A string, I suppose, could be special, but that's a pretty nonsense view of
the world since what most people use strings commonly.

I'm not arguing against including a tree, but I have no advice on which
one, and the one-- one!-- time I've needed a tree I got one off pypi. Not
everything needs to be in the stdlib.

But to call strings and floats special purpose is really a silly argument
to make.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue4431] Distutils MSVC doesn't create manifest file

[Stephen Hansen]

Stephen Hansen added the comment:

Just to be clear, I ran into this exact issue recently in VS2010 professional 
as I indicated earlier. I don't know about what should or should not be needed, 
but the solution in the original comment fixed it exactly for me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Command prompt not shown when running Python script with subprocess on Windows

[Stephen Hansen]

You need to call python.exe path-to-script.py, I think, not just
path-to-script.py. See sys.executable (though that depends on if you're a
frozen app or not).

I can't be sure though because there's no code. Show code when asking
questions, it helps frame the discussion and get a better answer ;)


On Mon, May 26, 2014 at 5:03 PM,
ps16thypresenceisfullnessof...@gmail.comwrote:

 I have written a Python script with a wxPython GUI that uses
 subprocess.Popen to open a list of files that the user provides. One of my
 users would like to be able to run a Python script with my application. The
 Python script he is trying to run uses the command line and gets keyboard
 input from the user several times.

 The problem is that if the Python script is run on Windows with
 subprocess.Popen, no command prompt is shown (my GUI application is a .pyw
 file). The user's script runs silently but then does not quit because it is
 waiting for input, but there is no way for the input to be given, since
 there is no command prompt visible.

 I think this may be related to the fact that I am calling subprocess.Popen
 with shell=True. I tried calling it with shell=False (the default), but
 then I got an error that the file is not a valid Win32 application.

 I would appreciate any help with this problem.

 -- Timothy
 --
 https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue4431] Distutils MSVC doesn't create manifest file

[Stephen Hansen]

Stephen Hansen added the comment:

FYI. Windows 8.1, Visual Studio 2010 SP1 Pro just installed, Python 3.3.3; a 
random extension did this as a 'test' in its setup.py:

compiler = distutils.ccompiler.new_compiler()
if not compiler.has_function('rand', includes = ['stdlib.h']):
...

And this failed. Further investigation turned brought me here, and adding 
/MANIFEST to my ld_args as this patch does fixed it.

--
nosy: +ixokai

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Both Python 2.5.2 and 2.7.2 flop the same way under Win 7

[Stephen Hansen]

On 11/17/11 8:34 PM, W. eWatson wrote:
 On 11/17/2011 7:04 PM, alex23 wrote:
 On Nov 18, 2:55 am, W. eWatsonwolftra...@invalid.com  wrote:
 Comments?

 Are you using the vanilla installer or ActiveState's ActivePython? I
 find the latter integrates better with Windows.

 Also, out of curiousity, 32 or 64 bit Windows?
 64-bit and plain old python msi installer.

That'd be it, I expect.

Windows has two parallel registry trees; if you launch a 32-bit program,
it sees one.. if you launch a 64-bit program, it sees the other. What
looks like the exact same keys can be different as a result.

The MSI is probably writing the keys into the 32-bit registry, and
Explorer is now a 64-bit application.

You can add the associations for Edit manually. Though not to
idle.pyw, which you keep -- repeatedly -- saying you try and errors.
Of course it errors. That's a python script, not an executable.

Associate to pythonw.exe, and pass it the path to idle.pyw, and then -n
-e %1 -- which will be replaced with the actual filename. But you were
told that already and seem to have just dismissed it out of hand.

Like: PATH\pythonw.exe PATH\idle.pyw -n -e %1

... where PATH is whatever the location of those files are in your
environment.

Yes, its moderately annoying that you have to do this yourself; maybe
you wouldn't if you installed 64-bit python, but I can't be sure. Maybe
it has nothing to do with 32 or 64-bitness at all and my guess is wrong.
Maybe your profile has gone wonky. But it doesn't matter. You can add
the Edit association yourself. Its a one-time fix.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Suggested coding style

[Stephen Hansen]

On 9/30/11 6:54 AM, rantingrick wrote:
 a misplaced and rarely used functionality of the stdlib.

Have you tried putting \.zfill\( and selecting Python in
google.com/codesearch?

It seems to have quite a few users, among only openly available code.
Considering there is a much larger body of code that isn't openly
available, its not hard to extrapolate from the search. (Then again, the
Python Community is made up of only people on this list! Everyone knows
that.)

Sure, you can use format strings, but that is a significantly more
complicated thing to do.

It may not be /common/, but is not at all unusual for one to have in a
string a number that they want to line up numerically with zeros. You
may not have had to do it a lot. But, golly, you are not a
representative sample of the world. Your code, your projects, the things
you have done, are not a representative sample of all the Python code,
projects, and things people have done with Python out there.

The zfill may not be a super wonderful function, used by many in most
places. But its there, its simple, its clear what it does.

Removing it means that anyone who wants to do what it did, now have to
learn a fairly complex mini-language, even if perhaps that is the only
time they will /ever/ need to use said language. That's a burden which
is just, frankly, silly.

Its nice that the format mini-language is powerful. But its nice that
there are also simple, clear, direct primitives people can use to
accomplish simple, fairly common needs.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with simple OOP Python question

[Stephen Hansen]

On 9/4/11 11:47 PM, Kristofer Tengström wrote:
 Hi, I'm having trouble creating objects that in turn can have custom
 objects as variables. The code looks like this:
 
 -
 
 class A:
 sub = dict()

You are sharing this single sub dictionary with all instances of your
A class.

If you want to define instance-specific attributes, define them in the
__init__ method, like so:

class A:
def __init__(self):
self.sub = dict()

def sub_add(self, cls):
obj = cls()
self.sub[obj.id] = obj

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] allow line break at operators

[Stephen Hansen]

On 9/3/11 3:33 AM, Yingjie Lan wrote:
 but at least we can have such 'freedom' :)

Freedom is not and never has been, IMHO, a virtue or goal or even desire
in Python. Where it occurs, it is at best a happy coincidence, and even
if that happy coincidence happens often, it is not a design feature, IMHO.

Simplicity and readability are virtues in Python. Freedom is even
declared a vice to be avoided by the Zen, that holy document which
defines all things Pythonic in clear and unambiguously absolute terms*.

Looking over this whole thread at the various examples -- they add
complication (a vice!). Complication to the parser, complication to the
language itself and worse, understanding of code (as your head has to
parse things too), all for what?

So you don't have to use parens, which quite explicitly (another
virtue!) do the job, to wrap around a long expression? Escaping newlines
is arguably a bit on the ugly side (a vice!), so maybe the proposal has
a little weight there, but since you can just avoid that by using
parens, that's pretty much nullified. (Since it is also a virtue to
offer only the Dutch way of doing things -- at least without hiding the
alternatives in modules with a little bit of shame -- and this is
clearly a case of the Dutch liking limited use of grouping parens).

There just isn't even vaguely enough justification based on
Python-virtues (readability, simplicity, explicitness, things like that)
to even kind of make it worth the complication.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/

* Obvious exaggeration :P



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting Ctrl-Alt-Del in Windows

[Stephen Hansen]

On 9/1/11 8:52 AM, Den wrote:
 Obviously, this is a windows-based question.  I know that Ctrl-Alt-Del
 is handled deep inside the OS, and I'm not trying to interrupt that.
 But is there some way to detect that a C-A-D has been pressed?

IIUC, by definition, Ctrl-Alt-Delete can't be responded to in any way.
Its the entire point of the sequence: when you type it you can be
entirely, totally, 100% certain that what happens next is the true OS
and not any app faking things.

That's why you have to hit CAD to get to the login form in some versions
of Windows. The whole point of that secure sequence is that the OS and
only the OS responds.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Threads in Python

[Stephen Hansen]

On 9/1/11 2:45 PM, George Kovoor wrote:
 Hi,
 Why doesn't python threads show an associated PID?  On spawning python
 threads using the threading module I can only see the main thread's pid on
 using top or ps unix command, no  subprocesses are displayed. In otherwords
 top or ps in not aware of any subprocesses created using threading module in
 python.

 Whereas in Java , creating threads will result in separate pid , these
 subprocesses can be listed using top or ps. Java threads get mapped to the
 cores in the system.

I think you're confused about what threads and subprocesses are. They
are completely different mechanisms for concurrent code. Threads never
show up on top or ps, in any language ... or the language isn't offering
threads. I don't know Java, so I can't really comment on it much, but it
may be misusing the 'thread' word, but I somehow doubt it. I suspect
you're just mistaken about what Java is offering.

Threads are separate operating ..er, chains-of-instructions within a
single process... Notably with threads, they share the same address
space so you can easily share objects amongst threads, without any
copying and with no overhead ... Also notably with threads, this can be
dangerous, so you often end up wrapping lots of locks around those
shared objects and have to take extreme care to make sure nothing goes
haywire.

Subprocesses are different; they are a whole, separate process with its
own address space and no shared memory (unless you go out of your way to
do it manually). Heck, each subprocess can have any number of threads.
Anything you want to share between them you have to take special care to
set up and do -- multiprocessing exists to make this easier and make
subprocesses easier to use, like threads are.

They're very distinct. Threads are a lot more lightweight and start up a
lot faster, but doing multithreaded programming right with any sort of
shared objects is really, really, really hard to get right. Some say you
can't.

But, in Python, only one thread actually ever executes actual Python
code at any given time. This does not actually make threading useless as
some people claim; if you're making a lot of calls into C-code, for
instance, the lock gets released while said C-code runs and other Python
code can continue along. Its just not useful if your program is
CPU-bound and wants to take advantage of multiple cores. But there's
lots of other reasons to go concurrent.

But if you do need lots of CPU power, multiprocessing lets you chew up
multiple cores and does so /fairly/ easily. Communication between the
processes can be expensive depending on the types of objects you need to
pass back and forth, but it depends on how you're designing your app.

They're just different ways of achieving concurrency, and the two
primary ways Python provides. (Greenlets is another, available as a
third-party module; Twisted's asynch dispatching isn't really exactly
concurrency, but it does a better job then concurrency does for some
operations; someone's always working on coroutines in some fashion or
another, which is another kind of concurrency.)

Lots of different ways to go concurrent, depending on your needs.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Web hosting when you need to install your own modules (was Re: Interact with SQL Database using Python 2.4 or lower)

[Stephen Hansen]

On 8/28/11 10:52 PM, Chris Angelico wrote:
 * DNS record changes required a support ticket (this was shared web
 hosting, so I didn't have control over the BIND files - that's what
 they said, anyway)

Ouch: I never let a webhost near my domain names. I was burned somewhere
around that a long time ago, and now always keep the jobs of managing my
DNS record and hosting my sites /totally/ separate.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Interact with SQL Database using Python 2.4 or lower

[Stephen Hansen]

On 8/28/11 9:49 PM, Sascha wrote:
 My Problem: the webhost runs Python 2.4 so I cannot communicate
 with(query or modify) my SQLite3 database. The webhost will not allow
 me to install my own version of python or upload modules unless I
 upgrade to VPS.

Get a new webhost. Seriously. This is a seriously absurd requirement --
it goes past absurd into malicious incompetence, frankly. Not being able
to upload your own modules?

There has to be another option. Personally, I'm a major fan of
Webfaction -- from price to plans to what's supported to actual
effectiveness of their tech support.

But I don't know if they have a warehouse in Australia, if their latency
with any of their various data centers is suitable for you. Maybe, maybe
not -- but there /has/ to be a better option then this site... Good
hosts these days are not all that uncommon and are fairly competitive.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Interact with SQL Database using Python 2.4 or lower

[Stephen Hansen]

On 8/28/11 10:23 PM, Chris Angelico wrote:
 On Mon, Aug 29, 2011 at 3:09 PM, Stephen Hansen
 me+list/pyt...@ixokai.io wrote:
 Get a new webhost. ...

 But I don't know if they have a warehouse in Australia, if their latency
 with any of their various data centers is suitable for you. Maybe, maybe
 not -- but there /has/ to be a better option then this site... Good
 hosts these days are not all that uncommon and are fairly competitive.
 
 Having burnt my fingers with a couple of web hosts, and finally
 decided to host my own web site, I have one major piece of advice
 regarding this:
 
 Get a personal recommendation.

This is good advice, though with prices as they are in many cases --
provided you don't need to start out immediately solid and have some
development wiggle-room -- its not a bad thing to experiment.

Just don't get too tied to a certain host until you feel them out.
Sending them emails with detailed questions before you sign up is a good
thing, for example.

Good hosts will respond with detailed, specific answers, from real
people. Bad hosts will point you to a vague website or stock reply.

Real people, reading your real questions, and answering with real
answers is a major, major sign of the kind of company I want to do
business with. (Bonus points if they respond to complex, technical and
legal questions with specific answers within 24 hours -- bonus++ points
if the non-legal questions usually get responses in 1, at absurd times
even).

 BTW, don't take the fact that I host my own site as a negative
 recommendation for every hosting company out there. My requirements
 are somewhat unusual - I want to host a MUD, not just a web site.
 Hosts that let you do THAT much are usually quite expensive :)

Hehe, I don't want to get overly advertising in my comments (so I'm so
not including a referrer link anywhere), but amusingly enough, my first
Webfaction account was signed up for the MUD reason.

They officially don't give a rats ass what you run in the background,
provided you're just not using more then your RAM allotment and that its
not spiking the CPU to a point that affects the rest of the system.

I have one account that runs a mud, one that does often semi-significant
background processing regularly via cron jobs (which they mailed me
about once when it got out of hand-- but they were entirely professional
and nice about it, and I fixed it with some controls so it behaved in a
more friendly way towards the rest of the system), and one for my
personal site where I run an IRC bouncer on, and all is cool.

(Why three accounts? One is paid for by a client, one half by me, one by
me -- it was just easier, and no way it all would fit under a single plan)

Anyways. I shall not further ramble as a satisfied-customer.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Run time default arguments

[Stephen Hansen]

On 8/25/11 1:54 PM, t...@thsu.org wrote:
 On Aug 25, 10:35 am, Arnaud Delobelle arno...@gmail.com wrote:
 You're close to the usual idiom:

 def doSomething(debug=None):
 if debug is None:
 debug = defaults['debug']
 ...

 Note the use of 'is' rather than '=='
 HTH
 
 Hmm, from what you are saying, it seems like there's no elegant way to
 handle run time defaults for function arguments,

Well, elegance is in the eye of the beholder: and the above idiom is
generally considered elegant in Python, more or less. (The global nature
of 'defaults' being a question)

 meaning that I should
 probably write a sql-esc coalesce function to keep my code cleaner. I
 take it that most people who run into this situation do this?
 
 def coalesce(*args):
   for a in args:
 if a is not None:
   return a
   return None
 
 def doSomething(debug=None):
   debug = coalesce(debug,defaults['debug'])
   # blah blah blah

Er, I'd say that most people don't do that, no. I'd guess that most do
something more along the lines of if debug is None: debug = default as
Arnaud said. Its very common Pythonic code.

In fact, I'm not quite sure what you think you're getting out of that
coalesce function. Return the first argument that is not None, or
return None? That's a kind of odd thing to do, I think. In Python at
least.

Why not just:

debug = defaults.get(debug, None)

(Strictly speaking, providing None to get is not needed, but I always
feel odd leaving it off.)

That's generally how I spell it when I need to do run time defaults.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: is there any principle when writing python function

[Stephen Hansen]

On 8/27/11 3:21 PM, Emile van Sebille wrote:
 On 8/27/2011 2:57 PM Ben Finney said...
 Emile van Sebilleem...@fenx.com  writes:

 Code is first and foremost written to be executed.

 
 
  “Programs must be written for people to read, and only
 incidentally for
  machines to execute.”
  —Abelson  Sussman, _Structure and Interpretation of Computer
 Programs_

 
 That's certainly self-fulfilling -- code that doesn't execute will need
 to be read to be understood, and to be fixed so that it does run. Nobody
 cares about code not intended to be executed.  Pretty it up as much as
 you have free time to do so to enlighten your intended audience.

Er, you're interpreting the quote... way overboard. No one's talking
about code that isn't intended to be executed, I don't think; the quote
includes, and only incidentally for machines to execute. That's still
the there, and its still important. It should just not be the prime
concern while actually writing the code.

The code has to actually do something. If not, obviously you'll have to
change it.

The Pythonic emphasis on doing readable, pretty code isn't JUST about
making code that just looks good; its not merely an aesthetic that the
community endorses.

And although people often tout the very valid reason why readability
counts-- that code is often read more then written, and that coming back
to a chunk of code 6 months later and being able to understand fully
what its doing is very important... that's not the only reason
readability counts.

Readable, pretty, elegantly crafted code is also far more likely to be
*correct* code.

However, this:

 Code that runs from the offset may not ever again need to be read, so
 the only audience will ever be the processor.
 
 I find it much to easy to waste enormous amounts of time prettying up
 code that works.  Pretty it up when it doesn't -- that's the code that
 needs the attention.

... seems to me to be a rather significant self-fulfilling prophecy in
its own right. The chances that the code does what its supposed to do,
accurately, and without any bugs, goes down in my experience quite
significantly the farther away from pretty it is.

If you code some crazy, overly clever, poorly organized, messy chunk of
something that /works/ -- that's fine and dandy. But unless you have
some /seriously/ comprehensive test coverage then the chances that you
can eyeball it and be sure it doesn't have some subtle bugs that will
call you back to fix it later, is pretty low. In my experience.

Its not that pretty code is bug-free, but code which is easily read and
understood is vastly more likely to be functioning correctly and reliably.

Also... it just does not take that much time to make pretty code. It
really doesn't.

The entire idea that its hard, time-consuming, effort-draining or
difficult to make code clean and pretty from the get-go is just wrong.

You don't need to do a major prettying up stage after the fact. Sure,
sometimes refactoring would greatly help a body of code as it evolves,
but you can do that as it becomes beneficial for maintenance reasons and
not just for pretty's sake.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Arrange files according to a text file

[Stephen Hansen]

On 8/27/11 11:06 AM, Emile van Sebille wrote:
 from difflib import SequenceMatcher as SM
 
 def ignore(x):
 return x in ' ,.'
 
 for filename in filenames:
 ratios = [SM(ignore,filename,username).ratio() for username in
 usernames]
 best = max(ratios)
 owner = usernames[ratios.index(best)]
 print filename,:,owner

It amazes me that I can still find a surprising new tool in the stdlib
after all these years.

Neat.

/pinboards

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding .pth in site-packages

[Stephen Hansen]

On 8/27/11 3:41 PM, Josh English wrote:
 I have .egg files in my system path. The Egg file created by my setup script 
 doesn't include anything but the introductory text. If I open other eggs I 
 see the zipped data, but not for my own files.

Sounds like your setup.py isn't actually including your source.
 
 Is having a zipped egg file any faster than a regular package? or does it 
 just prevent people from seeing the code?

IIUC, its nominally very slightly faster to use an egg, because it can
skip a lot of filesystem calls. But I've only heard that and can't
completely confirm it (internal testing at my day job did not
conclusively support this, but our environments are uniquely weird).

But that speed boost (if even true) isn't really the point of
eggs-as-files -- eggs are just easy to deal with as files is all. They
don't prevent people from seeing the code*, they're just regular zip
files and can be unzipped fine.

I almost always install unzip my eggs on a developer machine, because I
inevitably want to go poke inside and see what's actually going on.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/

* Although you can make an egg and then go and remove all the .PY files
from it, and leave just the compiled .PYC files, and Python will load it
fine. At the day job, that's what we do. But, you have to be aware that
this ties the egg to a specific version of Python, and its not difficult
for someone industrious to disassemble and/or decompile the PYC back to
effectively equivalent PY files to edit away if they want.



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows No-Install Distribution?

[Stephen Hansen]

On 8/23/11 8:29 AM, Eric Lemings wrote:
 I would like to create/find a Python 3.x distribution that can be
 redeployed simply by copying a directory of required files; i.e.

Just take the default installer, install it, and then check the Python
directory: does it have the python DLL? If not, go look into the
system32 directory, grab it, drop it in the Python directory. (If you
installed for all-users this will happen,

Now copy/zip/whatever that Python directory to another machine where it
was not installed. It'll work fine.

You'll have to explicitly provide the path to the Python.exe of course;
you can't just double-click on a .py or run 'python blah.py', but if
your shortcuts/whatever all do C:\Where\You\Installed\Python.exe,
everything should just work.

We do that at work and never run into any trouble. (We actually provide
a MSI but only for convenience of customers who want to auto-install via
Group Policy).

In most situations, Python's good at finding itself, i.e. where the
python.exe is actually located -- and it boostraps the location of
everything else based on that.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__

[Stephen Hansen]

 a
Python class to /bypass/ the normal mechanism for normal (non-magic)
attributes.

If you're asking what the normal mechanism is, its broadly:

 - Check to see if the object's base-classes have a descriptor of the
attributes name. If so, call that.
 - Check to see if the object's instance dict has an attribute of the
name. If so, return that.
 - Check to see if the object's base-classes have an attribute of the name.

More or less. I think. I'm probably leaving something out there.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Error when deleting and reimporting subpackages

[Stephen Hansen]

On 8/22/11 11:51 AM, Matthew Brett wrote:
 Hi,
 
 I recently ran into this behavior:
 
 import sys
 import apkg.subpkg
 del sys.modules['apkg']
 import apkg.subpkg as subpkg
 Traceback (most recent call last):
   File stdin, line 1, in module
 AttributeError: 'module' object has no attribute 'subpkg'
 
 where 'apkg' and 'subpkg' comprise empty __init__.py files to simplify the 
 example.
 
 It appears then, that importing a subpackage, then deleting the containing 
 package from sys.modules, orphans the subpackage in an unfixable state. 
 
 I ran into this because the nose testing framework does exactly this kind of 
 thing when loading test modules, causing some very confusing errors and 
 failures.
 
 Is this behavior expected?

Yes. Doing an import of apkg.subpkg results in more then just test1
being cached in sys.modules, and you're removing half of that so leaving
Python in a weird state.

You also want to del sys.modules[apkg.subpkg], then you'll be able to
re-import apkg.subpkg. I.e:

Python 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type help, copyright, credits or license for more information.
 import sys
 import test1.test2
 del sys.modules['test1']
 del sys.modules['test1.test2']
 import test1.test2 as test2


-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list