[issue46003] os.replace is not cross-platform: at least improve documentation

[nobody]

nobody  added the comment:

Thank you again, closing this issue.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46003>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46003] os.replace is not cross-platform: at least improve documentation

[nobody]

nobody  added the comment:

Thank you for your replies. I have been reading more and conclude that I/O 
programming on Windows and Linux are two different things.

Python is not cross-platform in the sense that a Python program always works on 
different operating systems but that you can make it work on different 
operating systems (most of the time).

My application uses the PyPi package cachelib. The file system interface is 
implemented in this file:

https://github.com/pallets/cachelib/blob/main/src/cachelib/file.py

This package uses os.replace and other file I/O methods. 

>From what I understand is that this package must be modified to also handle 
>WinError 5 and Errno 13 if you want to use it on Windows,
meaning more specific exception handling and adding retries and a delay for 
every I/O related function accessing the os.replace file.

Please correct me if I am wrong.

--

___
Python tracker 
<https://bugs.python.org/issue46003>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46003] os.replace is not cross-platform: at least improve documentation

[nobody]

New submission from nobody :

When using threads, os.replace gives no errors on Linux. On Windows however you 
get PermissionErrors like: '[WinError 5] Access is denied' for os.replace and 
'[Errno 13] Permission denied' when reading the os.replace file.
The only way I could get this to work was to add retries with delay.
At least the documentation should be improved.

--
messages: 407864
nosy: hancos
priority: normal
severity: normal
status: open
title: os.replace is not cross-platform: at least improve documentation
type: behavior
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue46003>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

IPython and dumb terminal

[Nobody]

Recent versions of IPython ignore $TERM and blindly assume that you're
using something similar to xterm. Does it have an option to disable this
"feature"?

-- 
https://mail.python.org/mailman/listinfo/python-list

import inspect error

[nobody]

I have following errors running on Ubuntu 18, any insight how to fix it? Thank 
you.

Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import inspect
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.7/inspect.py", line 42, in 
from collections import namedtuple
  File "/usr/lib/python2.7/collections.py", line 22, in 
from keyword import iskeyword as _iskeyword
  File "keyword.py", line 3, in 
from inspect import currentframe, getframeinfo
ImportError: cannot import name currentframe
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Numpy slow at vector cross product?

[Nobody]

On Mon, 21 Nov 2016 14:53:35 +, BartC wrote:

> Also that the critical bits were not implemented in Python?

That is correct. You'll notice that there aren't any loops in numpy.cross.
It's just a wrapper around a bunch of vectorised operations (*, -, []).

If you aren't taking advantage of vectorisation, there's no reason to
expect numpy to be any faster than primitive operations, any more than
you'd expect

(numpy.array([1]) + numpy.array([2])[0]

to be faster than "1+2".

Beyond that, you'd expect a generic function to be at a disadvantage
compared to a function which makes assumptions about its arguments.
Given what it does, I wouldn't expect numpy.cross() to be faster for
individual vectors if it was written in C.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python include non-blocking keyboard input function?

[Nobody]

On Mon, 24 Oct 2016 11:14:05 -0700, jladasky wrote:

> I gather that non-blocking keyboard input functions aren't the easiest
> thing to implement.  They seem to depend on the operating system.

Indeed. It's somewhat harder to implement one on an OS which doesn't take
it for granted that the system actually *has* a keyboard (i.e. Unix).

If you're willing to compromise and accept the use of a terminal rather
than a keyboard, the next question is whether to use the process'
controlling terminal, or the terminal associated with stdin (if both
exist, they're probably the same terminal, but aren't required to be).

Other complications include the fact that, if the process isn't part of
the terminal's foreground process group, attempting to read from the
terminal (even a non-blocking read) will typically suspend the process
(unless you ignore SIGTTIN). And also the fact that the terminal itself
may be line buffered, so the computer has no idea of what's being typed on
it until Return/Enter (or Send, etc) is pressed.

Aside from that, receiving key presses as they are entered means disabling
canonical mode in the tty driver (which buffers input until Return or
Ctrl-D are pressed, so that you can edit the input line with Backspace or
Ctrl-U). That affects all processes using the terminal.

If the current process is in the foreground process group, then processes
in other groups probably won't be reading from the terminal ... at least
until you suspend the forground process group with Ctrl-Z. So you need to
install signal handlers for SIGTSTP and SIGCONT to restore the terminal
settings when the process is suspended. But what should you do about any
existing handlers for those signals?

All things considered, requiring the user to use one of the keys that
generates a signal might be simpler. Or at least not using Esc, which is
about the worst possible choice, given that its normal function is
as a prefix for, well, just about every control sequence (i.e. what is
sent when you press a key which doesn't correspond to a printable
character).

tl;dr: Unix is not MS-DOS.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: try-except with no exceptions

[Nobody]

On Thu, 13 Oct 2016 15:06:25 +0100, Daiyue Weng wrote:

> I know that such try-catch usage is generally a bad practice, since it
> can't locate the root of the exceptions.
> 
> I am wondering how to correct the code above

Either identify the specific exceptions you're expecting, or if you're
interested in "any error", use "except StandardError". That will catch
"errors" but won't catch e.g. KeyboardInterrupt or SystemExit.

Trying to further narrow the set of possible exceptions is often
impossible considering the use of duck typing.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[Nobody]

On Tue, 20 Sep 2016 15:12:39 +0200, Peter Otten wrote:

> because they don't build lists only to throw them away.

The lists could have been avoided by using iterators, e.g.

import itertools as it
keys = xrange(256)
vals = it.imap(chr, keys)
max(it.imap(operator.setitem, it.repeat(d), keys, vals))

> Also, creating a list of dicts or lists is a common gotcha because after
> 
> outer = [[]] * 3
> 
> the outer list contains *the* *same* list three times, not three empty
> lists.

But in this case, it's not a gotcha; it's intentional that each iteration
operates upon the same dictionary.

Essentially, [d]*len(keys) is a trick to get around the fact that map()
requires all of the arguments (apart from the function) to be sequences.

itertools.repeat() is possibly a better trick, although then you can't use
map(), because map() iterates until *all* sequences are exhausted,
appending None values for shorter sequences. itertools.imap() terminates
once the shortest sequence is exhausted.

In this specific case, a loop or comprehension would have been better. But
in situations where you don't control the iteration, the ability to coerce
something into a pre-determined iteration pattern is useful.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Strange behaviour with numbers in exponential notation

[Nobody]

On Fri, 02 Sep 2016 18:18:08 +0200, Christian Gollwitzer wrote:

> 1e26 denotes a *floating point number* Floating point has finite 
> precision, in CPython it is a 64bit IEEE number. The largest exact 
> integer there is 2**53 (~10^16), everything beyond cannot be accurately 
> represented.

Uh, that's wrong. All integers smaller than 2**53 can be represented
exactly. Some, but not all, of the integers above 2**53 can be represented
exactly.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing down in code rather than at top of file.

[Nobody]

On Tue, 30 Aug 2016 04:15:05 +1000, Chris Angelico wrote:

> Don't imagine; test.

Testing alone isn't really good enough. There may be perfectly valid
reasons to avoid the import which won't show up in anything less than the
most thorough testing imaginable.

Personally I wouldn't defer an import just because it might be slow,
particularly if the criterion affecting whether the import actually occurs
is "probabilistic".

[If it might be slow because the module is doing stuff it really
shouldn't, like executing external commands during import, I'd probably
just avoid the package altogether.]

If the import /might/ happen as a result of normal usage, I'd want any
failure to occur when the parent module is imported, not at some rather
arbitrary later point.

If I defer imports, it's because they're only used for features which are
in some sense optional. And the documentation will state the conditions
under which the import is performed (if it's for a specific method, it
will be in the method's docstring).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ctypes Usage Note

[Nobody]

On Mon, 01 Aug 2016 18:41:53 -0700, Lawrence D’Oliveiro wrote:

> Sometimes people load a library with ctypes like this:
> 
> libc = ctypes.cdll.LoadLibrary("libc.so")

That specific example is unlikely to work on any modern Linux system, as
libc.so is typically a linker script rather than a symlink to a DSO.
Likewise for libm and libpthread.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ImportError: Import by filename is not supported when unpickleing

[Nobody]

On Wed, 27 Jul 2016 22:47:15 -0400, Larry Martell wrote:

> Also let me add that initially I was calling Popen with shell=False and
> the arguments in a list, and that was failing with:
> 
> arg 2 must contain only strings

That indicates that you're calling Popen() incorrectly.

> And when I debugged I found that this was happening in the execvp call in
> subprocess and arg 2 was a str. So then I changed it using shell=False
> with the command line in a string, and that is when I get the  ImportError
> error.

This is just hiding the error. Passing the command line as a string will
result in the string being parsed in some way (the details depend upon the
OS). Unless the string was assembled with the correct quoting, this will
corrupt the string.

I'd suggest starting off by ensuring that the string received by the child
process is the one which you intended to pass. Calculate and display (or
log) its hash (see the hashlib module) in both the sender and receiver and
confirm that they're the same.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reshape with xyz ordering

[Nobody]

On Tue, 26 Jul 2016 07:10:18 -0700, Heli wrote:

> I sort a file with 4 columns (x,y,z, somevalue) and I sort it using
> numpy.lexsort.
> 
> ind=np.lexsort((val,z,y,x))
> 
> myval=val[ind]
> 
> myval is a 1d numpy array sorted by x,then y, then z and finally val.
> 
> how can I reshape correctly myval so that I get a 3d numpy array
> maintaining the xyz ordering of the data?

Is it guaranteed that the data actually *is* a 3-D array that's been
converted to a list of x,y,z,val tuples?

In other words, does every possible combination of x,y,z for 0<=x<=max(x),
0<=y<=max(y), 0<=z<=max(z) occur exactly once?

If so, then see Peter's answer. If not, then how do you wish to handle
a) (x,y,z) tuples which never occur (missing values), and
b) (x,y,z) tuples which occur more than once?

If the data "should" to be a 3-D array but you first wish to ensure that
it actually is, you can use e.g.

nx,ny,nz = max(x)+1,max(y)+1,max(z)+1
if val.shape != (nx*ny*nz,):
raise ValueError
i = (x*ny+y)*nz+z
found = np.zeros(val.shape, dtype=bool)
found[i] = True
if not np.all(found):
raise ValueError
ind = np.lexsort((val,z,y,x))
myval = val[ind].reshape((nx,ny,nz))

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: math.frexp

[Nobody]

On Fri, 15 Jul 2016 21:39:31 +1000, Steven D'Aprano wrote:

> prod *= (m1*m2)

Should be:

prod = m1*m2

or:
prod *= m1

(in the latter case, there's no point in decomposing prod).

Of course, if the result overflows, it's going to overflow whether you use
the naive approach or frexp(); in the latter case, it's the 2.0**scale
which will overflow.

One advantage of processing the scale separately is that you can use e.g.

return ((int(prod * 2**53) * (2**(scale-53))) if scale >= 53
else prod * 2**scale)

which will return a (long) integer if the exponent is such that the
fractional bits are lost.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting number of neighbours for a 3d numpy arrays

[Nobody]

Some common ways to handle the boundary condition:

1. Generate clamped indices, test for validity and substitute invalid
entries with an "identity" element. E.g.

ijk = np.mgrid[:a,:b,:c]
i,j,k = ijk
i0,j0,k0 = np.maximum(0,ijk-1)
i1,j1,k1 = np.minimum(np.array(a,b,c).T-1,ijk+1)

n1 = (i>0  ) & myarray[i0,j,k]
n2 = (i0  ) & myarray[i,j0,k]
n4 = (j0  ) & myarray[i,j,k0]
n6 = (k0  ) & np.roll(myarray, 1,axis=0)
n2 = (i0  ) & np.roll(myarray, 1,axis=1)
n4 = (j0  ) & np.roll(myarray, 1,axis=2)
n6 = (k

Re: Compression of random binary data

[Nobody]

On Mon, 11 Jul 2016 10:52:08 -0700, jonas.thornvall wrote:

> What kind of statistic law or mathematical conjecture  or is it even a
> physical law is violated by compression of random binary data?

You can't create an invertable mapping between a set with 2^N elements
(e.g. the set of all N-bit binary sequences) and any set with fewer than
2^N elements (e.g. the set of all M-bit binary sequences for M

Re: Don't put your software in the public domain

[Nobody]

On Sat, 04 Jun 2016 12:28:33 +1000, Steven D'Aprano wrote:

>> OTOH, a Free software licence is unilateral; the author grants the user
>> certain rights, with the user providing nothing in return.
> 
> That's not the case with the GPL.
> 
> The GPL requires the user (not the end-user, who merely avails themselves
> of their common law right to run the software, but the developer user, who
> copies, distributes and modifies the code) to do certain things in return
> for the right to copy, distribute and modify the code:

The GPL places limitations on the granted licence. That isn't the same
thing as requiring the distributor to do something "in return".

This is why the (relatively few) cases where GPL infringements have
resulted in litigation, the legal basis of the litigation is copyright
infringement, not breach of contract.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Don't put your software in the public domain

[Nobody]

On Fri, 03 Jun 2016 09:15:55 -0700, Lawrence D’Oliveiro wrote:

>> [quoted text muted]
> 
> A licence is quite different from a contract. A contract requires some
> indication of explicit agreement by both parties, a licence does not.

More precisely, it requires "mutual consideration", i.e. each party must
provide something of value to the other. If a party doesn't provide
something of value, they can't claim any harm in the event of a breach, as
they haven't lost anything (failure to receive what the other party
promised doesn't count, as it didn't belong to the recipient to start with).

This is why you sometimes see contracts where one party pays a nominal sum
(e.g. one pound/dollar/euro) in return for assets which may have
significant value but also significant liabilities attached. The fact that
they paid /something/ allows them to enforce the contract.

OTOH, a Free software licence is unilateral; the author grants the user
certain rights, with the user providing nothing in return.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: setrecursionlimit

[Nobody]

On Wed, 18 May 2016 09:19:25 -0700, Ned Batchelder wrote:

> Is there a way to know how large the C stack can grow,

Yes. For the main thread, getrlimit(RLIMIT_STACK). For other threads,
pthread_attr_getstacksize().

> and how much it will grow for each Python function call?

No.

Depending upon the interpreter implementation, the stack might not grow
at all in the case where a function written in Python calls another
function written in Python. Calling out into native code (which may then
call back into Python code) is bound to grow the thread by some amount.

While the interpreter could keep track of how much space is left on the
stack, there's no way of knowing in advance how much any given C function
will need. If there isn't enough, there is no robust way to recover from
the resulting segfault.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Remove directory tree without following symlinks

[Nobody]

On Sat, 23 Apr 2016 00:56:33 +1000, Steven D'Aprano wrote:

> I want to remove a directory, including all files and subdirectories under
> it, but without following symlinks. I want the symlinks to be deleted, not
> the files pointed to by those symlinks.

Note that this is non-trivial to do securely, i.e. where an adversary has
write permission on any of the directories involved. Due to the potential
for race conditions between checking whether a name refers to a directory
and recursing into it, the process can be tricked into deleting any
directory tree for which it has the appropriate permissions.

The solution requires:

1. That you always chdir() into each directory and remove entries using
their plain filename, rather than trying to remove entries from a
higher-level directory using a relative path.

2. When chdir()ing into each subdirectory, you need to do e.g.:

st1 = os.stat(".")
os.chdir(subdir)
st2 = os.stat("..")
if st1.st_dev != st2.st_dev or st1.st_ino != st2.st_ino:
raise SomeKindOfException()

If the test fails, it means that the directory you just chdir()d into
isn't actually a subdirectory of the one you just left, e.g. because the
directory entry was replaced between checking it and chdir()ing into it.

On Linux, an alternative is to use fchdir() rather than chdir(), which
changes to a directory specified by an open file descriptor for that
directory rather than by name. Provided that the directory was open()ed
without any race condition (e.g. using O_NOFOLLOW), subsequent fstat() and
fchdir() calls are guaranteed to use the same directory regardless of any
filesystem changes.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detecting repeated subsequences of identical items

[Nobody]

On Thu, 21 Apr 2016 18:05:40 +1000, Steven D'Aprano wrote:

> The specific problem I am trying to solve is that I have a sequence of 
> strings (in this case, error messages from a Python traceback) and I'm 
> looking for repeated groups that may indicate mutually recursive calls. E.g. 
> suppose I have a function f which calls g, and g calls h, and h calls f 
> again, and there's an exception, you will see a traceback in part:

This is a specific case of finding cycles in a directed graph. But
treating it as such probably isn't useful here, because you're interested
in a specific traversal of that graph rather than the graph itself.

One way to approach it is:

sofar = []
for line in traceback:
if line in sofar:
j = sofar.index(line)
if sofar[:j] == sofar[j:j*2]:
# found repeat
sofar = [line] + sofar

Note that sofar needs to be in reverse order, because list doesn't have
.rindex() or .rfind().

Detecting nested cycles is somewhat harder because given e.g.

ababxabababababxababab

you'd want the five repeats of ab in the middle to be treated as two
repeats of ab followed by three repeats of ab, but there's no way to
spot that until later.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Nobody]

On Sat, 26 Mar 2016 23:30:30 +, John Pote wrote:

> So I have sympathy with the OP, I would expect the compiler to pick this
> up

Why? The code is valid, the compiler knows how to generate the
appropriate bytecode for it.

The compiler isn't "lint".

Reporting code which is actually invalid is fairly straightforward.
When the parser attempts to match the next token against a parse rule
and finds that nothing matches (e.g. the "fnc next" example), it just
needs to raise a SyntaxError. The point at which the exception needs to be
raised naturally exists in the code.

But to identify code which is perfectly valid yet is "probably" a mistake
first requires someone to identify such cases, then someone needs to start
adding the appropriate tests to the compiler to distinguish such code from
the rest.

> It would be all to easy to write a series of lines just calling
> functions and forget the () on one of them. Not fun programming. It's
> also a good reminder that the meaning of a keyword in language A is not
> necessarily the same in language B (ie 'next', Python)

"next" isn't a keyword, it's a built-in function. It's perfectly valid to
re-use that name for your own variables or functions.

> So on this last point is this behaviour of Python defined somewhere in
> the docs? 

What behaviour? Evaluating a name?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy arrays

[Nobody]

> What you want is called *transposing* the array:
> 
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html
> 
> That should be a sufficiently fast operation.

Transposing itself is fast, as it just swaps the strides and dimensions
without touching the data (i.e. it returns a new view of the original
array), but subsequent operations may be slower as the data is no longer
contiguous (i.e. iterating over the flattened array in order won't result
in sequential memory access).

If that's an issue, you can use numpy.ascontiguousarray() to make a
contiguous copy of the data.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Make a unique filesystem path, without creating the file

[Nobody]

On Mon, 15 Feb 2016 15:28:27 +1100, Ben Finney wrote:

> The behaviour is already implemented in the standard library. What I'm
> looking for is a way to use it (not re-implement it) that is public API
> and isn't scolded by the library documentation.

So, basically you want (essentially) the exact behaviour of
tempfile.mktemp(), except without any mention of the (genuine) risks that
such a function presents?

I suspect that you'll have to settle for either a) using that function and
simply documenting the reasons why it isn't an issue in this particular
case, or b) re-implementing it (so that you can choose to avoid mentioning
the issue in its documentation).

At the outside, you *might* have a third option: c) persuade the
maintainers to tweak the documentation to further clarify that the risk
arises from creating a file with the returned name, not from simply
calling the function. But actually it's already fairly clear if you
actually read it.

If it's the bold-face "Warning:" and the red background that you don't
like, I wouldn't expect those to go away either for mktemp() or for any
other function with similar behaviour (i.e. something which someone
*might* try to use to actually create temporary files). The simple fact
that it might get used that way is enough to warrant a prominent warning.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 3.4, os.walk does not walk into cdroms

[Nobody]

On Thu, 17 Dec 2015 14:03:25 +0100, Siegfried Kaiser wrote:

>  I have a problem with os.walk - it does not walk into a mounted cdrom, I
>  do not see the cdrom in the walk at all.
> What can I do to walk into cdrom?

1. Are you sure that the directory tree contains the actual mount point,
not just a symlink to it? os.walk() doesn't follow symlinks unless
followlinks=True is given explicitly.

2. Have you tried using the onerror= parameter to report errors? The
default is to silently ignore errors from os.listdir().

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why my image is in bad quality ?

[Nobody]

On Wed, 16 Dec 2015 06:04:37 -0800, fsn761304 wrote:

> pixbufObj = Gdk.pixbuf_get_from_window(window, x, y, width, height) ...
> image = Image.frombuffer("RGB", (width, height),
>  pixbufObj.get_pixels(), 'raw', 'RGB', 0, 1)

The second-to-last argument should probably be Gdk.pixbuf_get_rowstride()
rather than 0.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What does a list comprehension do (was: Late-binding of function defaults (was Re: What is a function parameter =[] for?))

[Nobody]

On Wed, 25 Nov 2015 14:51:23 +0100, Antoon Pardon wrote:

> Am I missing something?

The issue is with lambdas rather than with list comprehensions per se.

Python's lambdas capture free variables by reference, not value.

> x = 3
> f = lambda y: x + y
> f(0)
3
> x = 7
> f(0)
7

The same issue applies to nested functions:

> def foo():
=x = 3
=def f(y):
=   return x + y
=print f(0)
=x = 7
=print f(0)
= 
> foo()
3
7

And also to non-nested functions (but most people expect that):

> x = 3
> def f(y,x=x):
=   return x + y
=
> print f(0)
3
> x=7
> print f(0)
3

If you want to capture a variable by value, add a parameter with a default
value using that variable:

> def foo():
=x = 3
=def f(y, x=x):
=   return x + y
=print f(0)
=x = 7
=print f(0)
= 
> foo()
3
3

This also works for lambdas:

> x = 3
> f = lambda y,x=x: x + y
> f(0)
3
> x = 7
> f(0)
3

Returning to the original expression:

> q = [lambda x: i * x for i in range(4)]
> q[0](1), q[3](1)
(3, 3)
> q = [lambda x,i=i: i * x for i in range(4)]
> q[0](1), q[3](1)
(0, 3)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Regular expressions

[Nobody]

On Wed, 04 Nov 2015 14:23:04 +1100, Steven D'Aprano wrote:

>> Its very name indicates that its default mode most certainly is regular
>> expressions.
> 
> I don't even know what grep stands for.

>From the ed command "g /re/p" (where "re" is a placeholder for an
arbitrary regular expression). Tests all lines ("g" for global) against
the specified regexp and prints ("p") any which match.

> But I think what Michael may mean is that if you "grep foo", no regex
> magic takes place since "foo" contains no metacharacters.

At least the GNU version will treat the input as a regexp regardless of
whether it contains only literal characters. I.e. "grep foo" and
"grep [f][o][o]" will both construct the same state machine then process
the input with it.

You need to actually use -F to change the matching algorithm.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unbuffered stderr in Python 3

[Nobody]

On Mon, 02 Nov 2015 18:52:55 +1100, Steven D'Aprano wrote:

> In Python 2, stderr is unbuffered.
> 
> In most other environments (the shell, C...) stderr is unbuffered.
> 
> It is usually considered a bad, bad thing for stderr to be buffered. What
> happens if your application is killed before the buffer fills up? The
> errors in the buffer will be lost.
> 
> So how come Python 3 has line buffered stderr? And more importantly, how
> can I turn buffering off?

It's probably related to the fact that std{in,out,err} are Unicode
streams. 

> type(sys.stderr)

> type(sys.stderr.buffer)

> type(sys.stderr.buffer.raw)


It appears that you can turn it off with:

sys.stderr = io.TextIOWrapper(sys.stderr.buffer.raw)
or:
sys.stderr = io.TextIOWrapper(sys.stderr.detach().detach())

This results in a sys.stderr which appears to work and whose
.line_buffering property is False.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Postscript to pdf

[Nobody]

On Sun, 20 Sep 2015 23:11:20 +0200, Baladjy KICHENASSAMY wrote:

> i tried this
> 
> def save():
>  Canevas.update()
>  Canevas.postscript(file=tkFileDialog.asksaveasfilename(),
> colormode='color')
>  subprocess.call(["ps2pdf", "-dEPSCrop", "test.ps", "test.pdf"])
> 
> 
> i got the ps file but i didn't get the pdf file :/

Check that subprocess.call() returns zero, or use subprocess.check_output()
instead. Also, if this is a GUI program and you have no easy way to check
what is written to stdout or stderr, try:

p = subprocess.Popen(["ps2pdf", "-dEPSCrop", "test.ps", "test.pdf"],
 stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode != 0:
raise RuntimeError(err)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Putting the main program in a main function

[Nobody]

On Mon, 14 Sep 2015 09:13:47 +0200, ast wrote:

> is it advised to always write programs like that ?

If global (module-scope) variables are initialised by main, then those
variables won't exist unless main() is run, which means that you can't use
it as a module, only as a script.

IMHO, global variables whose initial values can be evaluated without
depending upon or modifying external state should be initialised at the
top level.

If a module has variables which cannot be so initialised, the module needs
to provide an initialisation function which must be called explicitly by
any program or module which imports it.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subdividing a rectangle using numpy

[Nobody]

On Thu, 10 Sep 2015 22:05:07 -0500, Seb wrote:

> The key to my solution was to use numpy's meshgrid to generate the
> coordinates for defining the sub-units.  However, it seems awfully
> complex and contrived,

Half a dozen lines of code is "complex and contrived"?

Also, you should lose marks for having those for loops. Hint:

corners = np.array([[0,0],[0,1],[1,1],[1,0],[0,0]])

> or perhaps some package offers this functionality. 

People don't write packages for such trivial tasks.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: -2146826246 in win32com.client for empty #N/A cell in Excel

[Nobody]

On Sun, 16 Aug 2015 09:53:32 -0700, Sven Boden wrote:

 Anyone knows how to handle a #N/A cell in Excel in the proper way?

0x800A07FA is how xlErrNA (error 2042) is marshalled. This isn't specific
to Python; you'll get the same value using e.g C# or VB.NET.

There's a fairly thorough article on this topic at:

https://xldennis.wordpress.com/2006/11/22/
https://xldennis.wordpress.com/2006/11/29/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: So what's happening here?

[Nobody]

On Fri, 05 Jun 2015 13:11:13 +, Paul Appleby wrote:

 (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
 were the same element via different views, but the id's seem to change
 according to rules that I can't fathom.)

First, a[1] and b[1] aren't views, they're scalars.

Second, different views on the same data are different objects, they just
share the same underlying data. Consider the case where the slice doesn't
cover the entire range:

 a = np.array([1,2,3])
 b = a[:2]
 a
array([1, 2, 3])
 b
array([1, 2])
 id(a)
139682716078288
 id(b)
139682716078368
 b[0] = 99
 a
array([99,  2,  3])
 b
array([99,  2])

The case where a slice *does* cover the entire range isn't special; the
resulting view is still a different object.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help with pipes, buffering and pseudoterminals

[Nobody]

On Sun, 05 Apr 2015 12:20:48 -0700, Daniel Ellis wrote:

 This only seems to print from the parent process.  I read that I need to
 do the os.read call for the fork to happen.  I've also tried printing
 *after* the os.read call.

The child process has its std{in,out,err} attached to the newly-created
pty, so that's where the output from the child's print goes.

You'll see that output if the parent prints the string returned from the
os.read() call.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subprocess and stdin.write(), stdout.read()

[Nobody]

On Tue, 24 Mar 2015 12:08:24 -0700, Tobiah wrote:

 But if I want to send a string to stdin, how can I do that without
 stdin.write()?

p.communicate(string)

 This seems to work:

Only because the amounts of data involved are small enough to avoid
deadlock.

If both sides write more data in one go than will fit into a pipe buffer,
you will get deadlock. The parent will be blocked waiting for the child to
consume the input, which doesn't happen because the child will be blocked
waiting for the parent to consume its output, which doesn't happen because
he parent will be blocked waiting for the child to consume the input, ...

That's a textbook example of deadlock: each side waiting forever for the
other side to make the first move.

This is exactly why POSIX' popen() function lets you either write to stdin
(mode==w) or read from stdout (mode==r) but not both.

 Will this always avoid the deadlock problem?

No.

 This also works:

Again, only because the amounts of data involved are small enough to avoid
deadlock.

 Is that vulnerable to deadlock?

Yes.

 Is there a better way to write to and read from the same process?

Use threads; one for each descriptor (stdin, stdout, stderr).

Non-blocking I/O is an alternative (and that's what .communicate() uses on
Unix), but threads will work on all common desktop and server platforms.

If you need to support platforms which lack threads, either

a) have the parent first write to a file (instead of .stdin), then have
the child read from the file while the parent reads .stdout and .stderr,
or

b) have the parent write to .stdin while the child writes its
stdout/stderr to files (or a file). Once the child completes, have
the parent read the file(s).

Using files allows for potentially gigabytes of data to be buffered. With
pipes, the amount may be as low as 512 bytes (the minimum value allowed by
POSIX) and will rarely be much more (a typical value is 4096 bytes, i.e.
one page on x86).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a cairo like surface for the screen without the window hassle

[Nobody]

On Mon, 02 Feb 2015 14:20:56 +0100, Antoon Pardon wrote:

 I need to have a program construct a number of designs. Of course I can
 directly use a pfd surface and later use a pdf viewer to check. But that
 becomes rather cumbersome fast. But if I use a cairo-surface for on the
 screen I suddenly have to cope with expose events and all such things I
 am not really interested in.

If you're writing a GUI application, something has to deal with those
events regardless of which rendering API you use.

 So does someone know of a package that provides a cairo like surface but
 that would take care of the events in a rather straight forward matter,
 so that my program could make it's design in a window on the screen just
 as if it is designing it in a pdf file.

What exactly is the issue here? That you want to be able to construct a
graphical representation then discard the data used to construct it,
rather than having to keep it around in order to handle subsequent expose
events?

If that's the case, the simplest solution is probably to render to an
image surface then handle expose events by drawing the image onto the
screen (or handing the image to some kind of widget which does this for
you).

Or, if you want to support dynamic scaling, you can render to an SVG
surface instead, but widgets which can display SVG aren't as common as
those which deal with raster formats.

Another solution is to render to a recording surface. This can then be
used as a source surface, so handling expose events boils down to a single
cairo_paint() operation with the recording surface as the source surface
and the window as the destination surface. Maybe there's even a widget
somewhere which does this, but I don't know of one.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can numpy do better than this?

[Nobody]

On Thu, 08 Jan 2015 09:56:50 -0800, Rustom Mody wrote:

 Given a matrix I want to shift the 1st column 0 (ie leave as is) 2nd by
 one place, 3rd by 2 places etc.
 
 This code works.
 But I wonder if numpy can do it shorter and simpler.

def shiftcols(mat):
iy,ix = np.indices(mat.shape)
return mat[(iy+ix)%mat.shape[0],ix]

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Neat little programming puzzle

[Nobody]

On Tue, 16 Dec 2014 00:46:16 -0500, Jason Swails wrote:

 I liked this problem because naive solutions scale as O(2^N), begging for
 a more efficient approach.

Project Euler has this one, twice; problems 18 and 67.

The difference between the two is that problem 18 has 15 rows while
problem 67 has 100 rows. Problem 18 can realistically be solved by a
brute-force approach, problem 67 can't.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: why can't download file from linux server into local window disk c:

[Nobody]

  To: pengsir
On Tue, 09 Dec 2014 00:14:15 -0800, pengsir wrote:

 localpath = 'c:'
 sftp.get(filepath, localpath)

  with open(localpath, 'wb') as fl:
 PermissionError: [Errno 13] Permission denied: 'c:'

It's trying to open c:, which is a drive, as if it was a file.

You have to specify the destination filename, not just the directory.

Also, you probably shouldn't be trying to write to the root directory of
the C drive. You should probably be using a directory beneath either
%USERPROFILE% or %ALLUSERSPROFILE%.

Writing to the root of the system drive tends to require Administrator
privileges. Even if the current user is an administrator, the process must
have elevated privilege (e.g. via Run as Administrator or an explicit
privilege-elevation request from within the code).

--- SoupGate-Win32 v1.05
 * Origin: SpaceSST.BBS.Fidonetnntp.gatew...@.piz.noip.me (1:249/999)
--- Synchronet 3.15b-Win32 NewsLink 1.92
SpaceSST BBS Usenet  Fidonet Gateway
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: why can't download file from linux server into local window disk c:?

[Nobody]

On Tue, 09 Dec 2014 00:14:15 -0800, pengsir wrote:

 localpath = 'c:'
 sftp.get(filepath, localpath)

  with open(localpath, 'wb') as fl:
 PermissionError: [Errno 13] Permission denied: 'c:'

It's trying to open c:, which is a drive, as if it was a file.

You have to specify the destination filename, not just the directory.

Also, you probably shouldn't be trying to write to the root directory of
the C drive. You should probably be using a directory beneath either
%USERPROFILE% or %ALLUSERSPROFILE%.

Writing to the root of the system drive tends to require Administrator
privileges. Even if the current user is an administrator, the process must
have elevated privilege (e.g. via Run as Administrator or an explicit
privilege-elevation request from within the code).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: time.monotonic() roll over

[Nobody]

On Thu, 04 Dec 2014 16:25:44 +0100, ast wrote:

 There is no roll over problem with time.time() since the very
 first one in planned far in the future, but time.time() can go
 backward when a date update throught NTP server is done.

 time.monotonic() is monotonic but roll over often (every 49.7 
 days)
 
 So what to do ? 
 
 Using time.monotonic() and take some actions if a roll over 
 is detected ?

One possibility: use both and monitor the skew (the difference between the
two clocks).

The first time you read them, calculate and store the epoch for
time.monotic(). On subsequent calls, repeat the calculation and check that
the epoch is approximately the same. If it isn't, the difference between
the actual and expected values should be close to an exact multiple of
2**32 milliseconds, which tells you how many times time.monotonic() has
rolled over.

Adjustments to the clock on which time.time() is based shouldn't exceed a
few seconds per year, so there's not much risk that you won't be able to
figure out the correct adjustment.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cherrypy - prevent browser prefetch?

[Nobody]

On Tue, 02 Dec 2014 21:41:33 +, John Gordon wrote:

 GET shouldn't cause any business data modifications, but I thought it was
 allowed for things like logging out of your session.

GET isn't supposed to have observable side-effects. Observable excludes
things like logs and statistics, but getting logged out of your session is
definitely observable.

X-moz: prefetch, X-Purpose etc (note that these all have X- prefixes,
meaning that they're not in any standard) exist because web developers
seem to be uniquely bad at distinguishing between specified behaviour
and seems to work.

But realistically, this horse is not only out of the barn but half way
around the world by now. So many sites misuse GET that there has to be
workarounds for it (to be honest, I'm surprised that they haven't made it
into a standard yet).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cherrypy - prevent browser prefetch?

[Nobody]

On Mon, 01 Dec 2014 11:28:42 -0900, Israel Brewster wrote:

 I'm running to a problem, specifically from
 Safari on the Mac, where I start to type a URL, and Safari auto-fills the
 rest of a random URL matching what I started to type, and simultaneously
 sends a request for that URL to my server, occasionally causing unwanted
 effects.

A GET request should not cause *any* effects. That's what PUT/POST are
for.

GET is for retrieval, not modification.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fileno() not supported in Python 3.1

[Nobody]

On Thu, 13 Nov 2014 15:48:32 -0800, satishmlmlml wrote:

 import sys 
 for stream in (sys.stdin, sys.stdout, sys.stderr): 
print(stream.fileno()) 
 
 
 io.UnsupportedOperation: fileno 
 
 Is there a workaround?

Try:
sys.stdin.buffer.fileno()

or maybe

sys.stdin.buffer.raw.fileno()

In Python 3.x, sys.stdin isn't actually a file, it's a TextIOWrapper
around a BufferedReader around a file (io.FileIO).

TextIOWrapper is responsible for converting a stream of bytes to a stream
of (Unicode) characters. BufferedReader is responsible for buffering (like
C stdio).


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: accents in windows

[Nobody]

On Thu, 30 Oct 2014 08:30:49 -0400, C@rlos wrote:

 thanks U, but the real problem is:
 
 i have a path C:\Users\yanet\Desktop\áaaéeeíiiióooúuuñnn this path
 is correct, áaaéeeíiiióooúuuñnn is the name of a directory but when
 i try to use os.walk() usin this path, dont work, for os this path dont
 exist, i try every things but nothing works.

http://www.joelonsoftware.com/articles/Unicode.html

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Status of side-effecting functions in python

[Nobody]

On Mon, 27 Oct 2014 17:14:58 +0200, Marko Rauhamaa wrote:

 In POSIX, a write(2) system call on file blocks until all bytes have been
 passed on to the file system. The only exception (no pun intended) I know
 is the reception of a signal.

Writing to a file (or block device) will return a short count in the event
that it results in the size of the file exceeding

* the space available on the partition,
* the user's quota,
* the process' file size limit (RLIMIT_FSIZE), or
* any implementation limit on the maximum size of a file,

and at least one byte can be written. This behaviour is mandated by POSIX.

This is different to writing to a socket, pipe or character device,
where a short count is considered an entirely normal result, and
a subsequent write for the remaining bytes will often succeed.

 Even then, I'm not sure Linux file systems ever cut writes short because
 of signals.

Linux never interrupts I/O on discs or block devices due to signals.
These are restarted regardless of whether the signal is set for
automatic restarting (SA_RESTART flag).

 I think the lack of nonblocking file access in Linux is one of the OS's
 main shortcomings. 

It doesn't really matter. In the absence of an explicit mlock() or
mlockall(), the actual code which would be controlling the access is
demand-paged from disc, as is the memory to/from which the data is
transferred (along with the memory which would hold the return code from
read() or write(), for that matter).

Asynchronous I/O in the sense of select(), poll(), O_NONBLOCK etc is meant
for situations where delays could be indefinite, e.g. network connections
or terminals. For short delays (i.e. disc access), there's not much
point having a mechanism so that you can avoid blocking while the data is
read from disc just so that you can block while the code in the else
branch is read from disc.

If you want the program to be able to do something else while waiting for
I/O, use threads. The introduction of threads made most concurrency-
related issues in the POSIX API moot.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Flush stdin

[Nobody]

On Sat, 18 Oct 2014 18:42:00 -0700, Dan Stromberg wrote:

 On Sat, Oct 18, 2014 at 6:34 PM, Dan Stromberg drsali...@gmail.com wrote:
 Once the nc process actually write()s the data to its standard
 output (i.e. desriptor 1, not the stdout FILE*)
 I'm not sure why you're excluding stdout, but even if nc is using
 filedes 1 instead of FILE * stdout, isn't it kind of irrelevant?
 
 On further reflection, isn't it stdio that does the varied buffering,
 and filedes 1 that's always unbuffered?  IOW, the OP might wish nc was
 using 1, but it probably can't be given what they're seeing.

Yes. stdio does buffering. Writing to stdout stores data in a buffer; that
data should eventually be written to descriptor 1, although perhaps not
until immediately prior to termination.

Which is probably the cause of the OP's problem.

If it is, using a pseudo-tty would probably fix it. At startup,
stdin and stdout are line-buffered if they are associated with a tty and
fully-buffered otherwise (file, pipe, ...); stderr is unbuffered.

At least, this is the case on Unix and Windows. The exact requirements of
the C standard are:

As initially opened, the standard error stream is not fully
buffered; the standard input and standard output streams are
fully buffered if and only if the stream can be determined not
to refer to an interactive device.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Flush stdin

[Nobody]

On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote:

 I am using netcat to listen to a port and python to read stdin and print
 to the console.
 
 nc -l 2003 | python print_metrics.py
 
 sys.stdin.flush() doesn’t seem to flush stdin,

You can't flush an input stream.

 so I am using the termios module.

 I am receiving this exception
 termios.error: (25, 'Inappropriate ioctl for device')

termios only works on terminals, not pipes.

It's a safe bet that your problem is that nc isn't flushing its stdout
after each line (this is the default behaviour for stdio streams which
don't correspond to a terminal).

Check whether nc has a flag to line-buffer its output. If it doesn't,
the simplest solution is probably to write a Python script which creates a
pseudo-tty (using the pty module) and executes nc with its stdout
associated with the pty.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Flush stdin

[Nobody]

On Sat, 18 Oct 2014 12:32:07 -0500, Tim Chase wrote:

 On 2014-10-18 17:55, Nobody wrote:
 On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote:
 
  I am using netcat to listen to a port and python to read stdin and
  print to the console.
  
  nc -l 2003 | python print_metrics.py
  
  sys.stdin.flush() doesn’t seem to flush stdin,
 
 You can't flush an input stream.
 
 You can't flush it, but you can make it unbuffered.  You can either force
 python to use unbuffered stdio:

[snipped]

None of this helps in any way, as it's not the behaviour of the Python
script which is causing the problem, but that nc is (probably) buffering
its output, so the data isn't passed to the OS (let alone to the Python
script) in a timely manner.

Once the nc process actually write()s the data to its standard
output (i.e. desriptor 1, not the stdout FILE*), it will be available to
the Python script immediately thereafter without requiring any low-level
tweaks.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Most discussion on comp.lang.python is about developing with Python

[Nobody]

On Wed, 13 Nov 2013 15:35:56 -0500, bob gailer wrote:

 I joined a week or so ago.
 
 The subject line was copied from the description of comp.lang.python aka 
 python-list@python.org.
 
 I am very disappointed to see so much energy and bandwidth going to 
 conversations that bash individuals.
 
 Is there a moderator for this list?

list? As you have already noted, the mailing list is bi-directionally
gatewayed to a usenet group. The advantage of usenet is that anyone can
read or post without having to sign up. That's also its disadvantage.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: getpeername() on stdin?

[Nobody]

On Fri, 01 Nov 2013 14:55:38 -0400, random832 wrote:

 If it's possible to get this information with only the fd, then why does
 socket.fromfd require them?

The only person who can answer that is whoever came up with
socket.fromfd() in the first place.

I initially suspected that it might have been a Windows limitation, but
socket.fromfd() is only available on Unix.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: getpeername() on stdin?

[Nobody]

On Thu, 31 Oct 2013 12:16:23 -0400, Roy Smith wrote:

 I want to do getpeername() on stdin.  I know I can do this by wrapping a
 socket object around stdin, with
 
 s = socket.fromfd(sys.stdin.fileno(), family, type)
 
 but that requires that I know what the family and type are.   What I want
 to do is discover the family and type by looking at what getpeername()
 and/or getsockname() return.  Can this be done with the standard library?

I think that you'd need to use ctypes to access the underlying
getpeername() function from libc.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to avoid checking the same condition repeatedly ?

[Nobody]

On Mon, 28 Oct 2013 09:50:19 +, Wolfgang Maier wrote:

 So my question is: is there an agreed-upon generally best way of dealing
 with this?

Yes. Just leave the test inside the loop.

If you're sufficiently concerned about performance that you're willing to
trade clarity for it, you shouldn't be using Python in the first place.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: trying to strip out non ascii.. or rather convert non ascii

[Nobody]

On Sat, 26 Oct 2013 20:41:58 -0500, Tim Chase wrote:

 I'd be just as happy if Python provided a sloppy string compare
 that ignored case, diacritical marks, and the like.

Simply ignoring diactrics won't get you very far.

Most languages which use diactrics have standard conversions, e.g.
ö - oe, which are likely to be used by anyone familiar with the
language e.g. when using software (or a keyboard) which can't handle
diactrics.

OTOH, others (particularly native English speakers) may simply discard the
diactric. So to be of much use, a fuzzy match needs to handle either
possibility.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What version of glibc is Python using?

[Nobody]

On Sat, 12 Oct 2013 05:43:22 -0600, Ian Kelly wrote:

 Easier said than done.  The module is currently written in pure
 Python, and the comment Note: Please keep this module compatible to
 Python 1.5.2 would appear to rule out the use of ctypes to call the
 glibc function.

Last I heard, there was a standing policy to avoid using ctypes from
within the standard library. The stated rationale was that ctypes is
unsafe (it allows pure Python code to crash the process) and site
administrators should be able to remove the ctypes module without breaking
any part of the standard library other than ctypes itself.

There appear to be a few exceptions to this rule, i.e. a few standard
library modules import ctypes. But they are all within try/except blocks
(so they degrade gracefully if ctypes isn't present), and are limited to
improving the handling of edge cases rather than being essential to
providing documented functionality.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Complex literals (was Re: I am never going to complain about Python again)

[Nobody]

On Thu, 10 Oct 2013 14:12:36 +, Grant Edwards wrote:

 Nope.  i is electical current (though it's more customary to use upper
 case).

I is steady-state current (either AC or DC), i is small-signal
current.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Applying 4x4 transformation to 3-element vector with numpy

[Nobody]

On Tue, 08 Oct 2013 23:10:16 -0700, John Nagle wrote:

 I only need affine transformations.  This is just moving
 the coordinate system of a point, not perspective rendering. I have to do
 this for a lot of points, and I'm hoping numpy has some way to do this
 without generating extra garbage on the way in and the way out.

In which case, Christian's answer is correct.

For an affine transformation, the bottom row of the 4x4 matrix will be
[0 0 0 1], so you have:

[x']   [a b c d] [x]   [ax+by+cz+d]
[y'] = [e f g h] [y] = [ex+fy+gz+h]
[z']   [i j k l] [z]   [ix+jy+kz+l]
[1 ]   [0 0 0 1] [1]   [1 ]

 = 

[x']   [ax+by+cz+d]   [a b c] [x]   [d]
[y'] = [ex+fy+gz+h] = [e f g] [y] + [h]
[z']   [ix+jy+kz+l]   [i j k] [z]   [l]

IOW:
xyz_ = m[:3,:3] * xyz + m[:3,3:]

(where xyz is a column vector or 3xN array)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Receive packet using socket

[Nobody]

On Wed, 09 Oct 2013 08:37:39 -0700, tspiegelman wrote:

 I am trying to use socket to send / receive a packet (want to recreate
 some functionality of hping3 and port it to windows and mac as a tcp
 ping).  I am having some problems with the recv functionality of socket. 
 Below is the script I am using.  I get an ack from the server (used
 wireshark to ensure it was working) when I run this, but the script
 doesn't see the ack for some reason and the script exits with this error
 or a timeout:

 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

If you're trying to send/receive individual packets, you need to use
SOCK_RAW (which normally requires root/administrator privilege).

 s.recv(24)

Alternatively, if you use a normal TCP stream socket, the receiver must
consume all data which is sent to it, and only close the socket after the
writer has closed its end. Depending upon the application-layer protocol,
this may involve some form of QUIT command, or a half-close using
the .shutdown() method.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to streamingly read text file and display whenever updated text

[Nobody]

On Sat, 05 Oct 2013 00:38:51 -0700, galeomaga wrote:

 #!/usr/bin/python
 import time
 f = open('/home/martin/Downloads/a.txt')
 while 1:
   for line in f:
   print line;
   time.sleep(1);

So you're trying to implement tail -f?

First, check that tail -f actually works for your particular use case.

If the process writing the file uses buffered output, data will only
actually be appended to the file when the buffer is full. You can't read
what isn't there.

And if the process creates a new file with the same name, rather than
appending to the existing file, you'll still be reading the old file. You
would need to open the file again to read the new file.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handling 3 operands in an expression without raising an exception

[Nobody]

On Thu, 26 Sep 2013 10:26:48 +0300, Νίκος wrote:

 How can i wrote the two following lines so for NOT to throw out
 KeyErrors when a key is missing?
 
 city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
 gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
 Άγνωστη Πόλη

tz = None
ip = os.environ.get('HTTP_CF_CONNECTING_IP')
if ip:
  tz = gi.time_zone_by_addr(ip)
if not tz:
  ip = os.environ.get('REMOTE_ADDR')
  if ip:
tz = gi.time_zone_by_addr(ip)
if not tz:
  tz = ÎγνÏÏÏη Î Ïλη

Likewise for the hostname.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do I have to use global so much when using Turtle?

[Nobody]

On Sat, 21 Sep 2013 21:39:07 -0700, John Ladasky wrote:

 However, neither Screen.ontimer() not Screen.onkeypress() appear to give
 me a way to pass arguments to functions of my own.  Why don't they?  Is
 this some limitation of Tk?  I have worked with other GUI's before, and I
 don't remember having to jump through this particular hoop.

The usual approach in Python is to pass a bound method of an object, and
keep all of the data on that object.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Nobody]

On Wed, 11 Sep 2013 00:53:53 +, Steven D'Aprano wrote:

 and most routines that handle file names accept either text strings or 
 bytes strings:

I was going to say that just leaves environ and argv. But I see that
os.environb was added in 3.2. Which just leaves argv.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Send alt key to subprocess.PIPE stdin

[Nobody]

On Fri, 13 Sep 2013 01:27:53 +1000, Chris Angelico wrote:

 That said, though: These sorts of keystrokes often can be represented
 with escape sequences (I just tried it in xterm and Alt-D came out as
 \e[d),

Technically, that would be Meta-D (even if your Meta key has Alt printed
on it).

Alt-char produces chr(ord(char)+128); Meta-char produces \e+char.

At least, that's the historical distinction between Meta and Alt. With
xterm, the behaviour is configurable via the resources altIsNotMeta,
altSendsEscape and metaSendsEscape.

None of which has anything to do with trying to feed a GUI program key
events via stdin ...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Send alt key to subprocess.PIPE stdin

[Nobody]

On Wed, 11 Sep 2013 07:26:32 -0700, Wanderer wrote:

 How do I send the command 'Alt+D' to subprocess.PIPE?

You don't. GUI programs don't read stdin, they receive key press events
from the windowing system.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Nobody]

On Tue, 10 Sep 2013 17:07:09 +1000, Ben Finney wrote:

 * Python requires every programmer to know, or quickly learn, the basics
   of Unicode: to know that text is not, and never will be again,
   synonymous with a sequence of bytes.

If only the Python developers would learn the same lesson ...

Some of them are so hooked on Unicode that they won't accept that
sometimes a sequence of bytes really is just a sequence of bytes. Primary
example: most of the POSIX API.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can I trust downloading Python?

[Nobody]

On Sun, 08 Sep 2013 03:37:15 +, Dave Angel wrote:

 You can run a 32bit Python on 64bit OS, but not the oter way
 around.  And most people just match the bitness of Python against the
 bitness of the OS.

AFAICT, most people run 32-bit Python on any version of Windows.

[And this isn't limited to Python; most of the software on my Win64
system is 32-bit. And most of the 64-bit software is accounted for by
software which has to be 64-bit due to containing device drivers, shell
extensions or similar.]

Any add-on package which provides pre-compiled binaries will provide
32-bit binaries. Some of them will also provide 64-bit binaries, some of
them won't.

So unless you think that you might need to use more than 3-4 GiB of RAM
for a single Python process, or you need to use certain libraries
which are only available as 64-bit, getting the 32-bit version is
typically the safest option.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Monitor key presses in Python?

[Nobody]

On Mon, 09 Sep 2013 10:39:43 -0700, eamonnrea wrote:

 Is there a way to detect if the user presses a key in Python that works on
 most OS's? I've only seen 1 method, and that only works in Python 2.6 and
 less.

There's no generic solution to this.

At a minimum, there's getting key presses from a windowing system and
getting character input from a terminal or console. The two cases are
themselves quite different, and each case has differences between
operating systems.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script hangs when run from subprocess

[Nobody]

On Sat, 07 Sep 2013 03:55:02 -0700, larry.mart...@gmail.com wrote:

 I have a python script and when I run it directly from the command line
 it runs to completion. But I need to run it from another script. I do
 that like this: 
 
 p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 rv = p.wait()
 out_buf = p.stdout.read()
 
 When I do this, wait never returns.

The last two statements are the wrong way around. If you're reading a
process' output via a pipe, you shouldn't wait() for it until it has
closed its end of the pipe.

As it stands, you have a potential deadlock. If the subprocess tries to
write more data than will fit into the pipe, it will block until the
parent reads from the pipe. But the parent won't read from the pipe until
after the subprocess has terminated, which won't happen because the
subprocess is blocked waiting for the parent to read from the pipe ...


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A Pragmatic Case for Static Typing

[Nobody]

On Mon, 02 Sep 2013 09:44:20 +, Steven D'Aprano wrote:

 One factor I don't see very often mentioned is that static typing 
 increases coupling between distant parts of your code. If func() changes 
 from returning int to MyInt, everything that calls func now needs to be 
 modified to accept MyInt, no matter how similar MyInt is to int. You have 
 to make changes just to satisfy the compiler.

Not if the language has type inference (e.g. Haskell, or C++ templates).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess.Popen instance hangs

[Nobody]

On Thu, 29 Aug 2013 17:00:21 -0800, Tim Johnson wrote:

 ## This appears to be what works.
 def __exec(self,args) :
 Run the process with arguments
p =
subprocess.Popen(args,stderr=subprocess.PIPE,stdout=subprocess.PIPE)
while 1 :
output = p.stdout.read()

If the process tries to write more than a pipe's worth of data to stderr,
before closing stdout, it will block indefinitely.

If you want to process both stdout and stderr, you have to be able to
consume the data in whatever order the process generates it, which means
either using multiple threads or (on Unix) select/poll or non-blocking
I/O. This is what the .communicate() method does (threads on Windows,
select/poll on Unix).

The alternative is to merge both streams with stderr=subprocess.STDOUT, or
redirect one of them to a file (or /dev/null, etc).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Interface and duck typing woes

[Nobody]

On Wed, 28 Aug 2013 18:09:22 -0300, Joe Junior wrote:

 Of course I don't want to check isistance(), I like duck typing, but
 should I check if hasattr() and callable() before adding to the container?

That won't tell you if the object has a quack() method but with
incompatible semantics (e.g. wrong number or types of arguments).

 What is the pythonic way to deal with it?

Ignore it. If you want early type checking, use a statically-typed
language.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and displaying on 10 bit monitors

[Nobody]

On Tue, 13 Aug 2013 05:25:34 -0700, rlkling wrote:

 Or, asking another way, are there any python libraries that display images
 to 10 bit monitors as 10 bit images, and not scaled to 8 bit?

This should be possible using PyOpenGL and GLUT, with:

glutInitDisplayString(red=10 green=10 blue=10)

That's based upon a quick look at:

http://www.amd.com/us/Documents/10-Bit.pdf
http://www.nvidia.com/docs/IO/40049/TB-04701-001_v02_new.pdf

I don't have a 30-bpp monitor to test it with.

You may be able to do the same thing using e.g. QGLWidget or wxGLCanvas,
but I don't think that you can get 30-bpp with GDI, so you're limited to
solutions involving OpenGL (or DirectX, if you can figure out the
necessary pieces).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pair of filenos read/write each other?

[Nobody]

On Tue, 13 Aug 2013 16:10:41 -0700, Jack Bates wrote:

 Is there anything like os.pipe() where you can read/write both ends?

There's socket.socketpair(), but it's only available on Unix.

Windows doesn't have AF_UNIX sockets, and anonymous pipes (like the ones
created by os.pipe()) aren't bidirectional. Named pipes are bidirectional,
but you would need to choose a name, create one, connect, and accept (like
with sockets); there's no convenience function like socketpair(). Also,
you need to consider the security implications, as other processes can
(try to) connect to a named pipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using sudo to write to a file as root from a script

[Nobody]

On Thu, 08 Aug 2013 23:11:09 -0500, Adam Mercer wrote:

 I'm trying to write a script that writes some content to a file root
 through sudo, but it's not working at all. I am using:

   command = ['echo', '-n', channel, '|', 'sudo', 'tee', config_file]

You can't create a pipeline like this. All of the list elements after the
first will be passed as arguments to echo.

Try:

  command = ['sudo', 'tee', config_file]
  p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  out, _ = p.communicate('channel')

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using sudo to write to a file as root from a script

[Nobody]

On Fri, 09 Aug 2013 21:12:20 +0100, Nobody wrote:

 Try:
 
   command = ['sudo', 'tee', config_file]
   p = subprocess.Popen(command, stdout=subprocess.PIPE,
 stderr=subprocess.PIPE)
   out, _ = p.communicate('channel')

Oops; you also need stdin=subprocess.PIPE.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginner. 2d rotation gives unexpected results.

[Nobody]

On Tue, 23 Jul 2013 15:11:43 +0200, Peter Otten wrote:

 The conversion to int introduces a rounding error that accumulates over 
 time.

Most floating point calculations introduce a rounding error. If the
calculations are iterated, the error will accumulate.

In general, you want to avoid accumulating entire transformations. E.g. if
you want a spinning object, maintain the cumulative rotation angle and
rotate the original points each frame.

If you must accumulate transformations, you need to actively work to
maintain any desired invariants. E.g. if a transformation is supposed to
be orthonormal (all axes perpendicular and of unit length), you should
renormalise it periodically, otherwise the lengths and angles will change
over time.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Simple Python script as SMTP server for outgoing e-mails?

[Nobody]

On Mon, 22 Jul 2013 14:19:57 +0200, Gilles wrote:

 Incidently, how do ISP MTAs find whether the remote MTA is legit or
 running on some regular user's computer?

Look up the IP address in a database. If they don't have a database,
perform a reverse DNS lookup and reject anything which looks like a
typical auto-generated name for a consumer DSL/cable connection.

FWIW, I've been running sendmail on my home system (ADSL with static IP)
for years, and have had very few problems with mail being rejected.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Concurrent writes to the same file

[Nobody]

On Wed, 10 Jul 2013 22:57:09 -0600, Jason Friedman wrote:

 Other than using a database, what are my options for allowing two processes
 to edit the same file at the same time?  When I say same time, I can accept
 delays.

What do you mean by edit? Overwriting bytes and appending bytes are
simple enough, but inserting or deleting bytes such that subsequent bytes
change their offsets is harder.

 I considered lock files,

Well, you shouldn't have, unless you're targeting a platform which doesn't
support file locks (are there any left?).

 but I cannot conceive of how I avoid race conditions.

By using locks. E.g. fcntl.lockf() or msvcrt.locking().

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I get the OS System Font Directory(Cross-Platform) in python?

[Nobody]

On Thu, 11 Jul 2013 08:32:34 -0700, Metallicow wrote:

 How do I get the OS System Font Directory(Cross-Platform) in python?

What makes you think the system *has* a system font directory?

In the traditional X11 model, the only program which needs fonts is the X
server, and that can be configured to get its fonts from a font server
rather than from a local directory. Even if it doesn't use a font server,
the font directory will be on the system running the X server, not the one
running the client.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte

[Nobody]

On Thu, 04 Jul 2013 13:38:09 +0300, Νίκος wrote:

 So you are also suggesting that what gesthostbyaddr() returns is not 
 utf-8 encoded too?

The gethostbyaddr() OS function returns a byte string with no specified
encoding. Python 3 will doubtless try to decode that to a character string
using some (probably unspecified) encoding.

Names obtained from DNS should consist entirely of ASCII characters
(gethostbyname shouldn't attempt to decode internationalised names
which use IDN, it should return the raw data).

Names obtained by other means (e.g. /etc/hosts or Active Directory) could
contain anything, but if you use non-ASCII hostnames you're asking for
trouble.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Popen in Python3

[Nobody]

On Wed, 19 Jun 2013 23:03:05 +, Joseph L. Casale wrote:

 I am trying to invoke a binary that requires dll's in two places all of
 which are included in the path env variable in windows. When running this
 binary with popen it can not find either, passing env=os.environ to open
 made no difference.
 
 Anyone know what might cause this or how to work around this?

Do any of the DLLs have dependencies of their own which need to be found?

Do DLLs with the same name exist in directories which are searched before
%PATH%? Directories listed in %PATH% are searched after all other options
have failed.

The relevant MSDN page appears to be:

http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eval of expr with 'or' and 'and' within

[Nobody]

On Fri, 14 Jun 2013 18:16:05 +0300, Nick the Gr33k wrote:

 My question is why the expr (name and month and year) result in the 
 value of the last variable whic is variable year?

For much the same reason that an OR expression returns the first true
value.

or and and only evaluate as many arguments are required in order to
determine the correct result (aka short-circuit evaluation). If the
first argument of or is true, or the first argument of and is false,
the second argument isn't evaluated (this is important if evaluation can
have side effects).

The operators can be expressed as:

True or X = True
False or X = X

False and X = False
True and X = X

Note that in the short-circuit case, the result has the same sense (true
or false) as the first argument, while in the other case the result has
the same sense as the second argument.

Python implements these operators by returning the actual value which
determined the result of the expression rather than simply True or False.

If the result is known after evaluating the first argument, the first
argument is returned. If it has to evaluate the second argument, the
second argument is returned (by that point it has already forgotten
the value of the first argument).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eval of expr with 'or' and 'and' within

[Nobody]

On Fri, 14 Jun 2013 19:30:27 +, Grant Edwards wrote:

  2. Returning one the objects that result from the evaluation of the
 operands instead of returning True or False.
 
 This is what seems to be confusing him.  This is much less common
 than short-circuit evaluation.

FWIW, Lisp also does this. But Lisp is slightly simpler as the only false
value is nil, while everything else is true (including integer zero).

Although Python's any() and all() (which are closer to Lisp's and and
or insofar as they all work with any number of values, including zero)
always return True or False rather than the final value.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eval of expr with 'or' and 'and' within

[Nobody]

On Fri, 14 Jun 2013 16:49:11 +, Steven D'Aprano wrote:

 Unlike Javascript though, Python's idea of truthy and falsey is actually 
 quite consistent:

Beyond that, if a user-defined type implements a __nonzero__() method then
it determines whether an instance is true or false. If it implements a
__len__() method, then an instance is true if it has a non-zero length.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eval of expr with 'or' and 'and' within

[Nobody]

On Sat, 15 Jun 2013 03:56:28 +1000, Chris Angelico wrote:

 With a few random oddities:
 
 bool(float(nan))
 True
 
 I somehow expected NaN to be false. Maybe that's just my expectations
 that are wrong, though.

In general, you should expect the behaviour of NaN to be the opposite of
what you expect.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A few questiosn about encoding

[Nobody]

On Thu, 13 Jun 2013 12:01:55 +1000, Chris Angelico wrote:

 On Thu, Jun 13, 2013 at 11:40 AM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 The *mechanism* of UTF-8 can go up to 6 bytes (or even 7 perhaps?), but
 that's not UTF-8, that's UTF-8-plus-extra-codepoints.
 
 And a proper UTF-8 decoder will reject \xC0\x80 and \xed\xa0\x80, even
 though mathematically they would translate into U+ and U+D800
 respectively. The UTF-16 *mechanism* is limited to no more than Unicode
 has currently used, but I'm left wondering if that's actually the other
 way around - that Unicode planes were deemed to stop at the point where
 UTF-16 can't encode any more.

Indeed. 5-byte and 6-byte sequences were originally part of the UTF-8
specification, allowing for 31 bits. Later revisions of the standard
imposed the UTF-16 limit on Unicode as a whole.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Don't rebind built-in names* - it confuses readers

[Nobody]

On Thu, 13 Jun 2013 01:23:27 +, Steven D'Aprano wrote:

 Python does have a globally-global namespace. It is called builtins, and
 you're not supposed to touch it. Of course, being Python, you can if you
 want, but if you do, you are responsible for whatever toes you shoot off.
 
 Modifying builtins will effect *all modules*. That's normally too much,
 although it can very, very, very occasionally be useful.

For a specific example, gettext.install() adds the _() function to the
builtins namespace.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A few questiosn about encoding

[Nobody]

On Wed, 12 Jun 2013 14:23:49 +0300, Νικόλαος Κούρας wrote:

 So, how many bytes does UTF-8 stored for codepoints  127 ?

U+..U+007F  1 byte
U+0080..U+07FF  2 bytes
U+0800..U+  3 bytes
=U+1   4 bytes

So, 1 byte for ASCII, 2 bytes for other Latin characters, Greek, Cyrillic,
Arabic, and Hebrew, 3 bytes for Chinese/Japanese/Korean, 4 bytes for dead
languages and mathematical symbols.

The mechanism used by UTF-8 allows sequences of up to 6 bytes, for a total
of 31 bits, but UTF-16 is limited to U+10 (slightly more than 20 bits).

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Popen and reading stdout in windows

[Nobody]

On Tue, 11 Jun 2013 01:50:07 +, Joseph L. Casale wrote:

 I am using Popen to run the exe with communicate() and I have sent stdout
 to PIPE without luck. Just not sure what is the proper way to iterate over
 the stdout as it eventually makes its way from the buffer.

The proper way is:

p = subprocess.Popen(..., stdout=subprocess.PIPE)
for line in p.stdout:
# use 'line'
p.wait()

If the program uses stdin, matters get more complicated.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A few questiosn about encoding

[Nobody]

On Sun, 09 Jun 2013 03:44:57 -0700, Νικόλαος Κούρας wrote:

 Since 1 byte can hold up to 256 chars, why not utf-8 use 1-byte for 
 values up to 256? 
 
Because then how do you tell when you need one byte, and when you need 
two? If you read two bytes, and see 0x4C 0xFA, does that mean two 
characters, with ordinal values 0x4C and 0xFA, or one character with 
ordinal value 0x4CFA? 
 
 I mean utf-8 could use 1 byte for storing the 1st 256 characters. I
 meant up to 256, not above 256.

But then you've used up all 256 possible bytes for storing the first 256
characters, and there aren't any left for use in multi-byte sequences.

You need some means to distinguish between a single-byte character and an
individual byte within a multi-byte sequence.

UTF-8 does that by allocating specific ranges to specific purposes.
0x00-0x7F are single-byte characters, 0x80-0xBF are continuation bytes of
multi-byte sequences, 0xC0-0xFF are leading bytes of multi-byte sequences.

This scheme has the advantage of making UTF-8 non-modal, i.e. if a byte is
corrupted, added or removed, it will only affect the character containing
that particular byte; the encoder can re-synchronise at the beginning of
the following character.

OTOH, with encodings such as UTF-16, UTF-32 or ISO-2022, adding or
removing a byte will result in desyncronisation, with all subsequent
characters being corrupted.

 A surrogate pair is like itting for example Ctrl-A, which means is a
 combination character that consists of 2 different characters? Is this
 what a surrogate is? a pari of 2 chars?

A surrogate pair is a pair of 16-bit codes used to represent a single
Unicode character whose code is greater than 0x.

The 2048 codepoints from 0xD800 to 0xDFFF inclusive aren't used to
represent characters, but surrogates. Unicode characters with codes
in the range 0x1-0x10 are represented in UTF-16 as a pair of
surrogates. First, 0x1 is subtracted from the code, giving a value in
the range 0-0xF (20 bits). The top ten bits are added to 0xD800 to
give a value in the range 0xD800-0xDBFF, while the bottom ten bits are
added to 0xDC00 to give a value in the range 0xDC00-0xDFFF.

Because the codes used for surrogates aren't valid as individual
characters, scanning a string for a particular character won't
accidentally match part of a multi-word character.

 'a' to be utf8 encoded needs 1 byte to be stored ? (since ordinal = 65)
 'α΄' to be utf8 encoded needs 2 bytes to be stored ? (since ordinal is
  127 ) 'a chinese ideogramm' to be utf8 encoded needs 4 byte to be
 stored ? (since ordinal   65000 )

Most Chinese, Japanese and Korean (CJK) characters have codepoints within
the BMP (i.e. = 0x), so they only require 3 bytes in UTF-8. The
codepoints above the BMP are mostly for archaic ideographs (those no
longer in normal use), mathematical symbols, dead languages, etc.

 The amount of bytes needed to store a character solely depends on the
 character's ordinal value in the Unicode table?

Yes. UTF-8 is essentially a mechanism for representing 31-bit unsigned
integers such that smaller integers require fewer bytes than larger
integers (subsequent revisions of Unicode cap the range of possible
codepoints to 0x10, as that's all that UTF-16 can handle).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Game Development?

[Nobody]

On Fri, 07 Jun 2013 08:53:03 -0700, letsplaysforu wrote:

 I was planning on making a small 2D game in Python. Are there any
 libraries for this? I know of: 

[snip]

There's also Pyglet.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to detect the character encoding in a web page ?

[Nobody]

On Thu, 06 Jun 2013 03:55:11 +1000, Chris Angelico wrote:

 The HTTP header is completely out of band. This is the best way to
 transmit encoding information. Otherwise, you assume 7-bit ASCII and start
 parsing. Once you find a meta tag, you stop parsing and go back to the
 top, decoding in the new way.

Provided that the meta tag indicates an ASCII-compatible encoding, and you
haven't encountered any decode errors due to 8-bit characters, then
there's no need to go back to the top.

 ASCII-compatible covers a huge number of
 encodings, so it's not actually much of a problem to do this.

With slight modifications, you can also handle some
almost-ASCII-compatible encodings such as shift-JIS.

Personally, I'd start by assuming ISO-8859-1, keep track of which bytes
have actually been seen, and only re-start parsing from the top if the
encoding change actually affects the interpretation of any of those bytes.

And if the encoding isn't even remotely ASCII-compatible, you aren't going
to be able to recognise the meta tag in the first place. But I don't think
I've ever seen a web page encoded in UTF-16 or EBCDIC.

Tools like chardet are meant for the situation where either no encoding is
specified or the specified encoding can't be trusted (which is rather
common; why else would web browsers have a menu to allow the user to
select the encoding?).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing filenames from Greeklish = Greek (subprocess complain)

[Nobody]

On Mon, 03 Jun 2013 23:28:21 -0700, nagia.retsina wrote:

 I can't believe Chrome whcih by default uses utf8 chosed iso-8859-1 to
 presnt the filenames. 

Chrome didn't choose ISO-8859-1, the server did; the HTTP response says:

  Content-Type: text/html;charset=ISO-8859-1

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing filenames from Greeklish = Greek (subprocess complain)

[Nobody]

On Tue, 04 Jun 2013 00:58:42 -0700, Νικόλαος Κούρας wrote:

 Τη Τρίτη, 4 Ιουνίου 2013 10:39:08 π.μ. UTC+3, ο
 χρήστης Nobody έγραψε:
 
 Chrome didn't choose ISO-8859-1, the server did; the HTTP response says:
   Content-Type: text/html;charset=ISO-8859-1
 
 From where do you see this

$ wget -S -O - http://superhost.gr/data/apps/
--2013-06-04 14:00:10--  http://superhost.gr/data/apps/
Resolving superhost.gr... 82.211.30.133
Connecting to superhost.gr|82.211.30.133|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Server: ApacheBooster/1.6
  Date: Tue, 04 Jun 2013 13:00:19 GMT
  Content-Type: text/html;charset=ISO-8859-1
  Transfer-Encoding: chunked
  Connection: keep-alive
  Vary: Accept-Encoding
  X-Cacheable: YES
  X-Varnish: 2000177813
  Via: 1.1 varnish
  age: 0
  X-Cache: MISS

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing filenames from Greeklish = Greek (subprocess complain)

[Nobody]

On Sat, 01 Jun 2013 08:44:36 -0700, Νικόλαος Κούρας wrote:

 CalledProcessError: Command '/home/nikos/public_html/cgi-bin/files.py' 
 returned non-zero exit status 1 
   args = (1, '/home/nikos/public_html/cgi-bin/files.py') 
   cmd = '/home/nikos/public_html/cgi-bin/files.py' 
   output = b'Content-type: text/html; charset=utf-8\n\nbod...n position 
 74: surrogates not allowed\n\n--\n\n' 
   returncode = 1 
   with_traceback = built-in method with_traceback of CalledProcessError 
 object

The traceback indicates that files.py terminated with a non-zero exit
code, indicating an error.

And that's *all* that can be determined from the information which you
have posted.

If you want to solve the problem, you'll need to make files.py generate
useful error messages, then capture them.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Create a file in /etc/ as a non-root user

[Nobody]

On Fri, 31 May 2013 02:12:58 -0700, BIBHU DAS wrote:

 I am a python novice;request all to kindly bear with me.
 
 fd = open('/etc/file','w')
 fd.write('jpdas')
 fd.close()
 
 
 The above snippet fails with:

 IOError: [Errno 13] Permission denied: '/etc/file'

As it should.

 Any Idea how to create a file in /etc as non-root user?

This should not be possible. The language used is irrelevant.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Short-circuit Logic

[Nobody]

On Thu, 30 May 2013 12:07:40 +0300, Jussi Piitulainen wrote:

 I suppose this depends on the complexity of the process and the amount
 of data that produced the numbers of interest. Many individual
 floating point operations are required to be within an ulp or two of
 the mathematically correct result, I think, and the rounding error
 when parsing a written representation of a number should be similar.

Elementary operations (+, -, *, /, %, sqrt) are supposed to be within
+/- 0.5 ULP (for round-to-nearest), i.e. the actual result should be the
closest representable value to the exact result.

Transcendental functions should ideally be within +/- 1 ULP, i.e. the
actual result should be one of the two closest representable values to the
exact result. Determining the closest value isn't always feasible due to
the table-maker's dilemma, i.e. the fact that regardless of the number
of digits used for intermediate results, the upper and lower bounds
can remain on opposite sides of the dividing line.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Short-circuit Logic

[Nobody]

On Thu, 30 May 2013 19:38:31 -0400, Dennis Lee Bieber wrote:

   Measuring 1 foot from the 1000 foot stake leaves you with any error
 from datum to the 1000 foot, plus any error from the 1000 foot, PLUS any
 azimuth error which would contribute to shortening the datum distance.

First, let's ignore azimuthal error.

If you measure both distances from the same origin, and you have a
measurement error of 0.1% (i.e. 1/1000), then the 1000' measurement will
actually be between 999' and 1001', while the 1001' measurement will be
between 1000' and 1002' (to the nearest whole foot).

Meaning that the distance from the 1000' stake to the 1001' stake could be
anywhere between -1' and 3' (i.e. the 1001' stake could be measured as
being closer than the 1000' stake).

This is why technical drawings which include regularly-spaced features
will normally specify the positions of features relative to their
neighbours instead of (or as well as) relative to some origin.

When you're dealing with relative error, the obvious question is
relative to what?.

-- 
http://mail.python.org/mailman/listinfo/python-list