[issue20699] Document that binary IO classes work with bytes-likes objects

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, I forgot to say my LGTM on your last patch. It LGTM.

But I'm feeling the documentation epic about bytes-like objects is far from the 
end.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27115] IDLE/tkinter: in simpledialog, != [OK] click

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is not Tkinter issue, but IDLE issue.

Other similar cases:

Open search dialog and find something. The status bar is not updated. It is 
updated only if you close the search dialog.

Select a text and delete it by a mouse. The status bar is not updated. Choice 
the Edit|Redo action by a mouse. The status bar is not updated.

--
components:  -Tkinter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27233] Missing documentation for PyOS_FSPath

[Berker Peksag]

Changes by Berker Peksag :


--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20699] Document that binary IO classes work with bytes-likes objects

[Martin Panter]

Martin Panter added the comment:

Thanks for you help figuring this out Serhiy, especially for the Python 2 case.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27243] __aiter__ should return async iterator instead of awaitable

[Martin Panter]

Martin Panter added the comment:

I didn’t realize, sorry for the noise

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24136] document PEP 448: unpacking generalization

[Martin Panter]

Martin Panter added the comment:

Here is a new patch that also updates the documentation for list etc displays 
as well as function calls. Let me know what you think.

The 3.5 What’s New notes were written separately; Neil’s patch was never 
applied. But I have rescued his update for functools.partial() in my new patch.

While experimenting with the current behaviour, I found some surprising 
inconsistencies. The following syntaxes are allowed:

>>> x, *y
>>> a = x, *y
>>> f"{x, *y}"  # New in 3.6
>>> async def f(): await x, *y

But the following all produce “SyntaxError: invalid syntax”:

>>> a += x, *y
>>> eval("x, *y")
>>> def f(): return x, *y
>>> def f(): yield x, *y
>>> for i in x, *y: ...

Also, the expressions allowed for unpacking in general are more limited than in 
function calls:

>>> f(x, *y == z)  # Allowed
>>> (x, *y == z)
SyntaxError: invalid syntax

--
Added file: http://bugs.python.org/file43339/unpacking-doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5124] IDLE - pasting text doesn't delete selection

[Terry J. Reedy]

Terry J. Reedy added the comment:

I had some trouble committing and pushing these patches.  After deleting TEntry 
and committed the 2.7 and 3.5 patches and tried to do the usual 3.5 to default 
forward merge in TortoiseHG.  I got

% hg merge --tool :merge --verbose 101861
resolving manifests
abort: case-folding collision between Lib/idlelib/pyshell.py and 
Lib/idlelib/PyShell.py
[command returned code 255 Fri Jun 10 22:33:09 2016]

This seems to be a known problem.  After some effort to resolve the situation, 
I gave up and used my replaced my repository with a copy of my backup copy, to 
undo the things I had tried.

Since the renames on 5/22, I have done other merges without problem, but I 
rechecked and none involved a file where case was the only difference. I will 
try again by temporarily renaming pyshell back to PyShell first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: how to solve memory

[meInvent bbird]

i use a version having better indentation,

and then remove redundant which if sum column == 54 , do not add this column
into initlist

and add deep > 0 before recursive call
and print number of initlist is 118,XXX

but it is still running, where it run , and why run a very long time

def DFS(b, deep, maxx, sourceoperators, path): 
initlist = [] 
if deep > 0: 
print("deep=", deep) 
for aa,bb in itertools.combinations(sourceoperators, 2): 
print(aa,bb) 
if deep == maxx: 
finalresult = [] 
op1xy = [aa[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
op1yz = [aa[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
op1xz = [aa[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
op2xy = [bb[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
op2yz = [bb[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
op2xz = [bb[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
if sum(op1xy) == 54: 
path.append([(deep, aa, "xy")]) 
else:
initlist.append(op1xy)
if sum(op1yz) == 54: 
path.append([(deep, aa, "yz")]) 
else:
initlist.append(op1yz)
if sum(op1xz) == 54: 
path.append([(deep, aa, "xz")]) 
else:
initlist.append(op1xz)
if sum(op2xy) == 54: 
path.append([(deep, bb, "xy")]) 
else:
initlist.append(op2xy)
if sum(op2yz) == 54: 
path.append([(deep, bb, "yz")]) 
else:
initlist.append(op2yz)
if sum(op2xz) == 54: 
path.append([(deep, bb, "xz")]) 
else:
initlist.append(op2xz)
else: 
level = [] 
for j in range(len(b)): 
op1xy = [aa[b[j][i]] for i in range(len(b[j]))] 
op2xy = [bb[b[j][i]] for i in range(len(b[j]))] 
if sum(op1xy) == 54: 
path.append([(deep, aa, "xy")]) 
else:
initlist.append(op1xy)
if sum(op2xy) == 54: 
path.append([(deep, bb, "xy")]) 
else:
initlist.append(op2xy)
level.extend([op1xy, op2xy]) 
if deep == maxx: 
b = [] 
print("initlist=")
print(len(initlist))
for aaa,bbb in itertools.combinations(initlist, 2): 
b.append([str(i)+str(j) for i,j in zip(aaa, bbb)]) 
if deep > 0:
path2 = DFS(b, deep-1, maxx, sourceoperators, path) 
path.append(path2)
return path 

path = [] 
mresult = DFS(b, 2, 2, mylist, path) 



On Saturday, June 11, 2016 at 8:34:16 AM UTC+8, MRAB wrote:
> On 2016-06-11 00:31, meInvent bbird wrote:
> > it is quite ridiculous,
> > this time i am sure that i put correct indentation
> > and add a else statement to make sure recursive call inside the
> > if statement , it still memory error,
> >
> > where is the memory error?
> >
> > is itertools.combinations so big for the list?
> >
> [snip]
> 
> How long is initlist?
> 
> When I ran the code, it said over 100_000 items.
> 
> How many combinations would there be?
> 
> Over 10_000_000_000.
> 
> That's how long the list 'b' would be.
> 
> You'll need 10s of gigabytes of memory and a lot of patience!

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27263] Tkinter sets the HOME environment variable, breaking scripts

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is Windows specific issue, I can't help with this, sorry. Zach should be 
more experienced in this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27263] Tkinter sets the HOME environment variable, breaking scripts

[Terry J. Reedy]

Terry J. Reedy added the comment:

Python does not set TCL_LIBRARY.  _tkinter finds tcl/tk installed with Python 
at its known location for the platform. tkinter and IDLE work fine for months.  
Then user installs another program. That other software (not user, and 
unbeknownst to the user) sets TCL_LIBRARY to something that _tkinter cannot 
use.  I am guessing that _tkinter, seeing TCL_LIBRARY, tries to use alternate 
install, cannot, and gives up, instead of falling back to the python-tcl it was 
using for months.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5124] IDLE - pasting text doesn't delete selection

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

It would be better to exclude TEntry from the list. This is a hack, and as 
every hack it can be not fully compatible with future Tk versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: how to solve memory

[meInvent bbird]

this time i remove redundant which not add full column sum already has beeb 27*2

but it always has syntax error, indentation error, 

where is wrong?

def DFS(b, deep, maxx, sourceoperators, path):
 initlist = []
 if deep > 0:
  print("deep=", deep)
  for aa,bb in itertools.combinations(sourceoperators, 2):
   print(aa,bb)
   if deep == maxx:
 finalresult = []
 op1xy = [aa[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
 op1yz = [aa[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
 op1xz = [aa[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
 op2xy = [bb[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
 op2yz = [bb[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
 op2xz = [bb[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
 if sum(op1xy) == 54:
  path.append([(deep, aa, b, "xy")])
 else:
  initlist.append(op1xy)
 if sum(op1yz) == 54:
  path.append([(deep, aa, b, "yz")])
 else:
  initlist.append(op1yz)  
 if sum(op1xz) == 54:
  path.append([(deep, aa, b, "xz")])
 else:
  initlist.append(op1xz)
 if sum(op2xy) == 54:
  path.append([(deep, bb, b, "xy")])
 else:
  initlist.append(op2xy)
 if sum(op2yz) == 54:
  path.append([(deep, bb, b, "yz")])
 else:
  initlist.append(op2yz)
 if sum(op2xz) == 54:  
  path.append([(deep, bb, b, "xz")])
 else:
  initlist.append(op2xz)
   else:
 level = []
 for j in range(len(b)):
  op1xy = [aa[b[j][i]] for i in range(len(b[j]))]
  op2xy = [bb[b[j][i]] for i in range(len(b[j]))]
  if sum(op1xy) == 54:
   path.append([(deep, aa, b[j], "xy")])
  else:
   initlist.append(op1xy)
  if sum(op2xy) == 54:
   path.append([(deep, bb, b[j], "xy")])
  else:
   initlist.append(op2xy)
  level.append(op1xy)
  level.append(op2xy)
   print("initlist=")
   print(len(initlist)
 if deep == maxx:
   if deep > 0:
 b = []
 for aaa,bbb in itertools.combinations(initlist, 2): 
  b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
 print("deep=")
 print(deep)
 path2 = DFS(b, deep-1, maxx, sourceoperators, path)
 path.append(path2)
   else:
 print("")
   print("path=")
   print(len(path))
 else:
   if deep > 0:
 for aaa,bbb in itertools.combinations(initlist, 2):
  b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
 print("deep=")
 print(deep)
 path2 = DFS(b, deep-1, maxx, sourceoperators, path)
 path.append(path2)
   else:
 print("")
   print("path=")
   print(len(path))
 return path



On Saturday, June 11, 2016 at 8:34:16 AM UTC+8, MRAB wrote:
> On 2016-06-11 00:31, meInvent bbird wrote:
> > it is quite ridiculous,
> > this time i am sure that i put correct indentation
> > and add a else statement to make sure recursive call inside the
> > if statement , it still memory error,
> >
> > where is the memory error?
> >
> > is itertools.combinations so big for the list?
> >
> [snip]
> 
> How long is initlist?
> 
> When I ran the code, it said over 100_000 items.
> 
> How many combinations would there be?
> 
> Over 10_000_000_000.
> 
> That's how long the list 'b' would be.
> 
> You'll need 10s of gigabytes of memory and a lot of patience!

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27051] Create PIP gui

[Terry J. Reedy]

Terry J. Reedy added the comment:

sys.executable has the path to the current executable.  But you don't even need 
that.  When you import pip, it gets that itself to find site-packages.

You are making this way too complicated and are trying to do what pip can do. I 
don't see that we need to know the system.  Pip does that.  You don't need to 
know all installed pythons, certainly not to begin.  The first thing is to 
install to the current version. Expand on that later.  Don't worry about 
virtual environments.  Beginners don't know what they are, and this is aimed at 
beginners.

3.6.0a2 will be tagged on Sunday, and I can look at this more after that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27173] Modern Unix key bindings for IDLE

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +IDLE: Fix deletion of custom themes and key bindings

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: i'm a python newbie & wrote my first script, can someone critique it?

[Matt Wheeler]

First of all welcome :)

The other suggestions you've received so far are good so I won't repeat
them... (note that in particular I've reused the names you've chosen in
your program where I've given code examples, but that's purely for clarity
and I agree with the others who've said you should use a more pythonic
naming convention)

On Fri, 10 Jun 2016, 23:52 mad scientist jr, 
wrote:

> Is this group appropriate for that kind of thing?
> (If not sorry for posting this here.)
>
> So I wanted to start learning Python, and there is s much information
> online, which is a little overwhelming. I really learn best from doing,
> especially if it's something actually useful. I needed to create a bunch of
> empty folders, so I figured it was a good exercise to start learning Python.
> Now that it's done, I am wondering what kind of things I could do better.
> Here is the code:
>
> # WELCOME TO MY FIRST PYTHON SCRIPT!
> # FIRST OF ALL, IT ***WORKS***!!! YAY!
> # I AM A VBA AND JAVASCRIPT PROGRAMMER,
> # SO IT IS PROBABLY NOT VERY "PYTHONIC",
> # SO PLEASE FEEL FREE TO TEAR THE SCRIPT A NEW ONE
> # AFTER YOU ARE SHOCKED BY THIS BAD CODE,
> # ALL I ASK IS THAT IF YOU THINK SOMETHING IS BAD,
> # PLEASE POST AN EXAMPLE OF THE "CORRECT" WAY OF DOING IT
> # COMING FROM VBA, I KNOW A LITTLE OOP,
> # BUT NOT C++ C# JAVA STUFF LIKE "INTERFACES" AND "ABSTRACT BASE CLASSES"
> # SO IF YOU GET INTO SUCH TERMINOLOGY MY EYES MIGHT START TO GLAZE OVER
> # UNLESS YOU CARE TO EXPLAIN THAT TOO
>
Don't worry, Python doesn't have "interfaces" like Java (though it does
have multiple inheritance instead, which is more powerful), and it's not
necessary to learn about advanced things like abstract classes until you're
comfortable :)

>
>
> 
> # GLOBAL VALUES
> sForPythonVersion="3"
> #sFolderPathTemplate = "c:\\myscripts\\MP3 Disc "
> sFolderPathTemplate = "c:\\temp\\MP3 Disc "
>
It's possible in Python to use forward slashes in paths, even on Windows,
which would allow you to avoid the double slashes above (and can make your
code more easily ported to other OSes, which pretty universally use a '/'
in paths.
I would also suggest you use python's string formatting capabilities
instead here (see below where I use this FOLDER_TEMPLATE string):

FOLDER_TEMPLATE = 'C:/temp/MP3 Disk {count:03}'

iFromCount = 1
> iToCount = 250
> iCountWidth = 3
>
>
> 
> # SUPPORT FUNCTIONS
>
> #
> //
> def is_string(myVar): # is_string IS MORE READABLE THAN isinstance (PLAIN
> ENGLISH!)
> #PYTHON 3 IS NOT LIKING THIS: return ( isinstance(myVar, str) or
> isinstance(myVar, unicode) )
> #PYTHON 3 IS NOT LIKING THIS: return isinstance(myVar, basestr)
> return isinstance(myVar, str)
>

I know this one's been asked already but I think it deserves expanding on.
In general in Python it's recommended that you don't try too hard (if at
all) to check types of things. Think about what purpose this serves.
Pythonic programs follow what's known as "duck typing". i.e. if it quacks
like a duck then we can treat it like it's a duck.
This lets you write code which expects, for example something like a
string, or a dict or list, but isn't strictly bound to those types. That
allows other programmers to create their own classes, perhaps a subclass of
dict or list, or some unconnected class which just happens to be list-like
enough, and it can just work with your code.

Of course there are places where you do need to check the type of
something, but if you think you need to it's always worth really asking
yourself why you think that.
(I think this is particularly true when it comes to `str`. Most classes can
be represented one way or another as a string, so why not let people pass
you anything if you're going to read it as a string anyway? Refusing to
accept a non-string is more work for you and more work for them for little
benefit)

#
> //
> # THIS IS SOME SAMPLE FUNCTION FROM A BOOK I AM READING "PYTHON IN EASY
> STEPS"
> def strip_one_space(s):
> if s.endswith(" "): s = s[:-1]
> if s.startswith(" "): s = s[1:]
> return s
>
> #
> //
> def get_exact_python_version():
> import sys
>

Imports belong at the top of the file unless there's a good reason you

[issue25733] Code and IDLE should catch all compile errors.

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

SystemError is serious bug. Please open separate issue for this.

> Why does compile not support null bytes in the first place?

Due to implementation detail of CPython tokenizer. I uses NUL-terminated C 
strings.

Yet one possible exception: MemoryError. I agree that the long tuple should be 
replaced by Exception.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27263] Tkinter sets the HOME environment variable, breaking scripts

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What is the problem? That user sets incorrect TCL_LIBRARY? I don't know how 
_tkinter can distinguish correct TCL_LIBRARY from the incorrect one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: i'm a python newbie & wrote my first script, can someone critique it?

[Larry Hudson via Python-list]

On 06/10/2016 03:52 PM, mad scientist jr wrote:

Is this group appropriate for that kind of thing?
(If not sorry for posting this here.)

So I wanted to start learning Python, and there is s much information 
online, which is a little overwhelming. I really learn best from doing, 
especially if it's something actually useful. I needed to create a bunch of 
empty folders, so I figured it was a good exercise to start learning Python.
Now that it's done, I am wondering what kind of things I could do better.
Here is the code:


It is FAR too complicated -- it's BASIC written Python.  Python is MUCH easier.

First a couple of general comments...

Drop the Hungarian notation!!  Python uses dynamic typing.  Variables do NOT have a type, the 
data they hold have types, but not the variable itself.  ANY variable can hod ANY data type at 
ANY time.  And Python uses duck typing -- get used to it.


Generally you should put all the imports at the beginning of the program, NOT in each function. 
 A possible exception could be if an import is only used in one function.




# GLOBAL VALUES
sForPythonVersion="3"
#sFolderPathTemplate = "c:\\myscripts\\MP3 Disc "
sFolderPathTemplate = "c:\\temp\\MP3 Disc "
iFromCount = 1
iToCount = 250
iCountWidth = 3


Drop the  from the template string.  (More on this below.)



# SUPPORT FUNCTIONS

# 
//
def is_string(myVar): # is_string IS MORE READABLE THAN isinstance (PLAIN 
ENGLISH!)
 #PYTHON 3 IS NOT LIKING THIS: return ( isinstance(myVar, str) or 
isinstance(myVar, unicode) )
 #PYTHON 3 IS NOT LIKING THIS: return isinstance(myVar, basestr)
 return isinstance(myVar, str)


Not necessary.


# 
//
# THIS IS SOME SAMPLE FUNCTION FROM A BOOK I AM READING "PYTHON IN EASY STEPS"
def strip_one_space(s):
 if s.endswith(" "): s = s[:-1]
 if s.startswith(" "): s = s[1:]
 return s


???  Is this used anyplace?  Delete it.


# 
//
def get_exact_python_version():
 import sys
 sVersion = ".".join(map(str, sys.version_info[:3]))
 sVersion = sVersion.strip()
 return sVersion

sys.version_info[:3] by itself gives a three-element tuple.  Probably easier to use than the 
string version.


A couple alternatives:
sys.version[:5] or perhaps a bit safer -- sys.version.split()[0] both directly give you the 
string version.  (Note this uses version not version_info.)



# 
//
# TO DO: RETURN TO THE LEFT OF FIRST "."
def get_python_version():
 sVersion = get_exact_python_version()
 return sVersion[0:1]


Probably unnecessary as a function.  A simple sVersion[0] in-line might be 
easier.


# 
//
# CHECK PYTHON VERSION, IF IT'S WRONG THEN GRACEFULLY EXIT BEFORE IT BLOWS UP
# (DAMN, PYTHON 2.x STILL COMPLAINS WITH SYNTAX ERRORS!!)
# MAYBE THIS COULD STILL BE USEFUL FOR CHECKING THE SUB-VERSION, IN THAT CASE
# TO DO: MORE GRANULAR CHECK, EG IF VERSION >= 3.5.0
def exit_if_wrong_python_version(sRightVersion):
 import os
 sCurrentVersion = get_python_version()
 if (sCurrentVersion != sRightVersion):
 print("" +
   "Wrong Python version (" +
   sCurrentVersion +
   "), this script should be run using Python " +
   sRightVersion +
   ".x. Exiting..."
   )
 os._exit(0)


Get used to Python string formatting...
print("Wrong Python version ({}), this script should be run using "
"Python {}.x,  Exiting...".format(sCurrentVersion, sRightVersion))

Notice how I split the long single-line string into two shorter strings on two lines relying on 
the automatic string concatenation in Python.  "string1 " "string2" becomes "string1 string2". 
Any whitespace (spaces, tabs, newlines) are ignored and the two strings are stuck together.


Also the common use is sys.exit() instead of os._exit().


# 

[issue27051] Create PIP gui

[Upendra Kumar]

Upendra Kumar added the comment:

I, actually want to make a 'configuration object' before running the 'pip' GUI. 
Therefore one of the parameters of the configuration object will be to detect 
the path to installed 'python.exe' in Windows. In order to get executable paths 
I need to read the Windows registry keys.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27051] Create PIP gui

[Terry J. Reedy]

Terry J. Reedy added the comment:

I don't know what you mean by 'print with winreg'.  I presume you need admin 
privilege to alter the registry.  But why are you involved with it?  I just 
want you to use the pip interface, and let pip worry about such low level stuff.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27291] two heap corruption issue

[Park Alex]

New submission from Park Alex:

Hello,

I would like to report two heap corruption issue.

Test environment:
python ersion: python 2.7.11+ 
hg id: d858eadf2602 (2.7)
compile: clang with ASAN
OS: ubuntu x86_64

One is heap-buffer-overflow, the other is heap-user-after-free. 
All of samples are attached in this bug report.

Thanks,
-- Alex

In detail,

1) heap-buffer-overflow bug could be triggerd at cpython/Python/ceval.c:1229
ASAN report is following:

=
==26786==ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x77ec56e8 at pc 0x5ec87f bp 0x7fffd2d0 sp 0x7fffd2c8
READ of size 8 at 0x77ec56e8 thread T0
#0 0x5ec87e in PyEval_EvalFrameEx /project/cpython/Python/ceval.c:1229
#1 0x5d3c6c in PyEval_EvalCodeEx /project/cpython/Python/ceval.c:3582
#2 0x5d2b11 in PyEval_EvalCode /project/cpython/Python/ceval.c:669
#3 0x6612d9 in run_pyc_file /project/cpython/Python/pythonrun.c:1406
#4 0x6612d9 in PyRun_SimpleFileExFlags 
/project/cpython/Python/pythonrun.c:946
#5 0x48e3dc in Py_Main /project/cpython/Modules/main.c:640
#6 0x76ce282f in __libc_start_main 
/build/glibc-GKVZIf/glibc-2.23/csu/../csu/libc-start.c:291
#7 0x48c518 in _start (/project/cpython/python.asan+0x48c518)

0x77ec56e8 is located 280 bytes to the left of 196608-byte region 
[0x77ec5800,0x77ef5800)
allocated by thread T0 here:
#0 0x476429 in __interceptor_malloc 
/project/clang-3.4/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74
#1 0x50e969 in dictresize /project/cpython/Objects/dictobject.c:643
#2 0x537844 in PyString_InternInPlace 
/project/cpython/Objects/stringobject.c:4757

SUMMARY: AddressSanitizer: heap-buffer-overflow 
/project/cpython/Python/ceval.c:1229 PyEval_EvalFrameEx
Shadow bytes around the buggy address:
  0x10007efd0a80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0a90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0aa0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0ab0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0ac0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x10007efd0ad0: fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]fa fa
  0x10007efd0ae0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0af0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x10007efd0b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007efd0b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007efd0b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:   00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone: fa
  Heap right redzone:fb
  Freed heap region: fd
  Stack left redzone:f1
  Stack mid redzone: f2
  Stack right redzone:   f3
  Stack partial redzone: f4
  Stack after return:f5
  Stack use after scope: f8
  Global redzone:f9
  Global init order: f6
  Poisoned by user:  f7
  ASan internal: fe
==26786==ABORTING

2) heap-user-after-free bug could be triggerd at 
cpython/Objects/dictobject.c:732
ASAN report is following:

=
==26918==ERROR: AddressSanitizer: heap-use-after-free on address 0x60f0ef98 
at pc 0x50f5ac bp 0x7fffa1d0 sp 0x7fffa1c8
READ of size 8 at 0x60f0ef98 thread T0
#0 0x50f5ab in PyDict_GetItem /project/cpython/Objects/dictobject.c:732
#1 0x537792 in PyString_InternInPlace 
/project/cpython/Objects/stringobject.c:4750
#2 0x64fad5 in r_object /project/cpython/Python/marshal.c:822
#3 0x650d00 in r_object /project/cpython/Python/marshal.c:1037
#4 0x64edf6 in r_object /project/cpython/Python/marshal.c:886
#5 0x650c1b in r_object /project/cpython/Python/marshal.c:1019
#6 0x64e1b1 in PyMarshal_ReadObjectFromString 
/project/cpython/Python/marshal.c:1183
#7 0x64e1b1 in PyMarshal_ReadLastObjectFromFile 
/project/cpython/Python/marshal.c:1144
#8 0x6429c1 in read_compiled_module /project/cpython/Python/import.c:823
#9 0x6429c1 in load_source_module /project/cpython/Python/import.c:1094
#10 0x644cda in import_submodule /project/cpython/Python/import.c:2722
#11 0x643e81 in load_next /project/cpython/Python/import.c:2537
#12 0x63e061 in import_module_level /project/cpython/Python/import.c:2254
#13 0x63e061 in PyImport_ImportModuleLevel 
/project/cpython/Python/import.c:2310
#14 0x5c4e1a in builtin___import__ /project/cpython/Python/bltinmodule.c:49
#15 0x5e2535 in do_call /project/cpython/Python/ceval.c:4564
#16 0x5e2535 in call_function /project/cpython/Python/ceval.c:4372
#17 0x5e2535 in PyEval_EvalFrameEx /project/cpython/Python/ceval.c:2987
#18 0x5d3c6c in PyEval_EvalCodeEx /project/cpython/Python/ceval.c:3582
#19 0x7237f3 in function_call /project/cpython/Objects/funcobject.c:523
#20 

[issue27051] Create PIP gui

[Upendra Kumar]

Upendra Kumar added the comment:

I am unable to print anything using winreg module. Does it require 
administrative privileges?

I am using this code sample :

import os
import errno
import winreg

proc_arch = os.environ['PROCESSOR_ARCHITECTURE'].lower()
try:
proc_arch64 = os.environ['PROCESSOR_ARCHITEW6432'].lower()
except KeyError:
proc_arch64 = None
pass

if proc_arch == 'x86' and not proc_arch64:
arch_keys = {0}
elif proc_arch == 'x86' or proc_arch == 'amd64':
arch_keys = {winreg.KEY_WOW64_32KEY, winreg.KEY_WOW64_64KEY}
else:
raise Exception("Unhandled arch {}".format(proc_arch))

for arch_key in arch_keys:
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r"SOFTWARE\Python", 0, 
winreg.KEY_READ | arch_key)
for i in range(0, winreg.QueryInfoKey(key)[0]):
skey_name = winreg.EnumKey(key, i)
skey = winreg.OpenKey(key, skey_name)
try:
print(winreg.QueryValueEx(skey, 'DisplayName')[0])
except OSError as e:
if e.errno == errno.ENOENT:
pass
finally:
skey.Close()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: fast dictionary initialization

[Tim Chase]

On 2016-06-10 14:07, maurice wrote:
> example:
> valuesList = [1,2,3]
> keysList   = ['1','2','3']
> 
> So the dictionary can basically convert string to int:
> 
> dictionary = {'1':1, '2':2, '3':3}

A couple similar options:


The most straightforward translation of your description:

  opt1 = dict(zip(keysList, valuesList))
  print(opt1["2"])

And one where you generate the strings on the fly:

  opt2 = dict((str(i), i) for i in range(1, 4))
  print(opt2["2"])

And one where you use the int() function instead of a mapping because
the whole idea of storing a dict worth of string-numbers-to-numbers
seems somewhat silly to me:

  print(int("2"))

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27173] Modern Unix key bindings for IDLE

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
assignee:  -> terry.reedy
nosy: +markroseman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27263] Tkinter sets the HOME environment variable, breaking scripts

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy: +markroseman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25733] Code and IDLE should catch all compile errors.

[Terry J. Reedy]

Terry J. Reedy added the comment:

How to cause
SyntaxError -- obvious
NameError -- ?, already caught in code module
OverflowError-- ?, already caught in code module
SystemError  - 22 nested for loops ('deeply nested blocks')
TypeError -- chr(0), 2.7
ValueError -- chr(0), 3.x; bytes(0), 2.7

--
components: +Library (Lib) -Interpreter Core
title: Idle does not handle non-standard compile errors -> Code and IDLE should 
catch all compile errors.
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: for / while else doesn't make sense

[Gregory Ewing]

Steven D'Aprano wrote:

I have a 2000 inch monitor, and by using a narrow proportional
font set to 5pt, I can display the entire Python standard library including
tests on screen at once. Then it's just a matter of using my trusty 4"
reflecting telescope to zoom in on any part of the screen I like.


You may think you're joking, but this sounds remarkably
similar to microfiche...

http://www.wisegeek.org/what-is-microfiche.htm

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

On Thursday, June 9, 2016 at 1:36:56 AM UTC-4, Lawrence D’Oliveiro wrote:
> On Wednesday, June 8, 2016 at 10:39:00 PM UTC+12, rocky wrote:
> 
> > In addition to the example programs which give the classic arithmetic
> > expression evaluator, I now include the beginnings of a full Python 2.6
> > language.
> 
> Does anybody bother with LR(k) parsers any more?

I don't understand. Python itself I think does. And By the way, SPARK is an 
Early-Algorithm parser [1], so it is a little more general than LL or LR 
parsers. 

[1] https://en.wikipedia.org/wiki/Earley_parser
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: i'm a python newbie & wrote my first script, can someone critique it?

[Joel Goldstick]

On Fri, Jun 10, 2016 at 6:52 PM, mad scientist jr
 wrote:
> Is this group appropriate for that kind of thing?
> (If not sorry for posting this here.)
>
> So I wanted to start learning Python, and there is s much information 
> online, which is a little overwhelming. I really learn best from doing, 
> especially if it's something actually useful. I needed to create a bunch of 
> empty folders, so I figured it was a good exercise to start learning Python.
> Now that it's done, I am wondering what kind of things I could do better.
> Here is the code:
>
> # WELCOME TO MY FIRST PYTHON SCRIPT!
> # FIRST OF ALL, IT ***WORKS***!!! YAY!
> # I AM A VBA AND JAVASCRIPT PROGRAMMER,
> # SO IT IS PROBABLY NOT VERY "PYTHONIC",
> # SO PLEASE FEEL FREE TO TEAR THE SCRIPT A NEW ONE
> # AFTER YOU ARE SHOCKED BY THIS BAD CODE,
> # ALL I ASK IS THAT IF YOU THINK SOMETHING IS BAD,
> # PLEASE POST AN EXAMPLE OF THE "CORRECT" WAY OF DOING IT
> # COMING FROM VBA, I KNOW A LITTLE OOP,
> # BUT NOT C++ C# JAVA STUFF LIKE "INTERFACES" AND "ABSTRACT BASE CLASSES"
> # SO IF YOU GET INTO SUCH TERMINOLOGY MY EYES MIGHT START TO GLAZE OVER
> # UNLESS YOU CARE TO EXPLAIN THAT TOO
>
>
Ditch the uppercase comments.
 

> # GLOBAL VALUES
> sForPythonVersion="3"
> #sFolderPathTemplate = "c:\\myscripts\\MP3 Disc "
> sFolderPathTemplate = "c:\\temp\\MP3 Disc "
> iFromCount = 1
> iToCount = 250
> iCountWidth = 3
>
Python has guidelines for naming things.  Don't use camel case.  Use
lower case and put underscores between words.  Globals are a bad idea.
If you don't know why, they are even a worse idea
> 
> # SUPPORT FUNCTIONS
>
> # 
> //
> def is_string(myVar): # is_string IS MORE READABLE THAN isinstance (PLAIN 
> ENGLISH!)
> #PYTHON 3 IS NOT LIKING THIS: return ( isinstance(myVar, str) or 
> isinstance(myVar, unicode) )
> #PYTHON 3 IS NOT LIKING THIS: return isinstance(myVar, basestr)
> return isinstance(myVar, str)

I'm not sure why you want to test for type.
>
> # 
> //
> # THIS IS SOME SAMPLE FUNCTION FROM A BOOK I AM READING "PYTHON IN EASY STEPS"
> def strip_one_space(s):
> if s.endswith(" "): s = s[:-1]
> if s.startswith(" "): s = s[1:]
> return s

This may be from some book, but it could be done:
s.strip()


>
> # 
> //
> def get_exact_python_version():
> import sys
> sVersion = ".".join(map(str, sys.version_info[:3]))
> sVersion = sVersion.strip()
> return sVersion
>
> # 
> //
> # TO DO: RETURN TO THE LEFT OF FIRST "."
> def get_python_version():
> sVersion = get_exact_python_version()
> return sVersion[0:1]
>
> # 
> //
> # CHECK PYTHON VERSION, IF IT'S WRONG THEN GRACEFULLY EXIT BEFORE IT BLOWS UP
> # (DAMN, PYTHON 2.x STILL COMPLAINS WITH SYNTAX ERRORS!!)
> # MAYBE THIS COULD STILL BE USEFUL FOR CHECKING THE SUB-VERSION, IN THAT CASE
> # TO DO: MORE GRANULAR CHECK, EG IF VERSION >= 3.5.0
> def exit_if_wrong_python_version(sRightVersion):
> import os
> sCurrentVersion = get_python_version()
> if (sCurrentVersion != sRightVersion):
> print("" +
>   "Wrong Python version (" +
>   sCurrentVersion +
>   "), this script should be run using Python " +
>   sRightVersion +
>   ".x. Exiting..."
>   )
> os._exit(0)
>
> # 
> //
> def get_script_filename():
> import os
> return os.path.basename(__file__)
>
> # 
> //
> def get_timestamp():
> import datetime
> return datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d 
> %H:%M:%S')
>
> # 
> 

Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

On Friday, June 10, 2016 at 4:33:28 AM UTC-4, Robin Becker wrote:
> On 08/06/2016 19:32, rocky wrote:
> ..
> >
> > Sorry that should have been 1998 which would make more sense for the 7th  
> > conference if the 1st one was around 2001. I've corrected the date in [1] 
> > https://pypi.python.org/pypi/spark_parser/1.3.0
> >
> > The automated tests in the package just don't  catch stuff like this. But 
> > I'm sure there are also other mistakes as well in there so feel free to let 
> > me know.
> >
> not a big deal; I like the spark parser :)
> -- 
> Robin Becker

It is cool for how small in code size, how general, (more general than LL or 
LR), and how fast it works for well-crafted grammars. Surprisingly little was 
needed to make it work on Python 3 (as well as addition on Python 2).

It has however long been neglected. John stopped working on it around 2002 or 
so without having put it on Pypy. Since it wasn't it's own package, so you had 
to find some other project (with possibly varying versions of the program) and 
copy the spark.py file from that or download the "0.7 pre-alpha" version of 
2002 from John's website and possibly modify it if you have Python 3. 

Ok. So I've got that addressed by putting it in pypy. 

But the other basic area of neglect as documentation. John's paper I find 
woefully inadequate. Most of the projects on John's site that say they use 
SPARK are now dead links. And using the various uncompyle/decompyle programs as 
a tutorial for the parser is a bit whacky too as these are very complicated, 
and not intended as a tutorial.

I've only started to address this by putting together some simple examples. And 
on http://github.com/rocky/python-spark there is now a wiki to try to address 
some of this as well. In the wiki for http://github.com/rocky/python-uncompyle6 
I've extracted the documentation for the table-driven formatting, but more is 
needed here as well.

So if you or other like this and know to to use it, please consider beefing up 
the wiki. Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: i'm a python newbie & wrote my first script, can someone critique it?

[Marc Brooks]

The structure of your program is really not that Pythonic.  I'd recommend
you take a look at PEP 8.

https://www.python.org/dev/peps/pep-0008/

It's not perfect, but it's a good start to get a feel for how to structure
your Python work.

Marc

On Fri, Jun 10, 2016 at 7:05 PM, Christopher Reimer <
christopher_rei...@icloud.com> wrote:

>
>
> Sent from my iPhone
>
> > On Jun 10, 2016, at 3:52 PM, mad scientist jr <
> mad.scientist...@gmail.com> wrote:
> > .
> > Now that it's done, I am wondering what kind of things I could do better.
>
> This is Python, not BASIC. Lay off on the CAP LOCK key and delete all the
> comments and separation blocks. No one is going to find your code in all
> that noise.
>
> Chris R.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to solve memory

[MRAB]

On 2016-06-11 00:31, meInvent bbird wrote:

it is quite ridiculous,
this time i am sure that i put correct indentation
and add a else statement to make sure recursive call inside the
if statement , it still memory error,

where is the memory error?

is itertools.combinations so big for the list?


[snip]

How long is initlist?

When I ran the code, it said over 100_000 items.

How many combinations would there be?

Over 10_000_000_000.

That's how long the list 'b' would be.

You'll need 10s of gigabytes of memory and a lot of patience!

--
https://mail.python.org/mailman/listinfo/python-list

[issue27263] Tkinter sets the HOME environment variable, breaking scripts

[Terry J. Reedy]

Terry J. Reedy added the comment:

As IDLE maintainer, this annoys me also. IDLE and in particular test_idle 
necessarily change the environment by calling tkinter.Tk().  When run under 
test.regrtest, test_idle gets slapped with a warning, and in 3.6, gets labelled 
a failure, even when it passes.

This issue cuts the other way too.  IDLE works for months and then stops 
working.  User posts "IDLE stopped working' on Stackoverflow.  In at least some 
cases, user installed a program that installed ab incompatible tcl/tk and set 
TCL_LIBRARY (which Python does not).  tk and tkinter get trapped by what for us 
is a bad setting.

Serhiy, could the _tkinter startup code recover from this and revert to using 
the one we installed?  Should I open a separate issue?

--
components:  -IDLE
nosy: +serhiy.storchaka, terry.reedy
title: IDLE sets the HOME environment variable breaking scripts -> Tkinter sets 
the HOME environment variable, breaking scripts

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27290] Turn heaps library into a more OOP data structure?

[James Lu]

James Lu added the comment:

Even a wrapper class would be helpful, it's simply more pythonic.

On Fri, Jun 10, 2016 at 6:02 PM, Raymond Hettinger 
wrote:

>
> Raymond Hettinger added the comment:
>
> The main reason is that there would be very little benefit.  Lists are a
> very efficient data structure and the heap manipulations are also very
> cheap.
>
> --
> assignee:  -> rhettinger
> nosy: +rhettinger
> resolution:  -> rejected
> status: open -> closed
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: i'm a python newbie & wrote my first script, can someone critique it?

[Christopher Reimer]

Sent from my iPhone

> On Jun 10, 2016, at 3:52 PM, mad scientist jr  
> wrote:
> . 
> Now that it's done, I am wondering what kind of things I could do better.

This is Python, not BASIC. Lay off on the CAP LOCK key and delete all the 
comments and separation blocks. No one is going to find your code in all that 
noise.

Chris R.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27223] _read_ready and _write_ready should respect _conn_lost

[Łukasz Langa]

Changes by Łukasz Langa :


--
stage:  -> patch review
title: _ready_ready and _write_ready should respect _conn_lost -> _read_ready 
and _write_ready should respect _conn_lost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23693] timeit accuracy could be better

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: how to solve memory

[meInvent bbird]

it is quite ridiculous, 
this time i am sure that i put correct indentation
and add a else statement to make sure recursive call inside the 
if statement , it still memory error, 

where is the memory error?

is itertools.combinations so big for the list?


  if deep == maxx:
   if deep > 0:
b = []
for aaa,bbb in itertools.combinations(initlist, 2): 
 b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
print("deep=")
print(deep)
path2 = DFS(b, deep-1, maxx, sourceoperators, path)
path.append(path2)
   else:
print ""
  else:
   if deep > 0:
for aaa,bbb in itertools.combinations(initlist, 2):
 b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
print("deep=")
print(deep)
path2 = DFS(b, deep-1, maxx, sourceoperators, path)
path.append(path2)
   else:
print ""
 return path


On Saturday, June 11, 2016 at 12:26:55 AM UTC+8, John Gordon wrote:
> In <8a1c372e-bd6c-4923-8ae1-8f129ec74...@googlegroups.com> meInvent bbird 
>  writes:
> 
> > > I already responded to your earlier post about this program.  Did you
> > > read it?
> 
> > sorry i missed your reply post, 
> > i find post by me and search your name, no this name
> > is your name changed in previous post?
> 
> My comment was that the recursive calls weren't indented in the
> "if deep > 0" block, therefore DFS was being called infinitely with smaller
> and smaller values of deep.  But it appears you have fixed that issue.
> 
> -- 
> John Gordon   A is for Amy, who fell down the stairs
> gor...@panix.com  B is for Basil, assaulted by bears
> -- Edward Gorey, "The Gashlycrumb Tinies"
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27256] email header indentation destroyed

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
stage:  -> test needed
title: header indentation destroyed -> email header indentation destroyed
type:  -> behavior
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27233] Missing documentation for PyOS_FSPath

[Jelle Zijlstra]

Jelle Zijlstra added the comment:

Brett Cannon added documentation for this function in another issue.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27238] Bare except: usages in turtle.py

[Terry J. Reedy]

Terry J. Reedy added the comment:

Add a comment to indicate that bare exception is intentional and briefly why.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: how to solve memory

[meInvent bbird]

i put final part of code inside one more space 
and add constraint deep > 0 
but still memory error 

and print the deep number to see, it do not run infinity again 


V6 is just my superstitution, because V is disable 6 is separation 

i run your version directly, it has key = '0' error 

M1 = {}
M2 = {}
M3 = {}
M4 = {}
M5 = {}
V6 = {}
M1['00']=0
M1['01']=1
M1['02']=1
M1['10']=2
M1['11']=2
M1['12']=1
M1['20']=1
M1['21']=1
M1['22']=1
M2['00']=0
M2['01']=1
M2['02']=1
M2['10']=1
M2['11']=2
M2['12']=2
M2['20']=1
M2['21']=1
M2['22']=1
M3['00']=0
M3['01']=1
M3['02']=1
M3['10']=0
M3['11']=1
M3['12']=2
M3['20']=0
M3['21']=2
M3['22']=2
M4['00']=0
M4['01']=1
M4['02']=0
M4['10']=1
M4['11']=2
M4['12']=2
M4['20']=0
M4['21']=1
M4['22']=1
M5['00']=0
M5['01']=1
M5['02']=1
M5['10']=2
M5['11']=1
M5['12']=2
M5['20']=0
M5['21']=2
M5['22']=1
V6['00']=1
V6['01']=1
V6['02']=2
V6['10']=1
V6['11']=1
V6['12']=1
V6['20']=2
V6['21']=2
V6['22']=2
MM = {}
MM[0] = M1
MM[1] = M2
MM[2] = M3
MM[3] = M4
MM[4] = M5
MM[5] = V6
m = 3
b = [str(i)+str(j)+str(k) for i in range(m) for j in range(m) for k in range(m)]
import itertools
deep = 3
final = []
mylist = [MM[i] for i in range(0,7-1)]
b = [str(i)+str(j)+str(k) for i in range(m) for j in range(m) for k in range(m)]
def DFS(b, deep, maxx, sourceoperators, path):
 initlist = []
 if deep > 0:
  print("deep=", deep)
  for aa,bb in itertools.combinations(sourceoperators, 2):
   print(aa,bb)
   if deep == maxx:
finalresult = []
op1xy = [aa[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
op1yz = [aa[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
op1xz = [aa[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
op2xy = [bb[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
op2yz = [bb[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
op2xz = [bb[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
if sum(op1xy) == 54:
  path.append([(deep, aa, "xy")])
if sum(op1yz) == 54:
  path.append([(deep, aa, "yz")])
if sum(op1xz) == 54:
  path.append([(deep, aa, "xz")])
if sum(op2xy) == 54:
  path.append([(deep, bb, "xy")])
if sum(op2yz) == 54:
  path.append([(deep, bb, "yz")])
if sum(op2xz) == 54:  
  path.append([(deep, bb, "xz")])
initlist.append(op1xy)
initlist.append(op1yz)
initlist.append(op1xz)
initlist.append(op2xy)
initlist.append(op2yz)
initlist.append(op2xz)
   else:
level = []
for j in range(len(b)):
 op1xy = [aa[b[j][i]] for i in range(len(b[j]))]
 op2xy = [bb[b[j][i]] for i in range(len(b[j]))]
 if sum(op1xy) == 54:
  path.append([(deep, aa, "xy")])
 if sum(op2xy) == 54:
  path.append([(deep, bb, "xy")])
 level.append(op1xy)
 level.append(op2xy)
 initlist.append(op1xy)
 initlist.append(op2xy)
  if deep == maxx:
   if deep > 0:
b = []
for aaa,bbb in itertools.combinations(initlist, 2): 
 b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
print("deep=")
print(deep)
path2 = DFS(b, deep-1, maxx, sourceoperators, path)
path.append(path2)
  else:
   if deep > 0:
for aaa,bbb in itertools.combinations(initlist, 2):
 b.append([str(i)+str(j) for i,j in zip(aaa, bbb)])
print("deep=")
print(deep)
path2 = DFS(b, deep-1, maxx, sourceoperators, path)
path.append(path2)
 return path

path = []
mresult = DFS(b, 2, 2, mylist, path)



On Friday, June 10, 2016 at 10:08:44 PM UTC+8, Steven D'Aprano wrote:
> On Fri, 10 Jun 2016 10:58 am, meInvent bbird wrote:
> 
> [snip unreadable code]
> 
> I just ran your code, and it almost crashed my computer. I think it is quite
> rude to post code without an explanation of what the problem is.
> 
> 
> I'm afraid that your code is virtually unreadable to me. It is too verbose,
> the indentation is awful (single space indents is impossible for me to
> track by eye) and it makes too much work out of simple operations. And none
> of the variable names mean anything.
> 
> I've started to simplify and clean the code, and got to the following, which
> I hope you will agree is easier to read and more compact:
> 
> 
> 
> import itertools
> M1 = {'00': 0, '01': 2, '02': 1, '10': 1, '11': 1, 
>   '12': 1, '20': 1, '21': 1, '22': 2}
> M2 = {'00': 0, '01': 1, '02': 1, '10': 1, '11': 1, 
>   '12': 1, '20': 1, '21': 1, '22': 1}
> M3 = {'00': 2, '01': 2, '02': 2, '10': 0, '11': 2, 
>   '12': 1, '20': 0, '21': 1, '22': 2}
> M4 = {'00': 1, '01': 2, '02': 1, '10': 2, '11': 2, 
>   '12': 2, '20': 0, '21': 1, '22': 2}
> M5 = {'00': 0, '01': 1, '02': 1, '10': 0, '11': 2, 
>   '12': 1, '20': 0, '21': 1, '22': 1}
> # Why V instead of M?
> V6 = {'00': 1, '01': 1, '02': 2, '10': 1, '11': 2, 
>   '12': 1, '20': 1, '21': 2, '22': 2}
> 
> MM = {0: M1, 1: M2, 2: M3, 3: M4, 4: M5, 5: V6}
> m = 3
> b = [str(i)+str(j)+str(k) for i in range(m) 
>  for j in range(m) for k in range(m)]
> mylist = [MM[i] for i in range(6)]
> 
> def DFS(b, deep, maxx, sourceoperators, path):
> initlist = []
> if deep > 0:

[issue16484] pydoc generates invalid docs.python.org link for xml.etree.ElementTree and other modules

[Terry J. Reedy]

Terry J. Reedy added the comment:

Those are the only failures on 2 of the 4 buildbots running gui tests.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

i'm a python newbie & wrote my first script, can someone critique it?

[mad scientist jr]

Is this group appropriate for that kind of thing? 
(If not sorry for posting this here.)

So I wanted to start learning Python, and there is s much information 
online, which is a little overwhelming. I really learn best from doing, 
especially if it's something actually useful. I needed to create a bunch of 
empty folders, so I figured it was a good exercise to start learning Python. 
Now that it's done, I am wondering what kind of things I could do better. 
Here is the code: 

# WELCOME TO MY FIRST PYTHON SCRIPT!
# FIRST OF ALL, IT ***WORKS***!!! YAY!
# I AM A VBA AND JAVASCRIPT PROGRAMMER, 
# SO IT IS PROBABLY NOT VERY "PYTHONIC", 
# SO PLEASE FEEL FREE TO TEAR THE SCRIPT A NEW ONE
# AFTER YOU ARE SHOCKED BY THIS BAD CODE, 
# ALL I ASK IS THAT IF YOU THINK SOMETHING IS BAD, 
# PLEASE POST AN EXAMPLE OF THE "CORRECT" WAY OF DOING IT
# COMING FROM VBA, I KNOW A LITTLE OOP,
# BUT NOT C++ C# JAVA STUFF LIKE "INTERFACES" AND "ABSTRACT BASE CLASSES"
# SO IF YOU GET INTO SUCH TERMINOLOGY MY EYES MIGHT START TO GLAZE OVER
# UNLESS YOU CARE TO EXPLAIN THAT TOO


# GLOBAL VALUES
sForPythonVersion="3"
#sFolderPathTemplate = "c:\\myscripts\\MP3 Disc "
sFolderPathTemplate = "c:\\temp\\MP3 Disc "
iFromCount = 1
iToCount = 250
iCountWidth = 3


# SUPPORT FUNCTIONS

# 
//
def is_string(myVar): # is_string IS MORE READABLE THAN isinstance (PLAIN 
ENGLISH!)
#PYTHON 3 IS NOT LIKING THIS: return ( isinstance(myVar, str) or 
isinstance(myVar, unicode) )
#PYTHON 3 IS NOT LIKING THIS: return isinstance(myVar, basestr)
return isinstance(myVar, str)

# 
//
# THIS IS SOME SAMPLE FUNCTION FROM A BOOK I AM READING "PYTHON IN EASY STEPS"
def strip_one_space(s):
if s.endswith(" "): s = s[:-1]
if s.startswith(" "): s = s[1:]
return s

# 
//
def get_exact_python_version():
import sys
sVersion = ".".join(map(str, sys.version_info[:3]))
sVersion = sVersion.strip()
return sVersion

# 
//
# TO DO: RETURN TO THE LEFT OF FIRST "."
def get_python_version():
sVersion = get_exact_python_version()
return sVersion[0:1]

# 
//
# CHECK PYTHON VERSION, IF IT'S WRONG THEN GRACEFULLY EXIT BEFORE IT BLOWS UP
# (DAMN, PYTHON 2.x STILL COMPLAINS WITH SYNTAX ERRORS!!)
# MAYBE THIS COULD STILL BE USEFUL FOR CHECKING THE SUB-VERSION, IN THAT CASE
# TO DO: MORE GRANULAR CHECK, EG IF VERSION >= 3.5.0
def exit_if_wrong_python_version(sRightVersion):
import os
sCurrentVersion = get_python_version()
if (sCurrentVersion != sRightVersion):
print("" +
  "Wrong Python version (" +
  sCurrentVersion +
  "), this script should be run using Python " +
  sRightVersion +
  ".x. Exiting..."
  )
os._exit(0)

# 
//
def get_script_filename():
import os
return os.path.basename(__file__)

# 
//
def get_timestamp():
import datetime
return datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d 
%H:%M:%S')

# 
//

def create_folder(sPath):
import os
import errno
#if not os.path.exists(directory):
#os.makedirs(directory)
try:
os.makedirs(sPath, exist_ok=True)
except OSError as exception:
#if exception.errno != errno.EEXIST:
print("ERROR #" + str(exception.errno) + "IN makedirs")
raise

# 

[issue24750] IDLE: Cosmetic improvements for main window

[Terry J. Reedy]

Terry J. Reedy added the comment:

Applied as 288e97680d84 (with # -> * typo): first ttk widgets.
Issue *24750: Switch all scrollbars in IDLE to ttk versions.
Where needed, add minimal tests to cover changes.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24759] Idle: require tk 8.5 and ttk widgets, and drop unneeded code.

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21386] ipaddress.IPv4Address.is_global not implemented

[Berker Peksag]

Berker Peksag added the comment:

Here is an updated patch that uses my proposal in msg219017. Since this is 
already documented at 
https://docs.python.org/3.5/library/ipaddress.html#ipaddress.IPv4Address.is_global
 I think it should also go into 3.5.

--
versions: +Python 3.6
Added file: http://bugs.python.org/file43337/issue21386_v3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27272] random.Random should not read 2500 bytes from urandom

[Raymond Hettinger]

Raymond Hettinger added the comment:

> But I'll be happiest if nothing changes here (given that Guido ruled
> yesterday that Python's current urandom() implementation has to be 
> reverted to once again match Linux's non-blocking urandom() behavior).

With urandom() behavior restored, can we close this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27290] Turn heaps library into a more OOP data structure?

[Raymond Hettinger]

Raymond Hettinger added the comment:

The main reason is that there would be very little benefit.  Lists are a very 
efficient data structure and the heap manipulations are also very cheap.

--
assignee:  -> rhettinger
nosy: +rhettinger
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3982] support .format for bytes

[Nick Coghlan]

Nick Coghlan added the comment:

The core problem with the idea of adding bytes.format to Python 3 is that the 
real power of str.format actually lies in the extensible __format__ protocol 
and the associated format() builtin, as those rely heavily on text-specific 
assumptions.

I interpreted Amber's comments at the language summit as referring more to our 
changing tune regarding mod formatting from:

- mod formatting is deprecated, use brace formatting instead; to
- they're both fully supported, neither is deprecated; to
- use brace formatting for text data, mod formatting for binary data

Folks that followed our original "stop using mod formatting" guidance thus 
needed to change course when it became our recommended technique for formatting 
binary data.

Since we now know format() and __format__ aren't suitable for binary data (PEP 
361 originally included it, and it got dropped as we kept finding awkward 
corner cases), that means any new binary formatting proposal needs to explain:

- how it compares to existing serialisation techniques (mod-formatting, the 
struct module, text-formatting+encoding, etc)
- why it needs to be a builtin method or function rather than a new 
serialisation module

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27290] Turn heaps library into a more OOP data structure?

[James Lu]

New submission from James Lu:

The heapq library uses a list or other mutable sequence time to represent a 
heap. Since Python is a highly OOP language, why not make heaps their own data 
type?

--
components: Library (Lib)
messages: 268159
nosy: James.Lu
priority: normal
severity: normal
status: open
title: Turn heaps library into a more OOP data structure?
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27186] add os.fspath()

[Brett Cannon]

Changes by Brett Cannon :


--
stage: resolved -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27186] add os.fspath()

[Brett Cannon]

Changes by Brett Cannon :


--
resolution: fixed -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27038] Make os.DirEntry exist

[Brett Cannon]

Changes by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27186] add os.fspath()

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5a62d682636e by Brett Cannon in branch 'default':
Issue #27186: Add os.PathLike support to DirEntry
https://hg.python.org/cpython/rev/5a62d682636e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: movie from pictures

[alex wright]

I find shlex.split to be most useful to make my arguments a list in these
cases.
On Jun 9, 2016 3:28 PM, "MRAB"  wrote:

> On 2016-06-09 19:58, Peter Otten wrote:
>
>> Nev wrote:
>>
>> Thank you for your reply. I tried something like this in python code:
>>>
>>> from subprocess import call
>>> call(["ffmpeg -framerate 4/1 -start_number 1 -i
>>> C:\\Projects\\data2\\img_%05d.png -c:v libx264 -r 30 -pix_fmt yuv420p
>>> C:\\Projects\\data2\\movie.mp4"])
>>>
>>> But it did not work. I get FileNotFoundError: [WinError 2] The system
>>> cannot find the file specified..
>>>
>>
>> You have to pass the command-line arguments as separate items in the list:
>>
>> call(["ffmpeg",
>>   "-framerate", "4/1",
>>   "-start_number", "1",
>>   "-i", "C:\\Projects\\data2\\img_%05d.png",
>>   ... # and so on
>>   ])
>>
>> You should also give it the full path of ffmpeg.
>
> On the other hand, in-loop solution would be more preferable since it lets
>>> me to use variable names of the images and paths..
>>>
>>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fast dictionary initialization

[Random832]

pOn Fri, Jun 10, 2016, at 17:07, maurice wrote:
> Hi. If I have already a list of values, let's call it valuesList and the
> keysList, both same sized lists, what is the easiest/quickest way to
> initialize a dictionary with those keys and list, in terms of number of
> lines perhaps?
> 
> example:
> valuesList = [1,2,3]
> keysList   = ['1','2','3']
> 
> So the dictionary can basically convert string to int:
> 
> dictionary = {'1':1, '2':2, '3':3}

dict(zip(keysList, valuesList))
-- 
https://mail.python.org/mailman/listinfo/python-list

fast dictionary initialization

[maurice]

Hi. If I have already a list of values, let's call it valuesList and the 
keysList, both same sized lists, what is the easiest/quickest way to initialize 
a dictionary with those keys and list, in terms of number of lines perhaps?

example:
valuesList = [1,2,3]
keysList   = ['1','2','3']

So the dictionary can basically convert string to int:

dictionary = {'1':1, '2':2, '3':3}

Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue3982] support .format for bytes

[Derek Wilson]

Derek Wilson added the comment:

Gregory - I'm glad that you're willing to consider this again. It still is a 
constant issue for me, and .format with variable width fields in binary 
protocols is so the right tool for the job. If there is anything I can do to 
help get this added to 3.6 let me know. The forward/backward compatibility 
issue is secondary to me to the flexibility gained from having .format 
available for bytes.

Also padding with null bytes that don't get converted would be awesome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26800] Don't accept bytearray as filenames part 2

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch that deprecates support of general bytes-like objects in os 
functions.

--
keywords: +patch
stage: needs patch -> patch review
Added file: 
http://bugs.python.org/file43336/path_converter-deprecate-buffer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27289] test_ftp_timeout fails with EOFError

[Berker Peksag]

Berker Peksag added the comment:

Here is a patch to skip the test.

--
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file43335/issue27289.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27289] test_ftp_timeout fails with EOFError

[Berker Peksag]

New submission from Berker Peksag:

>From 
>http://buildbot.python.org/all/builders/x86-64%20Ubuntu%2015.10%20Skylake%20CPU%203.5/builds/533/steps/test/logs/stdio

==
ERROR: test_ftp_timeout (test.test_urllib2net.TimeoutTest)
--
Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 1471, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 1493, in connect_ftp
persistent=False)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 2318, in __init__
self.init()
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 2327, in init
self.ftp.connect(self.host, self.port, self.timeout)
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 156, in connect
self.welcome = self.getresp()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 235, in getresp
resp = self.getmultiline()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 221, in getmultiline
line = self.getline()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 209, in getline
raise EOFError
EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/test/test_urllib2net.py",
 line 323, in test_ftp_timeout
u = _urlopen_with_retry(self.FTP_HOST, timeout=60)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/test/test_urllib2net.py",
 line 27, in wrapped
return _retry_thrice(func, exc, *args, **kwargs)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/test/test_urllib2net.py",
 line 23, in _retry_thrice
raise last_exc
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/test/test_urllib2net.py",
 line 19, in _retry_thrice
return func(*args, **kwargs)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 163, in urlopen
return opener.open(url, data, timeout)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 466, in open
response = self._open(req, data)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 484, in _open
'_open', req)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 444, in _call_chain
result = func(*args)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 1489, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 1471, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 1493, in connect_ftp
persistent=False)
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 2318, in __init__
self.init()
  File 
"/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/urllib/request.py",
 line 2327, in init
self.ftp.connect(self.host, self.port, self.timeout)
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 156, in connect
self.welcome = self.getresp()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 235, in getresp
resp = self.getmultiline()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 221, in getmultiline
line = self.getline()
  File "/home/buildbot/buildarea/3.5.intel-ubuntu-skylake/build/Lib/ftplib.py", 
line 209, in getline
raise EOFError
urllib.error.URLError: 

--
components: Tests
messages: 268154
nosy: berker.peksag
priority: normal
severity: normal
stage: needs patch
status: open
title: test_ftp_timeout fails with EOFError
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15468] Edit docs to hide hashlib.md5()

[Berker Peksag]

Berker Peksag added the comment:

The looks good to me in general, but I'd suggest to refer to the new 
algorithms_guaranteed and algorithms_available attributes in the following 
paragraph:

+:func:`sha512`. The :func:`md5` is typically available, but will be 
missing if
+Python has been compiled in FIPS-compliant mode. If hashlib was compiled 
with
+OpenSSL support then additional algorithms may also be available depending
+upon the OpenSSL library that Python uses on your platform. OpenSSL
+implementations of the builtin algorithms are used if available.

--
assignee:  -> docs@python
components: +Documentation
nosy: +berker.peksag, docs@python
stage:  -> patch review
type:  -> enhancement
versions: +Python 3.5, Python 3.6 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27288] secrets should use getrandom() on Linux

[Brett Cannon]

Changes by Brett Cannon :


--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27288] secrets should use getrandom() on Linux

[Tim Peters]

Tim Peters added the comment:

It was a primary purpose of `secrets` to be a place where security best 
practices could be implemented, and changed over time, with no concern about 
backward compatibility for people who don't use it.

So if `secrets` needs to supply a class with all the methods of random.Random, 
it should derive its own subclass (or derive from random.SystemRandom, and 
override the only two methods that explicitly invoke _urandom() - everything 
else ends up using .random() or .getrandbits()).

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20900] distutils register command should print text, not bytes repr

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the patch ingrid!

--
assignee: eric.araujo -> 
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20900] distutils register command should print text, not bytes repr

[Roundup Robot]

Roundup Robot added the comment:

New changeset 02824cee7624 by Berker Peksag in branch '3.5':
Issue #20900: distutils register command now decodes HTTP responses correctly
https://hg.python.org/cpython/rev/02824cee7624

New changeset b0be24a2f16c by Berker Peksag in branch 'default':
Issue #20900: Merge from 3.5
https://hg.python.org/cpython/rev/b0be24a2f16c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27288] secrets should use getrandom() on Linux

[Donald Stufft]

Donald Stufft added the comment:

As an additional aside, the documentation of secrets references the 
documentation of random.py for it's secrets.SystemRandom class, however 
random.py docouments random.SystemRandom as using os.urandom.

So the documentation for secrets.SystemRandom should be disentangled from the 
documentation of random.SystemRandom so that it can choose to use the stronger 
randomness function (or random.SystemRandom should be adapted to use the 
stronger randomness function, or both things).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27275] KeyError thrown by optimised collections.OrderedDict.popitem()

[William Pitcock]

William Pitcock added the comment:

It seems to me that calling __contains__() (PySequence_Contains()) isn't 
necessary, as the first and last elements of the list are already known, and 
therefore known to be in the list.  Revising the behaviour of popitem() to 
avoid calling _odict_popkey_hash() seems like it may provide a marginal 
performance benefit as well as fix the problem.  Calling PyObject_DelItem() 
directly on the node should work fine I believe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27288] secrets should use getrandom() on Linux

[Donald Stufft]

New submission from Donald Stufft:

In 3.5.0 and 3.5.1 os.urandom will, where available, use the getrandom() to 
block rather than get insecure random from the urandom pool on Linux. In 3.5.2 
this change is reverted so that os.urandom will return possibly predictable 
random numbers instead of blocking waiting for /dev/urandom to be intialized.

However, secrets.py is a module which is explicitly for getting 
cryptographically safe data for, and reverting that change means that the 
functions in this module are no longer cryptographically safe on Linux if they 
are called early enough in the boot process. Thus, secrets.py should be 
modified so that it no longer uses os.urandom on systems where there is a 
better source of randomness available-- namely getrandom() set to block on 
Linux.

--
messages: 268147
nosy: brett.cannon, dstufft, ned.deily
priority: release blocker
severity: normal
status: open
title: secrets should use getrandom() on Linux
type: security
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27186] add os.fspath()

[Roundup Robot]

Roundup Robot added the comment:

New changeset a5a013ca5687 by Brett Cannon in branch 'default':
Issue #27186: Add os.PathLike support to pathlib.
https://hg.python.org/cpython/rev/a5a013ca5687

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Demur Rumed]

Demur Rumed added the comment:

The patches LGTM & seem to be implementation of follow up ideas outlined in the 
first portion. It'd be good to verify that benchmarks are relatively unaffected

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26243] zlib.compress level as keyword argument

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Regenerated for review.

--
Added file: 
http://bugs.python.org/file43334/zlib_compress_positional_only_data.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26282] Add support for partial keyword arguments in extension functions

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

update patch replaces yet few magic constants.

--
Added file: http://bugs.python.org/file4/wordcode-refactor2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is not just about cleaning (to my eyes current code is not very readable, 
and I read it many times, perhaps more times than any other core developer in 
last months). There are other benefits. Changing jump offsets allows to get rid 
of EXTENDED_ARGs for the part of jump opcodes. Changing lnotab makes it more 
compact and allows the peepholer to optimize the code that it rejects now. 
Refactoring includes the change that decreases memory consumption of the 
peepholer (from 4 bytes per bytecode byte to 2 bytes per bytecode byte). 
Changing jump offsets together with changing f_lasti removes redundant 
multiplications and divisions by 2. Separate changes can complicate some parts 
of code, but next changes removes this complication. Only all changes together 
achieve maximal cleanness.

I think that converting to wordcode is not complete without these changes. I 
approved the wordcode patch only having in mind following changes. It is more 
painless to make all changes in one Python release than break compatibility 
during few releases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26282] Add support for partial keyword arguments in extension functions

[Roundup Robot]

Roundup Robot added the comment:

New changeset 69c0aa8a8185 by Serhiy Storchaka in branch 'default':
Issue #26282: PyArg_ParseTupleAndKeywords() and Argument Clinic now support
https://hg.python.org/cpython/rev/69c0aa8a8185

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20900] distutils register command should print text, not bytes repr

[Berker Peksag]

Berker Peksag added the comment:

issue20900.patch looks good to me. We are already testing output of "python 
-setup.py upload --show-response" in Lib/distutils/tests/test_upload.py so I 
removed test_show_reponse and tweaked test_upload to test the whole line:

-self.assertIn('xyzzy\n', results[-1])
+self.assertEqual(results[-1], 75 * '-' + '\nxyzzy\n' + 75 * '-')

--
nosy: +berker.peksag
versions: +Python 3.5, Python 3.6 -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27275] KeyError thrown by optimised collections.OrderedDict.popitem()

[Xiang Zhang]

Xiang Zhang added the comment:

Raymond, In single threaded case popitem may still fail.

I want to correct my last message that popitem does not fail in this case 
because it calls __getitem__ but instead it calls __contains__[1]. In 
__contains__ it deletes the item since it expires, and finally emit a 
KeyError[2]. Even if it passes __contains__, it will call __getitem__[3].

[1] https://hg.python.org/cpython/file/tip/Objects/odictobject.c#l1115
[2] https://hg.python.org/cpython/file/tip/Objects/odictobject.c#l1135
[3] https://hg.python.org/cpython/file/tip/Objects/odictobject.c#l1119

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27275] KeyError thrown by optimised collections.OrderedDict.popitem()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27275] KeyError thrown by optimised collections.OrderedDict.popitem()

[William Pitcock]

William Pitcock added the comment:

At least in my case, the application is single-threaded.  I don't think this is 
a locking-related issue as the expiringdict test case itself fails which is 
also single-threaded.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27272] random.Random should not read 2500 bytes from urandom

[Tim Peters]

Tim Peters added the comment:

Raymond, while I'm in general agreement with you, note that urandom() doesn't 
deliver "random" bytes to begin with.  A CSPRNG is still a PRNG.

For example, if the underlying urandom() generator is ChaCha20, _it_ has "only" 
512 bits of state.  Seeding the Twister with 2500 bytes from urandom() far 
exceeds the maximum possible entropy that ChaCha20's comparatively tiny 64 
bytes of state can deliver (but, yes, I'm ignoring the possibility that a good 
urandom() implementation may fold in fresh entropy during the time MT sucks out 
those 2500 bytes - nevertheless, MT's state is far larger).

That's why I said earlier I could live with seeding from 128 bytes - twice the 
size of a currently trendy urandom() implementation's state.

But I'll be happiest if nothing changes here (given that Guido ruled yesterday 
that Python's current urandom() implementation has to be reverted to once again 
match Linux's non-blocking urandom() behavior).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27285] Document the deprecation of pyvenv in favor of `python3 -m venv`

[Brett Cannon]

Brett Cannon added the comment:

Documenting the deprecation in 3.5 is fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27285] Document the deprecation of pyvenv in favor of `python3 -m venv`

[Steve Piercy]

Steve Piercy added the comment:

One more.
https://docs.python.org/3/library/venv.html

Should that be updated for 3.5.1 as well as 3.6?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27285] Document the deprecation of pyvenv in favor of `python3 -m venv`

[Steve Piercy]

Steve Piercy added the comment:

See also:
https://github.com/pypa/python-packaging-user-guide/issues/242

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27285] Document the deprecation of pyvenv in favor of `python3 -m venv`

[Steve Piercy]

Steve Piercy added the comment:

I forgot to include the specific URL to update:
https://docs.python.org/3/installing/

There may be other places.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25413] ctypes (libffi) fails to compile on Solaris X86

[Matthias Klose]

Matthias Klose added the comment:

CristiFati, please could you check if the libffi patch is still needed with the 
current upstream libffi?

See https://github.com/libffi/libffi

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Raymond Hettinger]

Raymond Hettinger added the comment:

I really don't think any of this should be done at all.  The current code is 
clean and fast (and to my eyes at least is very readable).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27275] KeyError thrown by optimised collections.OrderedDict.popitem()

[Raymond Hettinger]

Raymond Hettinger added the comment:

I'm wondering if the expiringdict(1) needs to have locked wrappers for the 
inherited methods:

def __delitem__(self, key):
with self.lock:
OrderedDict.__delitem__(self, key)

Otherwise, there is a risk that one thread is deleting a key with no lock held, 
while another thread is running expiringdict.popitem() which holds a lock while 
calling both __getitem__ and del.  If the first thread runs between the two 
steps in the second, the race condition would cause a KeyError.

This might explain why you've observed, '''Replacing use of popitem() with "del 
self[next(OrderedDict.__iter__(self))]" removes the KeyErrors and the structure 
otherwise works fine.'''

(1) https://github.com/mailgun/expiringdict/blob/master/expiringdict/__init__.py

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23693] timeit accuracy could be better

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +gvanrossum, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27272] random.Random should not read 2500 bytes from urandom

[Raymond Hettinger]

Raymond Hettinger added the comment:

With anything less than than full seeding of the 19937-bit state space, we lose 
all the guarantees (proved properties) documented at 
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/ARTICLES/mt.pdf .  It is very 
nice to be able to rely on those properties and I don't think we should abandon 
them because glibc currently has lower standards (the trend is towards making 
seeding stronger rather than weaker).

"Good enough" for random number generators is in the eye of the beholder.  If 
your agenda is to reduce the number of bytes extracted from urandom(), it is 
easy to not care how well the MT is seeded, but there are people who do care 
(or should care).

Note that sampling and shuffling both eat entropy like crazy.  Currently, we 
have enough to shuffle 2000 elements.   With 128 bits, we run out at 34 
elements, not even enough to shuffle a deck of cards and have each permutation 
as a possible outcome.

Over time, the random module has evolved away from "good enough" and 
traded-away speed in return for equi-distribution (i.e. we use _randbelow to 
get a balanced choice over ranges that aren't an exact power-of-two).

We should not allow a regression in quality.  In particular, I object to the 
cavalier assertion, "random.Random doesn't need a very high quality entropy."  
If you're running multiple simulations, this is something you should care 
about.  And to defend against criticism of random samples, it is nice to be 
able to say you were properly seeded (see RFC 1750) and that you have some 
basis for believing that every outcome was equally likely.

--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27286] str object got multiple values for keyword argument

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Since currently generated bytecode is not correct, I think we should update the 
magic number for forcing the regenerating pyc-files.

--
Added file: 
http://bugs.python.org/file43332/BUILD_MAP_UNPACK_WITH_CALL-function_pos2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26213] Document BUILD_*_UNPACK opcodes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

And BUILD_MAP_UNPACK_WITH_CALL.

All these opcodes were added in issue2292 for implementing PEP 448, but are not 
documented even in the PEP.

--
nosy: +Joshua.Landau, neil.g, serhiy.storchaka, twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: how to solve memory

[John Gordon]

In <8a1c372e-bd6c-4923-8ae1-8f129ec74...@googlegroups.com> meInvent bbird 
 writes:

> > I already responded to your earlier post about this program.  Did you
> > read it?

> sorry i missed your reply post, 
> i find post by me and search your name, no this name
> is your name changed in previous post?

My comment was that the recursive calls weren't indented in the
"if deep > 0" block, therefore DFS was being called infinitely with smaller
and smaller values of deep.  But it appears you have fixed that issue.

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue17500] move PC/icons/source.xar to http://www.python.org/community/logos/

[Matthias Klose]

Matthias Klose added the comment:

removed the PC/icons directory.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17500] move PC/icons/source.xar to http://www.python.org/community/logos/

[Roundup Robot]

Roundup Robot added the comment:

New changeset c6c55d1eadc4 by doko in branch '2.7':
- Issue #17500, and https://github.com/python/pythondotorg/issues/945: Remove
https://hg.python.org/cpython/rev/c6c55d1eadc4

New changeset 0c4d525a2f10 by doko in branch '3.5':
- Issue #17500, and https://github.com/python/pythondotorg/issues/945: Remove
https://hg.python.org/cpython/rev/0c4d525a2f10

New changeset a3060775b8ad by doko in branch 'default':
- Issue #17500, and https://github.com/python/pythondotorg/issues/945: Remove
https://hg.python.org/cpython/rev/a3060775b8ad

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27285] Document the deprecation of pyvenv in favor of `python3 -m venv`

[Brett Cannon]

Brett Cannon added the comment:

Basically I changed my mind about adding the code deprecation in 3.5.1 since it 
wasn't fair to people to run into that in a bugfix release. Documenting the 
deprecation, though, can happen in Python 3.5.1 as well as updating the 
documentation of the venv module to only use `python3 -m venv` which I missed 
the first time through.

--
assignee: docs@python -> brett.cannon
title: Deprecate pyvenv in favor of python3 -m venv -> Document the deprecation 
of pyvenv in favor of `python3 -m venv`

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: for / while else doesn't make sense

[Marko Rauhamaa]

Ian Kelly :

> On Jun 10, 2016 6:37 AM, "Marko Rauhamaa"  wrote:
>> If your display can show 1,500 lines at once, that's your limit. Mine
>> shows 70.
>
> I disagree on that point. For a typical-size display, it's a
> reasonable guideline.

The point I'm making is that if I'm accusing you for writing functions
that are too long (they don't fit on my editor window), you can counter
that they fit perfectly well on yours.

No-one has 1,500-line editor windows, I suppose. However, screen sizes
have been going up and some people prefer to turn their screens into
portrait mode (tried it, made my neck hurt). Years back, I had 24 lines
on the screen. As screen sizes have gone up, I've allowed for longer
functions as well. Now I'm at 70. Of course, most functions tend to be
much shorter.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue27286] str object got multiple values for keyword argument

[Raymond Hettinger]

Raymond Hettinger added the comment:

This patch looks correct.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

ANN: bcolz 1.1.0 released!

[Francesc Alted]

==
Announcing bcolz 1.1.0
==

What's new
==

This release brings quite a lot of changes.  After format stabilization
in 1.0, the focus is now in fine-tune many operations (specially queries
in ctables), as well as widening the available computational engines.

Highlights:

* Much improved performance of ctable.where() and ctable.whereblocks().
  Now bcolz is getting closer than ever to fundamental memory limits
  during queries (see the updated benchmarks in the data containers
  tutorial below).

* Better support for Dask; i.e. GIL is released during Blosc operation
  when bcolz is called from a multithreaded app (like Dask).  Also, Dask
  can be used as another virtual machine for evaluating expressions (so
  now it is possible to use it during queries too).

* New ctable.fetchwhere() method for getting the rows fulfilling some
  condition in one go.

* New quantize filter for allowing lossy compression of floating point
  data.

* It is possible to create ctables with more than 255 columns now.
  Thanks to Skipper Seabold.

* The defaults during carray creation are scalars now.  That allows to
  create highly dimensional data containers more efficiently.

* carray object does implement the __array__() special method now. With
  this, interoperability with numpy arrays is easier and faster.

For a more detailed change log, see:

https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst

For some comparison between bcolz and other compressed data containers,
see:

https://github.com/FrancescAlted/DataContainersTutorials

specially chapters 3 (in-memory containers) and 4 (on-disk containers).


What it is
==

*bcolz* provides columnar and compressed data containers that can live
either on-disk or in-memory.  Column storage allows for efficiently
querying tables with a large number of columns.  It also allows for
cheap addition and removal of column.  In addition, bcolz objects are
compressed by default for reducing memory/disk I/O needs. The
compression process is carried out internally by Blosc, an
extremely fast meta-compressor that is optimized for binary data. Lastly,
high-performance iterators (like ``iter()``, ``where()``) for querying
the objects are provided.

bcolz can use numexpr internally so as to accelerate many vector and
query operations (although it can use pure NumPy for doing so too).
numexpr optimizes the memory usage and use several cores for doing the
computations, so it is blazing fast.  Moreover, since the carray/ctable
containers can be disk-based, and it is possible to use them for
seamlessly performing out-of-memory computations.

bcolz has minimal dependencies (NumPy), comes with an exhaustive test
suite and fully supports both 32-bit and 64-bit platforms.  Also, it is
typically tested on both UNIX and Windows operating systems.

Together, bcolz and the Blosc compressor, are finally fulfilling the
promise of accelerating memory I/O, at least for some real scenarios:

http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots

Example users of bcolz are Visualfabriq (http://www.visualfabriq.com/),
and Quantopian (https://www.quantopian.com/):

* Visualfabriq:

  * *bquery*, A query and aggregation framework for Bcolz:
  * https://github.com/visualfabriq/bquery

* Quantopian:

  * Using compressed data containers for faster backtesting at scale:
  * https://quantopian.github.io/talks/NeedForSpeed/slides.html



Resources
=

Visit the main bcolz site repository at:
http://github.com/Blosc/bcolz

Manual:
http://bcolz.blosc.org

Home of Blosc compressor:
http://blosc.org

User's mail list:
bc...@googlegroups.com
http://groups.google.com/group/bcolz

License is the new BSD:
https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt

Release notes can be found in the Git repository:
https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst



  **Enjoy data!**

-- 
Francesc Alted
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/