Re: Friday Filosofical Finking: Import protections

[Chris Angelico]

On Thu, Apr 18, 2019 at 2:32 PM DL Neil  wrote:
>
> On 18/04/19 8:29 AM, Chris Angelico wrote:
> > On Thu, Apr 18, 2019 at 6:21 AM DL Neil  
> > wrote:
> >> Do you bother with exception handling for import statements?
> >> Can we assume that if such a catastrophic error occurs, it is quite
> >> acceptable for the code to fall-over in a tumbling-fumble?
> >
> > I try/except around import statements only if it's possible for the
> > program to recover from the exception. For instance, something that
> ...
>
> User reactions have been covered elsewhere 'here'.

You've given an example of how a user might respond to the exception
traceback. But can you actually offer anything that your program could
do that's BETTER than a traceback? If not, what's the point in
catching the exception?

> > For something that is absolutely required for the program to continue,
> > what would be in your exception handler? Print a message to stderr and
> > exit? That's exactly what not-catching-the-exception is for. I do
> > sometimes annotate the imports, though:
>
> We disagree. Although I hasten to add, if the error is trapped, then
> there should be some added-value over what is system-provided.
>
> Application's log? SysLog? Particularly in a centralised or server-based
> application especially web-services.

Or, you could stick with stderr, which means that the message will go
whereever the message ought to be sent. If you're running a web
service, you should be catching stderr and putting it somewhere.

> > from dataclasses import dataclass # ImportError? Upgrade to Python 3.7
> > or pip install dataclasses
> >
> > If that bombs out, the entire line will get printed, comment and all,
> > and there isn't really anything else that I would want to do with the
> > exception.
> > So I guess the best way to answer your question is with another
> > question: If such a catastrophic error occurs, what ELSE would your
> > code do than fall over? If there's an answer to that question, then
> > sure, catch the ImportError. Otherwise don't.
>
> Kudos! To whom are you addressing such annotations?
>

Whoever might be running the script. I don't particularly discriminate :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Importing module from another subdirectory

[dieter]

Rich Shepard  writes:

> What is the proper syntax to import the model class in the model/
> subdirectory into a tkinter view module, e.g., activities.py? The syntax,
> 'import model as m' fails because it is not in the same subdirectory as the
> importing module.
>
> The program directory tree is:
>
> bustrac/
>README.rst
>bustrac.py*
>controller/
>model/
>scripts/
>views/

Python knows about 2 kinds of "regular" imports:
absolute ones and relative ones. "Absolute" imports
are guided by "sys.path" -- in the simple case, a sequence
of folders containing modules and/or pacakges.
Relative imports are guided in a similar way by the current
packages's "__path__", which typically contains just one element -
the folder from which the current package was loaded.

When you start a script, Python adds the script's directory
to "sys.path". This makes (absolute) imports of other modules/packages in
this directory easy. Your structure, however, puts the infrastructure
elsewhere. I see two options for you:

1. put your scripts directly into "bustrac" (rather than a subdirectory)

2. extend "sys.path" in your scripts to contain the "bustrac" folder (
before you try to import infrastructure modules/packages)

Both options would allow you to import "model" via Python's "regular"
import. Apart from that you can use "importlib" services (--> runtime library
documentation) to import from any location you like.



-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

> "the single-threaded ProcessPoolExecutor test program"

I doubt it is single threaded, the .submit() method appears to spawn a thread 
internally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Filosofical Finking: Import protections

[Cameron Simpson]

On 18Apr2019 16:05, DL Neil  wrote:

On 18/04/19 8:45 AM, MRAB wrote:

On 2019-04-17 21:20, DL Neil wrote:

Do you bother with exception handling for import statements?

Can we assume that if such a catastrophic error occurs, it is quite
acceptable for the code to fall-over in a tumbling-fumble?



[snip]
Catch only what you (well, the script) can fix.

If it needs numpy, but can't import numpy, then when can it do? 
Might as well just let it fail.


I suppose an alternative might be to try to download and install 
numpy and then retry, but what if it can't be downloaded, or the 
installation fails?


No, you have to give up at some point. It's better just to report 
the problem and leave it to the user to fix it.


Ah, but is that not the point - the user cannot fix it (and neither 
should he be allowed to do so: see later comments about Operations 
functions).


I'm missing something here. To me there are usually 2 scenarios where a 
failed import obstructs the user:


- the user is someone like you or I - a person using a programme from 
 source they're fetched - trying to run some programme with a module 
 dependency; here we can reasonably assuming some responsibility for 
 obtaining the missing module; ideally the place we got the programme 
 from might have some pointers


- the user is running some kind of packaged app. With python that tends 
 to be either something obtained via pip from PyPI or a downloaded 
 package, such as a bundle Mac .app or a Windows .msi file or a package 
 from a linux distro. Here the responsibility is with the person who 
 made the package/bundle.


In the latter case the source _should_ have resolvable dependencies. For 
example pip packages can name the modules they require and pip itself 
will fetch those in turn. Bundles apps (.app, .msi etc) tend to include 
the modules within the bundle. Linux distro packages should include 
dependencies as well, and the package programme should fetch those.


All of these should fulfil the requirements at programme install time, 
not at programme run time.


Do you have a third scenario you could detail?


What the user faces (unless we more properly handle the exception) is:

import davids_brains
Traceback (most recent call last):
 File "", line 1, in 
ModuleNotFoundError: No module named 'davids_brains'

You and I will think this fine - and 'here' on the list will even beg 
OPs to give us this much data.


However, despite users who have been known to make other comments 
about my (?lack of) brains, they will tend to offer such helpful advice 
as: "we need the missing module"...!


I think this case is usually the former of the 2 above: the user has 
fetched from somewhere random. That somewhere should at least identify 
what is required, even if the user needs to do some further legwork 
themselves.


I think my stance here is that it is the person installing the 
app/module who needs to sort this, because they know the context in 
which the app/module is being installed. Do we use pip to get stuff? Do 
we use the OS package manager to get stuff? Are we in a managed 
environment where we have to ask the local sysadmins to do this?


The point is that the app/module can't know how to obtain missing 
components. If they didn't come with it, there are many unknowns about 
the environment and plenty of legitimate circumstances where an 
automatic install is infeasible or actually undesirable.


I think that _most_ ImportErrors should not be caught unless the 
recovery is extremely well defined.


While I'm not a fan of the "import settings" scenario (executable 
settings files? scary), I do catch ImportErrors in several places in my 
personal kit. Like Chris I think this is only appropriate where there's 
a very well defined coping mechanism, in particular _not_ assuming one 
can fetch more software.


So:

Python 2/3 imports: personally I try the python 3 import and fall back 
to the python 2 if that fails. But if the python 2 import fails, _that_ 
import error gets out. No "install" style recovery.


Optional modules: I've got a module which supports several possible 
index backends. So I've got some classes along these lines:


 class LMDBIndex(_Index):
   [...]
   @classmethod
   def is_supported(cls):
 ''' Test whether this index class is supported by the Python environment.
 '''
 try:
   import lmdb
 except ImportError:
   return False
 return True

and some code which looks for a working index class:

   for indexname, indexclass in indexclasses:
 if not indexclass.is_supported():
   continue
 ... some other checks ...
 return indexclass
   ...
   raise ValueError(
   "no supported index classes available: tried %r"
   % (indexclasses,))

So: no recovery or autoinstall. It just chooses the first (== most 
preferred) index class and returns it, and raises an exception if none 
is available.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Function to determine list max without itertools

[DL Neil]

On 18/04/19 4:10 PM, Sayth Renshaw wrote:

I have created a function that takes a list as an argument.
Without using itertools I want to compare each item in the list to find the max.

However instead of the max I keep getting the  last item in the list. Where is 
my logic wrong here?

...


Seems like it should work but doesn't.




I'd recommend rethinking your algorithm first, and then thinking about
debugging your actual code. You should be able to find the largest in
a collection without making many many passes over the items - a single
pass should be sufficient.




Most I am finding either use the max function or itertools. Both of which I am 
trying not to use so that I actually do it myself.



Did you understand the advice? Have you simply dropped the first-attempt 
and gone looking for another canned-solution?


Whilst you seem to be on-the-right-track, there is a definite error in 
the algorithm/the code.


What debugging did you perform?

Do you only assume, or can you see example (pertinent) values during 
each "pass" (time through the loop)? If not, why not?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[DL Neil]

On 18/04/19 1:24 PM, Cameron Simpson wrote:

On 17Apr2019 21:45, MRAB  wrote:

On 2019-04-17 21:20, DL Neil wrote:

Do you bother with exception handling for import statements?

[...]

Catch only what you (well, the script) can fix.

If it needs numpy, but can't import numpy, then when can it do? Might 
as well just let it fail.


I'm of this mind too, but...

I suppose an alternative might be to try to download and install numpy 
and then retry, but what if it can't be downloaded, or the 
installation fails?


As an example of what an open ended can of worms attempts recovery might 
be, yeah. How hard do you try? But also, "installation fails": that 
isn't always a clean situation: it can litter the install area with 
partial junk.


But this is also a bad example: it is something an _invoked_ programme 
should never try to do. Except by specific deliberate design and 
request, a running application shouldn't presume it has rights to 
install additional things, or even to try. I have personally (though 
metaphorically) clipped devs across the ear for doing themselves the 
moral equivalent of the above: try thing, then just "sudo try thing" 
when it was forbidden.

...

+1


Installing additional packages is the same as self modifying code: as a 
rule, the admins install packages, not the app.


+1



Sorry, ranting now over.


Not at all, an excellent point which needs to be taken out and 
dusted-off, every now-and-again - particularly for such ear-clipping 
training exercises!


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[DL Neil]

On 18/04/19 8:29 AM, Chris Angelico wrote:

On Thu, Apr 18, 2019 at 6:21 AM DL Neil  wrote:

Do you bother with exception handling for import statements?
Can we assume that if such a catastrophic error occurs, it is quite
acceptable for the code to fall-over in a tumbling-fumble?


I try/except around import statements only if it's possible for the
program to recover from the exception. For instance, something that

...

User reactions have been covered elsewhere 'here'.



For something that is absolutely required for the program to continue,
what would be in your exception handler? Print a message to stderr and
exit? That's exactly what not-catching-the-exception is for. I do
sometimes annotate the imports, though:


We disagree. Although I hasten to add, if the error is trapped, then 
there should be some added-value over what is system-provided.


Application's log? SysLog? Particularly in a centralised or server-based 
application especially web-services.




from dataclasses import dataclass # ImportError? Upgrade to Python 3.7
or pip install dataclasses

If that bombs out, the entire line will get printed, comment and all,
and there isn't really anything else that I would want to do with the
exception.
So I guess the best way to answer your question is with another
question: If such a catastrophic error occurs, what ELSE would your
code do than fall over? If there's an answer to that question, then
sure, catch the ImportError. Otherwise don't.


Kudos! To whom are you addressing such annotations?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[DL Neil]

On 18/04/19 8:44 AM, Grant Edwards wrote:

On 2019-04-17, DL Neil  wrote:


Do you bother with exception handling for import statements?


Sometimes.  There are two cases when I do that:

  1. When the module has different names under Python2 and Python3 and
 the program tries first one, then the other.


Excellent example - and a lot easier than interrogating os.environ (for 
example), ie permission cf forgiveness.




  2. When the program can still do something useful (if perhaps
 feature-limited) without the imported module by substituting
 something else in its place.


Any (publishable) examples?



Most of the code I read, both in books and during code review,
eschews any form of ImportError check. Even data science people who
'clean' every data field towards inclusion/exclusion in the
analysis, take for granted that numpy, scipy, pandas, et al, will be
available to their code.


You've omitted the second thing assumed by the authors: without numpy,
scipy, pandas, et alia the program can do nothing useful.


but... what of the third inherent assumption: that the user(s) will be 
able to handle the situation (discussed in another msg 'here')?




Does such a check seem un-pythonic? [sto] (maybe 'forgiveness cf
permission'?)


It's probably rather unpythonic if you're not going to anything useful
in the exception handler.


Indisputable!



Can we assume that if such a catastrophic error occurs, it is quite
acceptable for the code to fall-over in a tumbling-fumble?


It's certainly OK with me.  I'm not sure why you refer to raising an
exception as "fall-over in a tumbling fumble".  Raising an exception
is the normal way to indicate failure in Python.


Apologies! Evidently a cultural reference that did not export well.

To understand "tumbling fumble" perhaps think of dropping something, 
attempting to catch it with one hand, then having to make a second try 
with the other, and probably failing to intercede before the floor... I 
could have used other terms, but likely to be considered 
'politically-INcorrect'.


No, "tumbling fumble" describes having no exception handling and merely 
allowing the program to "crash".


Thus the basic question: why do we (apparently) so seldom consider the 
possibility of an ImportError?




Does it make a difference if operating in/out of a dev-ops
environment?


I have no idea what "a dev-ops environment means", and I plan on
keeping it that way. :)


I see that you value your sanity. On the other hand you don't seem so 
worried about your physical safety - please note that I'll be hiding 
behind you, should the (ever-so nice) dev-ops lady on one of my projects 
happen to read this...


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Function to determine list max without itertools

[Sayth Renshaw]

wrote:
> >
> >
> > I have created a function that takes a list as an argument.
> > Without using itertools I want to compare each item in the list to find the 
> > max.
> >
> > However instead of the max I keep getting the  last item in the list. Where 
> > is my logic wrong here?
> >
> > def maximum(listarg):
> > items = list(listarg)
> > myMax = 0
> > for index, item in enumerate(items):
> > for otheritem in items[index + 1 :]:
> > if item < otheritem:
> > myMax = otheritem
> > elif item > otheritem:
> > myMax = item
> > else:
> > myMax = myMax
> >
> > Seems like it should work but doesn't.
> >
> 
> I'd recommend rethinking your algorithm first, and then thinking about
> debugging your actual code. You should be able to find the largest in
> a collection without making many many passes over the items - a single
> pass should be sufficient.
> 
> ChrisA

Most I am finding either use the max function or itertools. Both of which I am 
trying not to use so that I actually do it myself.

This one on SO is where I was going https://stackoverflow.com/a/3990826/461887

def maxelements(seq):
''' Return list of position(s) of largest element '''
max_indices = []
if seq:
max_val = seq[0]
for i,val in ((i,val) for i,val in enumerate(seq) if val >= max_val):
if val == max_val:
max_indices.append(i)
else:
max_val = val
max_indices = [i]

return max_indices

This is probably the nicest one but still uses Max.

>>> a=[5,4,3,2,1]
>>> def eleMax(items, start=0, end=None):
... return max(items[start:end])

Thanks 

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[DL Neil]

On 18/04/19 8:45 AM, MRAB wrote:

On 2019-04-17 21:20, DL Neil wrote:

Do you bother with exception handling for import statements?


Can we assume that if such a catastrophic error occurs, it is quite
acceptable for the code to fall-over in a tumbling-fumble?



[snip]
Catch only what you (well, the script) can fix.

If it needs numpy, but can't import numpy, then when can it do? Might as 
well just let it fail.


I suppose an alternative might be to try to download and install numpy 
and then retry, but what if it can't be downloaded, or the installation 
fails?


No, you have to give up at some point. It's better just to report the 
problem and leave it to the user to fix it.


Ah, but is that not the point - the user cannot fix it (and neither 
should he be allowed to do so: see later comments about Operations 
functions).


What the user faces (unless we more properly handle the exception) is:

import davids_brains
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'davids_brains'

You and I will think this fine - and 'here' on the list will even beg 
OPs to give us this much data.


However, despite users who have been known to make other comments about 
my (?lack of) brains, they will tend to offer such helpful advice as: 
"we need the missing module"...!


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

[issue36277] pdb's recursive debug command is not listed in the docs

[Dave Nguyen]

Change by Dave Nguyen :


--
keywords: +patch
pull_requests: +12796
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Filosofical Finking: Import protections

[DL Neil]

On 18/04/19 8:53 AM, Larry Martell wrote:

On 2019-04-17 21:20, DL Neil wrote:

Do you bother with exception handling for import statements?


I often have to do something like this:

try:
 from settings import SITE_WAFER_DIAMETER
except ImportError:
 SITE_WAFER_DIAMETER = 300



That's an interesting application. Most of the multiple input 
configuration options examples (I've seen) start with a base of the 
least-prioritised values, and then 'add' higher-priority configuration 
option-values to that.


In this case we: import the config file, but if it's not available use 
the stated fall-back value(s).


Good one!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

[issue36654] Add example to tokenize.tokenize

[Windson Yang]

New submission from Windson Yang :

> The tokenize() generator requires one argument, readline, which must be a 
> callable object which provides the same interface as the io.IOBase.readline() 
> method of file objects. Each call to the function should return one line of 
> input as bytes.

Add an example like this should be easier to understand:

# example.py
def foo:
pass

# tokenize_example.py
import tokenize
f = open('example.py', 'rb')
token_gen = tokenize.tokenize(f.readline)

for token in token_gen:
# Something like this
# TokenInfo(type=1 (NAME), string='class', start=(1, 0), end=(1, 5), 
line='class Foo:\n')
# TokenInfo(type=1 (NAME), string='Foo', start=(1, 6), end=(1, 9), 
line='class Foo:\n')
# TokenInfo(type=53 (OP), string=':', start=(1, 9), end=(1, 10), 
line='class Foo:\n')
print(token)

--
assignee: docs@python
components: Documentation
messages: 340467
nosy: Windson Yang, docs@python
priority: normal
severity: normal
status: open
title: Add example to tokenize.tokenize
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36646] os.listdir() got permission error in Python3.6 but it's fine in Python2.7

[Steve Dower]

Steve Dower  added the comment:

This is weird.

Are you able to tell us anything about how L: is mounted or what it's pointing 
at? Do you know if it's SMB or similar, or if there are symlinks or similar?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14817] pkgutil.extend_path has no tests

[Windson Yang]

Windson Yang  added the comment:

I added some tests in the PR. Actually, there are some tests for extend_path 
already (see 
https://github.com/python/cpython/blob/master/Lib/test/test_pkgutil.py#L235). 
However, I didn't test every line of the code in the extend_path function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14817] pkgutil.extend_path has no tests

[Windson Yang]

Change by Windson Yang :


--
keywords: +patch
pull_requests: +12795
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

So:

#1 we've a bug: the single-threaded ProcessPoolExecutor test program should 
work 100% reliably - it does not

#2 we've a cause: ProcessPoolExecutor is implemented internally using an 
unfortunate combination of fork and threads, this is causing the deadlock

#3 we've got a workaround - something like:
   ProcessPoolExecutor(multiprocessing.get_context('spawn'))
but I'm guessing, the documentation is scant.

As for a fix, maybe:
- have ProcessPoolExecutor use 'spawn' by default; this way things always work
- have ProcessPoolExecutor properly synchronized its threads before 
"spawning"/"forking"/... so that "single-threaded" code works
- document that combining ProcessPoolExecutor's "fork" option and user threads 
isn't a good idea

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36646] os.listdir() got permission error in Python3.6 but it's fine in Python2.7

[Ryan]

Ryan  added the comment:

PS D:\workspace> python
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir(u'L:\\Temp')
[u'manager.txt', u'alps-trunk-m0.tk.log', u'2015102914.LOG', u'log', 
u'alps-trunk-m0.tk-k55mv1_64_om_c2k6m_eng-201511121
5.LOG', u'logof', u'1', u'dsp_1_ltg_n.bin', u'dsp_1_lwg_n.bin', 
u'modem_1_ltg_n.img', u'modem_1_lwg_n.img', u'kernel_sym
bols', u'k55v1_64_om_c2k6m_clang.log', u'k55v1_64_om_c2k6m_android.log', 
u'k50v1_64_om_c2k6m_android.log', u'collectBuil
dLog.sh', u'UtilitySpotlight', u'k55v1_bsp_android.log', u'specialowner.pm', 
u'DBbuild_GIT.pl', u'daily_out-k37mv1_basic
', u'mtkall_new.txt', u'Visa', u'insert_err_command_file.txt', u'test', 
u'DB_X_remake.list', u'aosp_img.txt', u'Jola', u
'04272', u'alps_dailybuild.ini', u'Luca', u'android-security-bulletin', 
u'michael', u'2018073115.LOG', u'LOG~1', u'mtk-d
b.pl', u'MCD_k79v1_64_android.log', u'build-full_k79v1_64.ninja']
>>> exit()
PS D:\workspace>
PS D:\workspace>
PS D:\workspace> python3
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir(u'L:\\Temp')
Traceback (most recent call last):
  File "", line 1, in 
PermissionError: [WinError 5] Access denied.: 'L:\\Temp'
>>> exit()
PS D:\workspace>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Filosofical Finking: Import protections

[Cameron Simpson]

On 17Apr2019 21:45, MRAB  wrote:

On 2019-04-17 21:20, DL Neil wrote:

Do you bother with exception handling for import statements?

[...]

Catch only what you (well, the script) can fix.

If it needs numpy, but can't import numpy, then when can it do? Might 
as well just let it fail.


I'm of this mind too, but...

I suppose an alternative might be to try to download and install numpy 
and then retry, but what if it can't be downloaded, or the installation 
fails?


As an example of what an open ended can of worms attempts recovery might 
be, yeah. How hard do you try? But also, "installation fails": that 
isn't always a clean situation: it can litter the install area with 
partial junk.


But this is also a bad example: it is something an _invoked_ programme 
should never try to do. Except by specific deliberate design and 
request, a running application shouldn't presume it has rights to 
install additional things, or even to try. I have personally (though 
metaphorically) clipped devs across the ear for doing themselves the 
moral equivalent of the above: try thing, then just "sudo try thing" 
when it was forbidden.


Particularly in managed environments, the setup is often deliberately 
designed to not permit this. Consider the app behind a web service: 
those which are able to install code are in theory open to being 
manipulated from the outside to install and run code -malicious code.  
For this reason such enivoronments are deliberately designed so that an 
app has the barest minimum privileges to perform its task.


So: the app _can't_ write to its code area or to the htdocs tree (in 
whatever form that may be) - that way lies site defacement and 
application subversion. It can't create tables in the database or modify 
schemas. It can't modify data it should not touch, or read data it 
should never see (think reading credential tables or modifying role 
definitions as some examples).


Installing additional packages is the same as self modifying code: as a 
rule, the admins install packages, not the app.


Sorry, ranting now over.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

[issue36071] Add support for Windows ARM32 in ctypes/libffi

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36638] typeperf.exe is not in all skus of Windows SKUs

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36071] Add support for Windows ARM32 in ctypes/libffi

[Steve Dower]

New submission from Steve Dower :


New changeset 11efd79076559cc6e4034bb36db73e5e4293f02d by Steve Dower (Paul 
Monson) in branch 'master':
bpo-36071 Add support for Windows ARM32 in ctypes/libffi (GH-12059)
https://github.com/python/cpython/commit/11efd79076559cc6e4034bb36db73e5e4293f02d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36638] typeperf.exe is not in all skus of Windows SKUs

[Steve Dower]

Steve Dower  added the comment:


New changeset 264a0b40b030fc0ff919b8294df91bdaac853bfb by Steve Dower (Paul 
Monson) in branch 'master':
bpo-36638: Fix WindowsLoadTracker exception on some Windows versions (GH-12849)
https://github.com/python/cpython/commit/264a0b40b030fc0ff919b8294df91bdaac853bfb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36466] Adding a way to strip annotations from compiled bytecode

[cary]

cary  added the comment:

Thanks for the feedback! I wasn't aware that modules depended on this during 
runtime.

Abandoning this :)

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

concurrent.futures.ProcessPoolExecutor uses both multiprocessing and threading. 
 multiprocessing defaults to using os.fork().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
pull_requests:  -12788

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36653] Dictionary Key is without ' ' quotes

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Hi PushkarVaity, and welcome!

I hope that Matthew's suggestion fixes your code for you, but please remember 
that this is a bug tracker for bugs in the Python language and standard 
library, not a help desk.

As a beginner, 99.9% of the times you think that you have found a bug in 
Python, you haven't, it will be a bug in your own code. There are many forums 
where you can ask for help with your code, such as the tutor mailing list 

https://mail.python.org/mailman/listinfo/tutor

Stackoverflow, Reddit's /rlearnpython, and more. You should check with other, 
more experienced programmers before reporting things as bugs.

Thank you.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Importing module from another subdirectory

[Sayth Renshaw]

On Thursday, 18 April 2019 06:59:43 UTC+10, Rich Shepard  wrote:
> What is the proper syntax to import the model class in the model/
> subdirectory into a tkinter view module, e.g., activities.py? The syntax,
> 'import model as m' fails because it is not in the same subdirectory as the
> importing module.
> 
> The program directory tree is:
> 
> bustrac/
> README.rst
> bustrac.py*
> controller/
> model/
> scripts/
> views/
> 
> Running pdb in python3-3.7.3 produces the same error:
> $ python3
> Python 3.7.3 (default, Mar 26 2019, 06:40:28) 
> [GCC 5.5.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pdb
> >>> import activities_data_entry
> Traceback (most recent call last):
>File "", line 1, in 
>File 
> "/home/rshepard/development/business_tracker/views/activities_data_entry.py", 
> line 1, in 
>  import model.activities as act
> ModuleNotFoundError: No module named 'model'
> 
> My previous use of python has had all files in the same directory so I've
> not before had to learn how to address this issue. Pointers appreciated.
> 
> Regards,
> 
> Rich

Morning

Apologies I don't know the answer but went looking. This guide should answer 
the question. Didn't know it was so difficult to be honest.

https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html#example-directory-structure

Then there was this rather long list of traps. 
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html

Most guides say it was fixed in 3.3 but still seems quite confusing post 3.3

Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Function to determine list max without itertools

[Chris Angelico]

On Thu, Apr 18, 2019 at 9:31 AM Sayth Renshaw  wrote:
>
>
> I have created a function that takes a list as an argument.
> Without using itertools I want to compare each item in the list to find the 
> max.
>
> However instead of the max I keep getting the  last item in the list. Where 
> is my logic wrong here?
>
> def maximum(listarg):
> items = list(listarg)
> myMax = 0
> for index, item in enumerate(items):
> for otheritem in items[index + 1 :]:
> if item < otheritem:
> myMax = otheritem
> elif item > otheritem:
> myMax = item
> else:
> myMax = myMax
>
> Seems like it should work but doesn't.
>

I'd recommend rethinking your algorithm first, and then thinking about
debugging your actual code. You should be able to find the largest in
a collection without making many many passes over the items - a single
pass should be sufficient.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Function to determine list max without itertools

[Sayth Renshaw]

I have created a function that takes a list as an argument.
Without using itertools I want to compare each item in the list to find the max.

However instead of the max I keep getting the  last item in the list. Where is 
my logic wrong here?

def maximum(listarg):
items = list(listarg)
myMax = 0
for index, item in enumerate(items):
for otheritem in items[index + 1 :]:
if item < otheritem:
myMax = otheritem
elif item > otheritem:
myMax = item
else:
myMax = myMax

Seems like it should work but doesn't.

Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue36653] Dictionary Key is without ' ' quotes

[Matthew Barnett]

Matthew Barnett  added the comment:

That should be:

def __repr__(self):
return repr(self.name)

Not a bug.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

@gregory.p.smith, I'm puzzled by your references to POSIX and/or os.fork().

The code in question looks like:

import concurrent.futures
import sys

def f():
import ctypes

while True:
with concurrent.futures.ProcessPoolExecutor() as executor:
ftr = executor.submit(f)
ftr.result()

which, to me, looks like pure Python.

Are you saying that this code can't work on GNU/Linux systems.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Hi Pablo,

Was this something you still wanted to clean up or can this be closed?  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32913] Improve regular expression HOWTO

[Brett Cannon]

Brett Cannon  added the comment:


New changeset a6de52c74d831e45ee0ff105196da8a58b9e43cd by Brett Cannon (josh) 
in branch 'master':
bpo-32913: Added re.Match.groupdict example to regex HOWTO (GH-5821)
https://github.com/python/cpython/commit/a6de52c74d831e45ee0ff105196da8a58b9e43cd


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33039] int() and math.trunc don't accept objects that only define __index__

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Rémi,

Are you still working on the patch for this?  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36653] Dictionary Key is without ' ' quotes

[PushkarVaity]

New submission from PushkarVaity :

I am using Python 3.7 with anaconda install.
I am trying to write out a dictionary with similar key's.

for ex: proc_dict = {'add': ('/home/file.tcl', 'args'), 'add': 
('/home/file2.tcl', 'args'), 'sub': ('/home/file2.tcl', 'args')}

To do this, I am using the following class definition and functions:
class ProcOne(object):
def __init__(self, name):
self.name = name

def __repr__(self):
return self.name

I am writing out the dictionary in the following way:
proc_dict[ProcOne(proc_name)] = (full_file, proc_args)

Now, the dictionary key as shown in the example at top is of string type. 
proc_name is the variable holding this string.

The values are tuples. Both elements in the tuple are strings.

When the dictionary is finally written out, the format is as below:
proc_dict = {add: ('/home/file.tcl', 'args'), add: ('/home/file2.tcl', 'args'), 
sub: ('/home/file2.tcl', 'args')} 

Please note the difference from the first example.
The key values don't have a ' ' quote in spite of being a string variable type. 

Since the string quotes are missing, it is very difficult to do post processing 
on this dictionary key.

I am a student and I though that this is an issue because now I am not able to 
compare the key value with a normal string because of the missing quotes. The 
in or not in checking operations do not evaluate to true/false because of the 
missing quotes.

Please let me know if this has never been reported before as I am just a novice 
programmer and would be a big boost to my morale :-) 
Also, please let me know if this issue was already known or wasn't an issue at 
all.

--
components: Regular Expressions
messages: 340452
nosy: PushkarVaity, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Dictionary Key is without ' ' quotes
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36277] pdb's recursive debug command is not listed in the docs

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +easy
stage:  -> needs patch
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

We're discussing vanilla Python, for instance v3.7.0 is:

  git clone .../cpython
  cd cpython
  git checkout v3.7.0
  ./configure --prefix=/home/python/v3.7.0
  make -j && make -j install

(my 3.6.x wasn't vanilla, but I clearly stated that)

Like I also mentioned, loading down the machine also helps.  Try something like 
running #cores*2 of the script in parallel?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[miss-islington]

miss-islington  added the comment:


New changeset 0d4f16d283fe3b8a183775ac7ac193988d971ad5 by Miss Islington (bot) 
in branch '3.7':
bpo-36649: Remove trailing spaces for registry keys when installed via the 
Store (GH-12865)
https://github.com/python/cpython/commit/0d4f16d283fe3b8a183775ac7ac193988d971ad5


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[miss-islington]

Change by miss-islington :


--
pull_requests: +12794

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:


New changeset 4c3efd9cd07194b5db2a60ae5951134cda8b69db by Steve Dower in branch 
'master':
bpo-36649: Remove trailing spaces for registry keys when installed via the 
Store (GH-12865)
https://github.com/python/cpython/commit/4c3efd9cd07194b5db2a60ae5951134cda8b69db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36635] Add _testinternalcapi module

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5c75f37d473140f0e0b7d9bf3a8c08343447ded1 by Victor Stinner in 
branch 'master':
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
https://github.com/python/cpython/commit/5c75f37d473140f0e0b7d9bf3a8c08343447ded1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Importing module from another subdirectory

[Rich Shepard]

What is the proper syntax to import the model class in the model/
subdirectory into a tkinter view module, e.g., activities.py? The syntax,
'import model as m' fails because it is not in the same subdirectory as the
importing module.

The program directory tree is:

bustrac/
   README.rst
   bustrac.py*
   controller/
   model/
   scripts/
   views/

Running pdb in python3-3.7.3 produces the same error:
$ python3
Python 3.7.3 (default, Mar 26 2019, 06:40:28) 
[GCC 5.5.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

import pdb
import activities_data_entry

Traceback (most recent call last):
  File "", line 1, in 
  File "/home/rshepard/development/business_tracker/views/activities_data_entry.py", 
line 1, in 
import model.activities as act
ModuleNotFoundError: No module named 'model'

My previous use of python has had all files in the same directory so I've
not before had to learn how to address this issue. Pointers appreciated.

Regards,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[mollison]

mollison  added the comment:

@brett.cannon: Yes, I will submit a PR. I have the code ready to go already. I 
just submitted the Python contributor agreement. I think it will take the 
system a day or two to register that. Then, I will submit the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

> I suspect that the former is implementation accident. I can't imagine why
would anyone want this behavior.
>
> NB, POSIX says that when PATH is unset or empty, the path search is
implementation-defined.

Not thank you POSIX for being clueless. Let's say that Python is
opiniatied, as when we decided that file descriptors must be created
non-inheritable by default even if "it goes against POSIX".

If you want to look if the current directory, you now have to ask for it
kindly and explicitly ;-)

IHMO Using CS_PATH rather than hardcoded os.defpath is also a major step
forward ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Filosofical Finking: Import protections

[Larry Martell]

On 2019-04-17 21:20, DL Neil wrote:
> Do you bother with exception handling for import statements?

I often have to do something like this:

try:
from settings import SITE_WAFER_DIAMETER
except ImportError:
SITE_WAFER_DIAMETER = 300
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue36652] Non-embedded zip distribution

[wheelerlaw]

New submission from wheelerlaw :

Pretty straight forward request. It would be nice if there was an installation 
method where I can just unzip a Python distribution rather than running an 
installer. 

Specifically this is for getting Python to run in Wine. Right now, Python for 
Windows runs fine under Wine, but the installer doesn't, so a manual process of 
running the installer on a Windows machine and then copying the installed 
resources to a Linux machine with Wine installed. A zip distribution would 
solve this, since I could just unzip it and run it under Wine.

--
components: Installation
messages: 340445
nosy: wheelerlaw
priority: normal
severity: normal
status: open
title: Non-embedded zip distribution
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in the socket function parsing a Bluetooth address

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Seconded Viktor's question.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Filosofical Finking: Import protections

[MRAB]

On 2019-04-17 21:20, DL Neil wrote:

(I know it's not Friday [exp], and after personal apologies[apo])


Do you bother with exception handling for import statements?


Most of the code I read, both in books and during code review, eschews
any form of ImportError check. Even data science people who 'clean'
every data field towards inclusion/exclusion in the analysis, take for
granted that numpy, scipy, pandas, et al, will be available to their code.


Does such a check seem un-pythonic? [sto] (maybe 'forgiveness cf
permission'?)

Can we assume that if such a catastrophic error occurs, it is quite
acceptable for the code to fall-over in a tumbling-fumble?

Does it make a difference if operating in/out of a dev-ops environment?

Might such only occur once, because once the environment has been
correctly-built, it will/can *never* happen again?

Can we assume Built-in and PSL modules *must* be present, so we only
need to talk about covering in-house code and pip-ed (or similar) modules?

Is it more/less important in __main__ than in an imported module?

Do you handle each import separately, or all in a single try..except block?

Do you try to anticipate and cover every import in the system at the top
of __main__, eg imports inside imported modules?

What about OpSys-specific imports (etc)?

Do modules import-ed only in specific situations, deserve more, or less,
attention?


[snip]
Catch only what you (well, the script) can fix.

If it needs numpy, but can't import numpy, then when can it do? Might as 
well just let it fail.


I suppose an alternative might be to try to download and install numpy 
and then retry, but what if it can't be downloaded, or the installation 
fails?


No, you have to give up at some point. It's better just to report the 
problem and leave it to the user to fix it.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[Grant Edwards]

On 2019-04-17, DL Neil  wrote:

> Do you bother with exception handling for import statements?

Sometimes.  There are two cases when I do that:

 1. When the module has different names under Python2 and Python3 and
the program tries first one, then the other.

 2. When the program can still do something useful (if perhaps
feature-limited) without the imported module by substituting
something else in its place.

> Most of the code I read, both in books and during code review,
> eschews any form of ImportError check. Even data science people who
> 'clean' every data field towards inclusion/exclusion in the
> analysis, take for granted that numpy, scipy, pandas, et al, will be
> available to their code.

You've omitted the second thing assumed by the authors: without numpy,
scipy, pandas, et alia the program can do nothing useful.

> Does such a check seem un-pythonic? [sto] (maybe 'forgiveness cf 
> permission'?)

It's probably rather unpythonic if you're not going to anything useful
in the exception handler.

> Can we assume that if such a catastrophic error occurs, it is quite
> acceptable for the code to fall-over in a tumbling-fumble?

It's certainly OK with me.  I'm not sure why you refer to raising an
exception as "fall-over in a tumbling fumble".  Raising an exception
is the normal way to indicate failure in Python.

> Does it make a difference if operating in/out of a dev-ops
> environment?

I have no idea what "a dev-ops environment means", and I plan on
keeping it that way. :)

-- 
Grant Edwards   grant.b.edwardsYow! WHOA!!  Ken and Barbie
  at   are having TOO MUCH FUN!!
  gmail.comIt must be the NEGATIVE
   IONS!!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Filosofical Finking: Import protections

[Chris Angelico]

On Thu, Apr 18, 2019 at 6:21 AM DL Neil  wrote:
>
> (I know it's not Friday [exp], and after personal apologies[apo])
>
>
> Do you bother with exception handling for import statements?
>
>
> Most of the code I read, both in books and during code review, eschews
> any form of ImportError check. Even data science people who 'clean'
> every data field towards inclusion/exclusion in the analysis, take for
> granted that numpy, scipy, pandas, et al, will be available to their code.
>
>
> Does such a check seem un-pythonic? [sto] (maybe 'forgiveness cf
> permission'?)
>
> Can we assume that if such a catastrophic error occurs, it is quite
> acceptable for the code to fall-over in a tumbling-fumble?

I try/except around import statements only if it's possible for the
program to recover from the exception. For instance, something that
runs on Py2 and Py3 might attempt to import from a Py3 name (urllib
comes to mind), and if it fails, redo the import from the Py2 name; or
something might try to import an optional subroutine, and if it isn't
present, set something to None, or to a null function.

For something that is absolutely required for the program to continue,
what would be in your exception handler? Print a message to stderr and
exit? That's exactly what not-catching-the-exception is for. I do
sometimes annotate the imports, though:

from dataclasses import dataclass # ImportError? Upgrade to Python 3.7
or pip install dataclasses

If that bombs out, the entire line will get printed, comment and all,
and there isn't really anything else that I would want to do with the
exception.

So I guess the best way to answer your question is with another
question: If such a catastrophic error occurs, what ELSE would your
code do than fall over? If there's an answer to that question, then
sure, catch the ImportError. Otherwise don't.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue10417] [2.7] unittest triggers UnicodeEncodeError with non-ASCII character in the docstring of the test function

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I mean that '\xe4'.encode(encoding, 'backslashreplace') will fail.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Friday Filosofical Finking: Import protections

[DL Neil]

(I know it's not Friday [exp], and after personal apologies[apo])


Do you bother with exception handling for import statements?


Most of the code I read, both in books and during code review, eschews 
any form of ImportError check. Even data science people who 'clean' 
every data field towards inclusion/exclusion in the analysis, take for 
granted that numpy, scipy, pandas, et al, will be available to their code.



Does such a check seem un-pythonic? [sto] (maybe 'forgiveness cf 
permission'?)


Can we assume that if such a catastrophic error occurs, it is quite 
acceptable for the code to fall-over in a tumbling-fumble?


Does it make a difference if operating in/out of a dev-ops environment?

Might such only occur once, because once the environment has been 
correctly-built, it will/can *never* happen again?


Can we assume Built-in and PSL modules *must* be present, so we only 
need to talk about covering in-house code and pip-ed (or similar) modules?


Is it more/less important in __main__ than in an imported module?

Do you handle each import separately, or all in a single try..except block?

Do you try to anticipate and cover every import in the system at the top 
of __main__, eg imports inside imported modules?


What about OpSys-specific imports (etc)?

Do modules import-ed only in specific situations, deserve more, or less, 
attention?



Refs:

[apo] Change of season coughs and sniffles, duly shared with the CO (oops!)

[exp] Explanation: in the Christian calendar, this Friday is "Good 
Friday" and thus a national/public holiday in much of the western world. 
Thus, also next ("Easter") Monday and possibly Tuesday. To continue the 
theme: next Thursday is also a holiday in New Zealand and Australia 
("ANZAC Day": equating to Veterans' Day, Memorial Day, Independence Day, 
etc)


[imp] 5. The import system https://docs.python.org/3.6/reference/import.html

[sto] A somewhat similar question: 
https://stackoverflow.com/questions/3131217/error-handling-when-importing-modules


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27679] set_bitfields() unused in _ctypes_test

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36651] Asyncio Event Loop documentation inconsistency (call_later and call_at methods)

[Enrico Carbognani]

Change by Enrico Carbognani :


--
keywords: +patch
pull_requests: +12793
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I am unable to get cf-deadlock.py to hang on my own builds of pure CPython 
3.7.2+ d7cb2034bb or 3.6.8+ be77fb7a6e (versions i had in a local git clone).

which specific python builds are seeing the hang using?  Which specific 
platform/distro version?  "3.7.2" isn't enough, if you are using a distro 
supplied interpreter please try and reproduce this with a build from the 
CPython tree itself.  distros always apply their own patches to their own 
interpreters.

...

Do realize that while working on this it is fundamentally *impossible* per 
POSIX for os.fork() to be safely used at the Python level in a process also 
using pthreads.  That this _ever_ appeared to work is a pure accident of 
implementations of underlying libc, malloc, system libraries, and kernel 
behaviors.  POSIX considers it undefined behavior.  Nothing done in CPython can 
avoid that.  Any "fix" for these kinds of issues is merely working around the 
inevitable which will re-occur.

concurrent.futures.ProcessPoolExecutor uses multiprocessing for its process 
management.  As of 3.7 ProcessPoolExecutor accepts a mp_context parameter to 
specify the multiprocessing start method.  Alternatively the default appears
to be controllable as a global setting 
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods.

Use the 'spawn' start method and the problem should go away as it'll no longer 
be misusing os.fork().  You _might_ be able to get the 'forkserver' start 
method to work, but only reliably if you make sure the forkserver is spawned 
_before_ any threads in the process (such as ProcessPoolExecutor's own queue 
management thread - which appears to be spawned upon the first call to 
.submit()).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21710] --install-base option ignored?

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:

Great, thanks.

I have a fix in PR, but I'm going to do a "real" build to check. My machine is 
running a beta build, unfortunately, so I can't validate it against the older 
version. Might ping you for some help with that.

--
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Jason R. Coombs  added the comment:

Hi Josh. Thanks for the tip on types.MethodType. I've updated the code to use 
that and the behavior seems to be the same, MethodType does seem a more 
appropriate way to create a bound method.

Regarding the referenced tickets, I suspect they're not pertinent, as the 
behavior did work in Python 3.7.1 (confirmed) and 3.7.2 (highly likely), so the 
regression appears to be in the changes for 3.7.3 
(https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-3-final).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Hmm... Although I can't seem to reproduce on 3.7.2 (ob.calls is 1), so assuming 
it is really happening in 3.7.3, it wouldn't be either of those issues (which 
were fully in place long before 3.7.2 I believe).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Josh Rosenberg]

Josh Rosenberg  added the comment:

It seems highly likely this is related to #26110 which optimizes method calls 
by avoiding the creation of bound methods in certain circumstances, and/or the 
follow-up #29263.

Side-note:

bound_method = functools.partial(method, self)

is not how you create real bound methods. The correct approach (that makes a 
bound method identical to what the interpreter makes when necessary) for your 
use case is:

bound_method = types.MethodType(method, self)

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[Rudolph Froger]

Rudolph Froger  added the comment:

Thanks all for the fixes!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Correct me if I'm wrong, but this isn't actually an issue for CPython, right? 
The GIL ensures that when a thread writes to _shutdown, nothing else is reading 
it until the GIL is released and acquired by a new thread (which synchronizes 
_shutdown).

It might conceivably be a problem on non-CPython interpreters if they have no 
other form of inter-thread synchronization involved; that said, the locking 
involved in the self.work_queue.get(block=True) call likely synchronizes by 
side-effect even there.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Hugues Valois]

Hugues Valois  added the comment:

C:\Users\huvalo>ver

Microsoft Windows [Version 10.0.17763.437]

--
versions:  -Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36651] Asyncio Event Loop documentation inconsistency (call_later and call_at methods)

[Enrico Carbognani]

New submission from Enrico Carbognani :

In the documentation for the call_later and the call_at methods there is a note 
which says that the delay cannot be longer than a day, but both methods have a 
note saying that this limitation was removed in Python 3.8.

--
assignee: docs@python
components: Documentation
files: documenation_incosistency.png
messages: 340434
nosy: Enrico Carbognani, docs@python
priority: normal
severity: normal
status: open
title: Asyncio Event Loop documentation inconsistency (call_later and call_at 
methods)
versions: Python 3.8
Added file: https://bugs.python.org/file48273/documenation_incosistency.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I've put together this full reproducer script:

```
import functools


def method_cache(method):
def wrapper(self, *args, **kwargs):
# it's the first call, replace the method with a cached, bound 
method
bound_method = functools.partial(method, self)
cached_method = functools.lru_cache()(bound_method)
setattr(self, method.__name__, cached_method)
return cached_method(*args, **kwargs)
return wrapper


class MyClass:
calls = 0

@method_cache
def call_me_maybe(self, number):
self.calls += 1
return number


ob = MyClass()
ob.call_me_maybe(0)
ob.call_me_maybe(0)
assert ob.calls == 1
```

That code fails on Python 3.7.3. If I bypass the `from _functools import 
_lru_cache_wrapper` in functools, the test no longer fails, so the issue seems 
to be only with the C implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +12792
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

script to capture stack backtrace at time of fork, last backtrace printed will 
be for hang

--
Added file: https://bugs.python.org/file48272/gdb.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

run ProcessPoolExecutor with one fixed child (over ride default of #cores)

--
Added file: https://bugs.python.org/file48271/cf-deadlock-1.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

Change by Jason R. Coombs :


--
components: +Library (Lib)
type:  -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[cagney]

cagney  added the comment:

Here's a possible stack taken during the fork():

Thread 1 "python3" hit Breakpoint 1, 0x77124734 in fork () from 
/lib64/libc.so.6

Thread 1814 (Thread 0x7fffe69d5700 (LWP 23574)):
#0  0x77bc24e5 in __pthread_mutex_unlock_usercnt () from 
/lib64/libpthread.so.0
#1  0x771928e3 in dl_iterate_phdr () from /lib64/libc.so.6
#2  0x7fffe5fcfe55 in _Unwind_Find_FDE () from /lib64/libgcc_s.so.1
#3  0x7fffe5fcc403 in uw_frame_state_for () from /lib64/libgcc_s.so.1
#4  0x7fffe5fcd90f in _Unwind_ForcedUnwind_Phase2 () from 
/lib64/libgcc_s.so.1
#5  0x7fffe5fcdf30 in _Unwind_ForcedUnwind () from /lib64/libgcc_s.so.1
#6  0x77bc7712 in __pthread_unwind () from /lib64/libpthread.so.0
#7  0x77bbf7e7 in pthread_exit () from /lib64/libpthread.so.0
#8  0x0051b2fc in PyThread_exit_thread () at Python/thread_pthread.h:238
#9  0x0055ed16 in t_bootstrap (boot_raw=0x7fffe8da0e40) at 
./Modules/_threadmodule.c:1021
#10 0x77bbe594 in start_thread () from /lib64/libpthread.so.0
#11 0x77157e5f in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x77fca080 (LWP 20524)):
#0  0x77124734 in fork () from /lib64/libc.so.6
#1  0x00532c8a in os_fork_impl (module=) at 
./Modules/posixmodule.c:5423
#2  os_fork (module=, _unused_ignored=) at 
./Modules/clinic/posixmodule.c.h:1913

where, in my source code, dl_iterate_phdr() starts with something like:

  /* Make sure nobody modifies the list of loaded objects.  */
  __rtld_lock_lock_recursive (GL(dl_load_write_lock));

i.e., when the fork occures, the non-fork thread has acquired 
dl_load_write_lock - the same lock that the child will later try to acquire 
(and hang)

no clue as to what that thread is doing though; other than it looks like it is 
trying to generate a backtrace?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36650] Cached method implementation no longer works on Python 3.7.3

[Jason R. Coombs]

New submission from Jason R. Coombs :

In [this ticket](https://github.com/jaraco/jaraco.functools/issues/12), I 
learned that 
[jaraco.functools.method_cache](https://github.com/jaraco/jaraco.functools/blob/6b32ee0dfd3e7c88f99e88cd87c35fa9b76f261f/jaraco/functools.py#L109-L180)
 no longer works on Python 3.7.3.

A distilled version of what's not working is this example:

```
>>> import jaraco.functools
>>> class MyClass:
...   calls = 0
...   @jaraco.functools.method_cache
...   def call_me_maybe(self, val):
... self.calls += 1
... return val
... 
>>> a = MyClass()
>>> a.call_me_maybe(0)
0
>>> a.call_me_maybe(0)
0
>>> a.calls
2
```

The second call to the cached function is missing the cache even though the 
parameters to the function are the same.


```
>>> a.call_me_maybe

>>> a.call_me_maybe.cache_info()
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)
```

Here's a further distilled example not relying on any code from 
jaraco.functools:

```
>>> def method_cache(method):
... def wrapper(self, *args, **kwargs):
... # it's the first call, replace the method with a cached, bound 
method
... bound_method = functools.partial(method, self)
... cached_method = functools.lru_cache()(bound_method)
... setattr(self, method.__name__, cached_method)
... return cached_method(*args, **kwargs)
... return wrapper
... 
>>> import functools
>>> class MyClass:
...   calls = 0
...   @method_cache
...   def call_me_maybe(self, val):
... self.calls += 1
... return val
... 
>>> a = MyClass()
>>> a.call_me_maybe(0)
0
>>> a.call_me_maybe(0)
0
>>> a.calls
2
```

I was not able to replicate the issue with a simple lru_cache on a partial 
object:

```
>>> def func(a, b):
...   global calls
...   calls += 1
... 
>>> import functools
>>> cached = functools.lru_cache()(functools.partial(func, 'a'))
>>> calls = 0
>>> cached(0)
>>> cached(0)
>>> calls
1
```

Suggesting that there's some interaction with the instance attribute and the 
caching functionality.

I suspect the issue arose as a result of changes in issue35780.

--
assignee: rhettinger
keywords: 3.7regression
messages: 340429
nosy: jaraco, rhettinger
priority: normal
severity: normal
status: open
title: Cached method implementation no longer works on Python 3.7.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Nick Davies]

Nick Davies  added the comment:

My preference would actually be number 3 because:

 1: I agree that this isn't really a safe option because it could slow things 
down (possibly a lot)
 2: I haven't found this to be rare in my situation but I am not sure how 
common my setup is. We have a threaded server with a mix of sync and asyncio so 
we use run in a bunch of places inside threads. Any time the server gets busy 
any task creation that occurs during the return of run crashes. My two main 
reservations about this approach are:
- There is a potentially unbounded number of times that this could need to 
retry.
- Also this is covering up a thread unsafe operation and we are pretty 
lucky based on the current implementation that it explodes in a consistent and 
sane way that we can catch and retry.
 3: Loop is already expected to be hashable in 3.7 as far as I can tell 
(https://github.com/python/cpython/blob/3.7/Lib/asyncio/tasks.py#L822) so other 
than the slightly higher complexity this feels like the cleanest solution.


> The fix can be applied to 3.7 and 3.8 only, sorry. Python 3.6 is in security 
> mode now.

Thats fine, you can work around the issue using asyncio.set_task_factory to 
something that tracks the tasks per loop with some care.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Steve Dower]

Steve Dower  added the comment:

Can you get me the *exact* Windows version number you're using?

I added those to work around a bug where keys were being incorrectly trimmed, 
and that format seemed to evaluate to nothing correctly, but perhaps that bug 
has been fixed in the OS?

--
assignee:  -> steve.dower
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36649] Windows Store app install registry keys have incorrect paths

[Hugues Valois]

New submission from Hugues Valois :

When reading registry values under HKCU\SOFTWARE\Python\PythonCore\3.7 that 
were written by the Windows Store app install, all file and folder paths are 
incorrect.

Notice the extra [ ] as well as the missing backslash before python.exe and 
pythonw.exe

Paths read from registry are:
```
C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0[
]

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0python.exe[
]

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0pythonw.exe[
]

```

Paths on disk are:
```
C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0\python.exe

C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1008.0_x64__qbz5n2kfra8p0\pythonw.exe
```

--
components: Installation, Windows
messages: 340426
nosy: Hugues Valois, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Store app install registry keys have incorrect paths
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The fix can be applied to 3.7 and 3.8 only, sorry.
Python 3.6 is in security mode now.

--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for the report!

I see 3 ways to fix the bug:
1. Guard _all_tasks with threading.Lock. It hurts performance significantly.
2. Retry list(_all_tasks) call in a loop if RuntimeError was raised. A chance 
of collision is very low, the strategy is good enough
3. Change _all_tasks from weak set of tasks to WeakDict[AbstractEventLoop, 
WeakSet[Task]]. This realization eliminates the collision it is a little 
complicated. Plus loop should be hashable now (perhaps ok but I'm not sure if 
all third-party loops support it).

Thus I'm inclining to bullet 2.
THoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[Brett Cannon]

Brett Cannon  added the comment:

@mollison: would you like to open a PR w/ how you would expect it to be 
formatted?

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[Jakub Wilk]

Jakub Wilk  added the comment:

When PATH is empty string:
* zsh and FreeBSD which look for binaries in cwd.
* debianutils and GNU which always fail.

I suspect that the former is implementation accident. I can't imagine why would 
anyone want this behavior.

NB, POSIX says that when PATH is unset or empty, the path search is 
implementation-defined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

[Tim Hatch]

Change by Tim Hatch :


--
nosy: +thatch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in the socket function parsing a Bluetooth address

[STINNER Victor]

Change by STINNER Victor :


--
title: Integer overflow in socketmodule -> Integer overflow in the socket 
function parsing a Bluetooth address
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in socketmodule

[STINNER Victor]

STINNER Victor  added the comment:

> In Modules/socketmodule.c , the bluetooth address supplied is vulnerable to 
> integer overflow.

Attached PR 12864 modifies the following code:


  unsigned int b0, b1, b2, b3, b4, b5;
  char ch;
  int n;
  n = sscanf(name, "%X:%X:%X:%X:%X:%X%c", , , , , , , );

Can someone please elaborate how this code can trigger an integer overflow? 
What is the consequence of an integer overflow? Does Python crash?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18564] Integer overflow in socketmodule

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +12791
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] On Unix, shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

I modified posixpath.defpath, shutil.which() and 
distutils.spawn.find_executable() in 3.7 and master (future Python 3.8) 
branches. I close the issue. Thanks everybody for the review and helping me to 
collect info about corner cases!

I chose to also change Python 3.7. IMHO there is a low risk of breaking 
applications: I expect that few users run Python with no PATH environment 
variable *and* expect that Python looks for programs in the current directory. 
But it enhances the security a little bit.

For Python 2.7... well, I don't think that this issue is important enough to 
justify a backport. I prefer to do nothing rather than having to deal with 
unhappy users complaining that Python 2.7 changed broke their application in a 
minor 2.7.x release :-) Even if, again, the risk of regression is very low.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
title: shutil.which() and subprocess no longer look for the executable in the 
current directory if PATH environment variable is not set -> On Unix, 
shutil.which() and subprocess no longer look for the executable in the current 
directory if PATH environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

I'm still confused by distutils.spawn.find_executable() function which *always* 
first look the current directory. I don't know the rationale for this behavior, 
so I made the conservation choice of keeping it.

If someone wants to change distutils.spawn.find_executable(), please open a 
separated issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

STINNER Victor  added the comment:

Gregory:
> I'm not arguing against this change, just trying to figure out where it came 
> from in the first place.  We should fix the value on all OSes.

In the meanwhile, I reverted the ntpath change. I'm not sure that it's ok to 
change the Windows case.

shutil.which() *always* starts by checking if the searched program is the 
current directory:

if sys.platform == "win32":
# The current directory takes precedence on Windows.
...
if curdir not in path:
path.insert(0, curdir)

If someone cares about changing Windows, please open a separated issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36639] Provide list.rindex()

[林自均]

林自均  added the comment:

Hi @rhettinger ,

Thank you for the reply. May I ask what is the known, strong use cases for 
str.rindex()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer look for the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

Change by STINNER Victor :


--
title: shutil.which() and subprocess no longer looks the executable in the 
current directory if PATH environment variable is not set -> shutil.which() and 
subprocess no longer look for the executable in the current directory if PATH 
environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] shutil.which() and subprocess no longer looks the executable in the current directory if PATH environment variable is not set

[STINNER Victor]

Change by STINNER Victor :


--
title: Remove current directory from posixpath.defpath to enhance security -> 
shutil.which() and subprocess no longer looks the executable in the current 
directory if PATH environment variable is not set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 394b991e41a2a4ce3afc8e6fde44de46e73bbb34 by Victor Stinner in 
branch '3.7':
[3.7] bpo-35755: shutil.which() uses os.confstr("CS_PATH") (GH-12862)
https://github.com/python/cpython/commit/394b991e41a2a4ce3afc8e6fde44de46e73bbb34


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Rudolph Froger for the bug report: the issue is now fixed in 3.7 and 
master (future Python 3.8) branches. Sorry for the delay.

--

Alexey Izbyshev wrote PR 5773 to also use fstat() on Linux.

I chose to merge my PR 12852 which is more conservative: it keeps dup() on 
Linux. I'm not sure why exactly, but I recall that the author of the function, 
Antoine Pitrou, wanted to use dup() on Linux.

I'm not convinced by the O_PATH issue on Linux (described above), so I merged 
my conservative change instead.

Later, we can still move to fstat() on Linux as well if someone comes with a 
more concrete example against dup().

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:

In short, Python 2.7 doesn't seem to be affected by fstat/dup issues.

Python 2.7 doesn't check if file descriptors 0, 1 and 2 at startup. Python 2 
uses PyFile_FromFile() to create sys.stdin, sys.stdout and sys.stderr which 
create a "file" object. The function calls fstat(fd) but it ignores the error: 
fstat() is only used to fail if the fd is a directory.

Python 2.7 doesn't have the is_valid_fd() function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b87a8073db73f9ffa96104e00c624052e34b11c7 by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852) (GH-12863)
https://github.com/python/cpython/commit/b87a8073db73f9ffa96104e00c624052e34b11c7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: doctest random output?

[duncan smith]

On 28/08/2017 20:17, Leam Hall wrote:
> On 08/28/2017 11:40 AM, Dennis Lee Bieber wrote:
> 
> ... a bunch of good stuff ...
> 
> I'm (re-)learning python and just trying make sure my function works.
> Not at the statistical or cryptographic level.   :)
> 
> Thanks!
> 
> Leam

If it's supposed to generate values that follow a particular
distribution, and they don't, then it doesn't work. I had a bunch of
functions for generating values from various distributions. My line
manager told me to just set the seed so that the outputs were
deterministic. Worse than no test at all. It relied on my original
implementation (that generated the values for comparison) working, and
only tested if the implementation (of random.random() or my code) had
changed. So I ignored my boss and simply generated samples of values and
tested using a KS goodness of fit test. The tests should fail 5% of the
time. Run them a few times and check that no individual test is failing
consistently. I don't see how you can do much better than that. Of
course, this doesn't relate directly to doctest.

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3092d6b2630e4d2bd200fbc3231c27a7cba4d6b2 by Victor Stinner in 
branch 'master':
bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852)
https://github.com/python/cpython/commit/3092d6b2630e4d2bd200fbc3231c27a7cba4d6b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[miss-islington]

Change by miss-islington :


--
pull_requests: +12790

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22454] Adding the opposite function of shlex.split()

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31094] asyncio: get list of connected clients

[LihuaZhao]

Change by LihuaZhao :


--
pull_requests: +12789

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36648] MAP_SHARED isn't proper for anonymous mappings for VxWorks

[LihuaZhao]

Change by LihuaZhao :


--
keywords: +patch
pull_requests: +12788
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com