from:"Mike"



Change by Mike Auty :


--
title: file_open doesn't handle UNC paths produced by pathlib's resolve() (but 
can handle UNC paths with additional slashes) -> urllib.request.urlopen doesn't 
handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with 
additional slashes)

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)



Mike Auty  added the comment:

I can confirm that url2pathname work with either number of slashes, and that 
open_file appears to have had the file: removed.

However, in even if the check in open_file were bypassed, it calls 
open_local_file, which then strips any host before calling url2pathname, 
meaning the host will never be included if only two slashes are used.

host, file = _splithost(url)
localname = url2pathname(file)

This is what seems to cause the issue when attempting to open 
file://server/host/file.ext on windows, even though 
file:server/host/file.ext open just fine.

The problem that I found, and was in bug #32442, is that pathlib only ever 
returns two slashes, which despite being a valid and correctly formed url, 
can't be opened by urllib.request.urlopen().  Since there doesn't seem to be an 
issue with opening these files (given it works for file:server...) and 
since nt2pathname will produce the correct result, it feels as though open_file 
should have special code on windows to allow servers to be accepted by the file 
handler (open_local_file should probably stay as is to not change the API too 
much).

--

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)



Mike Auty  added the comment:

Here's the revised code sample:

import pathlib
import urllib.request

path = "Z:\\test.py"

print(f"Stock open: {pathlib.Path(path).as_uri()}")
with urllib.request.urlopen(pathlib.Path(path).as_uri()) as fp:
data = fp.read()

print(f"Pathlib resolved open: {pathlib.Path(path).resolve().as_uri()}")
with urllib.request.urlopen(pathlib.Path(path).resolve().as_uri()) as fp:
data = fp.read()

and here's the output:

Z:\> python test.py
Stock open: file:///Z:/test.py
Pathlib resolved open: file://host/share/test.py
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\urllib\request.py", line 1505, in 
open_local_file
stats = os.stat(localfile)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 
'\\share\\test.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Z:\test.py", line 14, in 
with urllib.request.urlopen(pathlib.Path(path).resolve().as_uri()) as 
fp:
File "C:\Program Files\Python310\lib\urllib\request.py", line 216, in 
urlopen
return opener.open(url, data, timeout)
File "C:\Program Files\Python310\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Program Files\Python310\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Program Files\Python310\lib\urllib\request.py", line 496, in 
_call_chain
result = func(*args)
File "C:\Program Files\Python310\lib\urllib\request.py", line 1483, in 
file_open
return self.open_local_file(req)
File "C:\Program Files\Python310\lib\urllib\request.py", line 1522, in 
open_local_file
raise URLError(exp)
urllib.error.URLError: 

--

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)



Mike Auty  added the comment:

My bad, sorry, I realized I was conflating open with urllib.request.urlopen.  I 
believe the issue still exists though, sorry for the confusion.

--

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)



Mike Auty  added the comment:

> Why are you adding `.as_uri()`?

The API we provide accepts URIs, so whilst the example seems a little 
contrived, the code itself expects a URI and then calls open (making use of the 
ability to add open handlers).

> Builtin open() calls C open().

As best I can tell the file handler is defined in urllib/request.py as 
file_open.  This appears to do some preprocessing to remove the file scheme and 
(and explicitly throws an exception if there's a host that isn't localhost) 
before it gets to the C open().  I wondered why it didn't check if it was on 
windows and, if so, construct an appropriate path (since quadruple hash I don't 
think adheres to the URI RFC, but seems to open correctly)?

--

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)



New submission from Mike Auty :

I've found open to have difficulty with a resolved pathlib path:

Example code of:

   import pathlib
   path = "Z:\\test.py"
   with open(path) as fp:
   print("Stock open: works")
   data = fp.read()
   with open(pathlib.Path(path).resolve().as_uri()) as fp:
   print("Pathlib resolve open")
   data = fp.read()

Results in:

Z:\> python test.py
Stock open: works
Traceback (most recent call last):
  File "Z:\test.py", line 12, in 
with open(pathlib.Path(path).resolve().as_uri()) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 
"file://machine/share/test.py"

Interestingly, I've found that open("file:machine/share/test.py") succeeds, 
but this isn't what pathlib's resolve() produces.  It appears as though 
file_open only supports hosts that are local, but will open UNC paths on 
windows with the additional slashes.  This is quite confusing behaviour and 
it's not clear why file://host/share/file won't work, but 
file:host/share/file does.

I imagine this is a long time issue and a decision has already been reached on 
why file_open doesn't support such URIs, but I couldn't find the answer 
anywhere, just issue 32442 which was resolved without clarifying the 
situation...

--
messages: 412602
nosy: ikelos
priority: normal
severity: normal
status: open
title: file_open doesn't handle UNC paths produced by pathlib's resolve() (but 
can handle UNC paths with additional slashes)

___
Python tracker 
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32442] Result of pathlib Path.resolve() with UNC path is not very useful



Mike Auty  added the comment:

Sorry for the spam, thought I was in a different text box.  5:(

--
nosy: +ikelos
title: file_open unc -> Result of pathlib Path.resolve() with UNC path is not 
very useful

___
Python tracker 
<https://bugs.python.org/issue32442>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32442] file_open unc



Change by Mike Auty :


--
title: Result of pathlib.Path.resolve() with UNC path is not very useful -> 
file_open unc

___
Python tracker 
<https://bugs.python.org/issue32442>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32658] Metacharacter (\) documentation suggestion



mike mcleod  added the comment:

I would like to help with this issue. Is that acceptable?

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue32658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1521051] Allow passing DocTestRunner and DocTestCase in doctest



mike mcleod  added the comment:

I would like to help on this issue.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue1521051>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10434] Document the rules for "public names"



mike mcleod  added the comment:

I would like to help on this issue. Is there anyone available to push a PR 
through? If I make the changes.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue10434>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file



Change by mike mcleod :


--
keywords: +patch
pull_requests: +28934
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/30747

___
Python tracker 
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file

2022-01-15 Thread mike mcleod



mike mcleod  added the comment:

Working.. should be able to create pull request soon. Note part of suggestions 
include using SIOCOUTQ, but this does not have an equivalent for windows. And 
as Murphy's law goes this is likely to be where the problem is!

--

___
Python tracker 
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46377] TimedRotatingFileHandler "midnight" misleading when interval > 1

2022-01-14 Thread Mike Schiessl



Mike Schiessl  added the comment:

Yes, enforcing interval == 1 or interval == None (which pulls the 
TimedRotatingFileHandler class __init__ default value which is also 1) works 
perfectly with midnight.

I do not see any urge on that topic - as I personally now know the issue :D - 
but I really feel this fix could save someones else's time someday ;) 

So going the "safe" way via deprecation cycle seems to be the right approach

--

___
Python tracker 
<https://bugs.python.org/issue46377>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46377] TimedRotatingFileHandler "midnight" misleading when interval > 1

2022-01-14 Thread Mike Schiessl



Mike Schiessl  added the comment:

i've just checked PR and you're right, something with the PR went wrong.

Anyway, midnight (at least from the wording) specifies the "atTime". (which 
should be midnight). 


Again, if there's (by mistake) an interval bigger than 1 set(which in my mind 
makes no sense along to be used with midnight) things are getting pretty 
intransparent. The midnight handler created a logfile dated with 2021-12-15 
(last night). Took me some time to get this sorted. (I've discovered, that I've 
set 30 in a default value file).

Agreed on the backward compatibility, but I would assume someone using 
"midnight" would not expect any other behavior than "daily at midnight" besides 
using the atTime to modify the rollover time. (my opinion)

--

___
Python tracker 
<https://bugs.python.org/issue46377>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46377] TimedRotatingFileHandler "midnight" misleading when interval > 1

2022-01-14 Thread Mike Schiessl



New submission from Mike Schiessl :

Using the TimedRotatingFileHandler along with "when='midnight'" and interval > 
1, midnight is handled equally to "days" which is a little misleading.

Expectation:
setting when to 'midnight', the file is rotated every midnight (interval value 
should be ignored)

Current behavior:
If 'midnight' is given alongside with an interval greater than 1 (.e.g 5), the 
(internal) interval (24*60*60) is calculated with the given interval -> 
24*60*60 * 5.


In my case, this led to some unforeseeable and unexpected behavior.

--
components: Library (Lib)
messages: 410558
nosy: mschiess, vinay.sajip
priority: normal
severity: normal
status: open
title: TimedRotatingFileHandler "midnight" misleading when interval > 1
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue46377>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24364] Not all defects pass through email policy

2021-12-24 Thread mike mcleod



mike mcleod  added the comment:

I would like to help with this issue.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue24364>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4849] instantiating and populating xml.dom.minidom.Element is cumbersome

2021-12-24 Thread mike mcleod



mike mcleod  added the comment:

I would like to help with this issue. I'm new to this space hence I am not 
aware of what patch review means.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue4849>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284670] Allow to restrict ModuleFinder to get "direct" dependencies

2021-12-22 Thread mike mcleod



Change by mike mcleod :


--
pull_requests: +28450
pull_request: https://github.com/python/cpython/pull/30228

___
Python tracker 
<https://bugs.python.org/issue1284670>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-12-20 Thread mike mcleod



Change by mike mcleod :


--
pull_requests: +28437
pull_request: https://github.com/python/cpython/pull/30216

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-12-04 Thread mike mcleod



Change by mike mcleod :


--
keywords: +patch
pull_requests: +28141
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29624

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284670] Allow to restrict ModuleFinder to get "direct" dependencies

2021-12-01 Thread mike mcleod



mike mcleod  added the comment:

Minor fix after test.

--
Added file: https://bugs.python.org/file50467/2.html

___
Python tracker 
<https://bugs.python.org/issue1284670>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284670] Allow to restrict ModuleFinder to get "direct" dependencies

2021-12-01 Thread mike mcleod


mike mcleod  added the comment:

I have made the changes as indicated in the diff files. I have tested against 
the latest from my GitHub copy.
The result is:
./python -m test -j0
== CPython 3.11.0a2+ (heads/main:3a91617590, Dec 1 2021, 15:11:41) [GCC 11.2.0]
== Linux-5.13.0-22-generic-x86_64-with-glibc2.34 little-endian
== cwd: /home/dougal/Documents/cpython/build/test_python_300237æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
...
0:01:34 load avg: 8.77 [221/430/1] test_modulefinder passed
== Tests result: FAILURE ==
409 tests OK.
1 test failed:
test_embed
20 tests skipped:
test_curses test_devpoll test_epoll test_ioctl test_kqueue
test_msilib test_ossaudiodev test_smtpnet test_socketserver
test_startfile test_tix test_tk test_ttk_guionly test_urllib2net
test_urllibnet test_winconsoleio test_winreg test_winsound
test_xmlrpc_net test_zipfile64
Total duration: 5 min 30 sec
Tests result: FAILURE

The first diff is attached. But there was a bug in the code and I will upload 
in next comment.

--
Added file: https://bugs.python.org/file50466/1.html

___
Python tracker 
<https://bugs.python.org/issue1284670>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-11-27 Thread mike mcleod



mike mcleod  added the comment:

I will work on this next week.

--

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284670] Allow to restrict ModuleFinder to get "direct" dependencies

2021-11-27 Thread mike mcleod



mike mcleod  added the comment:

Ok, I will work on this soon and make further comments.

--

___
Python tracker 
<https://bugs.python.org/issue1284670>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284670] Allow to restrict ModuleFinder to get "direct" dependencies

2021-11-19 Thread mike mcleod


mike mcleod  added the comment:

I would like to help with this issue.

I note the history (using Git Dag) of the modulefinder.py shows that it has 
been modified with changes related here:

Author: Éric Araujo   2011-07-28 22:35:29
Committer: Éric Araujo   2011-07-28 22:35:29
Parent: 867754e3e3c907f87d1bb0282df628deb392cd62 (merge 11164)
Child:  cee6bb549946bb096aa380a49f3b11daf2d8713e (Fix bug I unwittingly added 
in 1521d9837d16 (found by Ezio Melotti))
Branches: main, remotes/origin/3.10, remotes/origin/3.6, remotes/origin/3.7, 
remotes/origin/3.8, remotes/origin/3.9, 
remotes/origin/bpo-45514/deprecate-importlib-resources-legacy, 
remotes/origin/buildbot-custom, remotes/origin/main, remotes/upstream/3.10, 
remotes/upstream/3.6, remotes/upstream/3.7, remotes/upstream/3.8, 
remotes/upstream/3.9, 
remotes/upstream/bpo-45514/deprecate-importlib-resources-legacy, 
remotes/upstream/buildbot-custom, remotes/upstream/main, 
remotes/upstream/windeepfreeze
Follows: v3.2.1
Precedes: v3.3.0a1

Modernize modulefinder module and tests a bit.

The tests don’t use an internal distutils function anymore, and use
regular assertEqual with sorted lists instead of a convoluted manual
diff.

and

Author: Éric Araujo   2011-08-01 14:29:07
Committer: Éric Araujo   2011-08-01 14:29:07
Parent: 1e3a68d36b08cd9d59084a37c8cb6c2d911868ce (Modernize modulefinder module 
and tests a bit.)
Child:  cb66eb0deca1d5cd232f97c76a215ecaab958d30 (Issue #13959: Deprecate 
imp.get_suffixes() for new attributes on)
Branches: main, remotes/origin/3.10, remotes/origin/3.6, remotes/origin/3.7, 
remotes/origin/3.8, remotes/origin/3.9, 
remotes/origin/bpo-45514/deprecate-importlib-resources-legacy, 
remotes/origin/buildbot-custom, remotes/origin/main, remotes/upstream/3.10, 
remotes/upstream/3.6, remotes/upstream/3.7, remotes/upstream/3.8, 
remotes/upstream/3.9, 
remotes/upstream/bpo-45514/deprecate-importlib-resources-legacy, 
remotes/upstream/buildbot-custom, remotes/upstream/main, 
remotes/upstream/windeepfreeze
Follows: v3.2.1
Precedes: v3.3.0a1

Fix bug I unwittingly added in 1521d9837d16 (found by Ezio Melotti)

And the modulefinder.py file has been subsequently modified by another 28 
changes.

So I am wondering if this issue is still relevant?

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue1284670>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-11-18 Thread mike mcleod



mike mcleod  added the comment:

Hi Victor,

Ok I'll do that.

Regards,
Mike

On Thu, 18 Nov 2021 at 15:36, STINNER Victor  wrote:

>
> STINNER Victor  added the comment:
>
> I searched for open issues which contain "cgi" in their title. I found 43
> open issues. The oldest is 101 months ago.
>
> In 10 years, Lib/cgi.py got 43 commits. Only 13 commits in the last 5
> years (since 2016-01-01).
>
> It seems like the cgi module is not really maintained anymore. One option
> is to do nothing: I guess that the most basic features continue to work.
> Another option is to start deprecating all code related to CGI in the
> stdlib.
>
> While CGI is not "commonly" used, it seems like it remains popular for
> specific usages. So I don't think that it's time to deprecate it :-)
>
>
> > You can for sure make a pull request and see if a core dev will review
> and merge it
>
> Yep, just do that ;-) So far, nobody proposed a PR to fix this issue.
>
> --
>
> ___
> Python tracker 
> <https://bugs.python.org/issue10483>
> ___
>

--

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10202] ftplib doesn't check close status after sending file

2021-11-12 Thread mike mcleod



mike mcleod  added the comment:

I would like to help on this issue. I understand the arguments here but it has 
been a lone time since this was raised and there does not seem to be any 
further issues discussed or support for this issue.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue10202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6669] TarFile.getmembers fails at struct.unpack: unpack requires a string argument of length 4

2021-11-12 Thread mike mcleod



mike mcleod  added the comment:

I would like to help with this issue.
I note that when I test against 3.10 this error does not show. Also, all tests 
for test_tarfile pass.
I am wondering if it may be reasonable to close this due to the age and 
currently its not an issue in the latest Python version.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue6669>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows



mike mcleod  added the comment:

I have put the question on Discuss and wait for (any) responses,

--

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21501] docs: mmap example for use in documentation



mike mcleod  added the comment:

One of the things I did when the example code didn't work was to see what other 
examples could be found and there are I would argue plenty or just enough, eg:
https://medium.com/analytics-vidhya/memory-mapping-files-and-mmap-module-in-python-with-lot-of-examples-d1a9a45fe9a3
And google will fetch more if needed.
May I suggest back in 2014 when the issue was raised that this may have been a 
good example, but mmap is not so new and in the intervening time anybody who 
needed further examples simply googled them. Hence, I suggest its not needed.

--

___
Python tracker 
<https://bugs.python.org/issue21501>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21501] docs: mmap example for use in documentation



mike mcleod  added the comment:

I would like to help with this issue.
I note that a few changes need to be made to the code example for compatibility 
for the latest version of Python. I can get this working but I am not sure this 
is a good example. Also, the example is more about threads and messaging 
between threads rather than mmap.
So my question is is this a good example?
And the suggestion of adding detailed explanation of the code, again, as to 
what it does not seem to be that relevant to mmap.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue21501>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows



mike mcleod  added the comment:

Should I go ahead and make the changes as per msg122208 ? on my local copy and 
test?

--

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2628] ftplib Persistent data connection

2021-11-02 Thread mike mcleod



mike mcleod  added the comment:

You're welcome.
Regards,
Mike

On Tue, 2 Nov 2021 at 05:52, Ethan Furman  wrote:

>
> Ethan Furman  added the comment:
>
> Ah.  Well, in that case closing seems like the best idea.
>
> Thank you, Jonathan, for getting the CLA signed and providing perspective.
>
> Thank you, Mike, for moving this issue forward.
>
> --
> resolution:  -> rejected
> stage: patch review -> resolved
> status: open -> closed
>
> ___
> Python tracker 
> <https://bugs.python.org/issue2628>
> ___
>

--

___
Python tracker 
<https://bugs.python.org/issue2628>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-11-01 Thread mike mcleod

mike mcleod  added the comment:

Hi  Éric,

Then would the easiest way of dealing with this issue, to close it without
any further work?
Possibly adding to the documents it doesn't work on windows..
I can move to the next issue.

Regards,
Mike

On Sun, 31 Oct 2021 at 18:03, Éric Araujo  wrote:

>
> Éric Araujo  added the comment:
>
> I don’t know if CGI on windows servers is very relevant nowadays.
>
> --
> nosy: +eric.araujo
> versions:  -Python 3.2
>
> ___
> Python tracker 
> <https://bugs.python.org/issue10483>
> ___
>

--

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2628] ftplib Persistent data connection

2021-11-01 Thread mike mcleod



mike mcleod  added the comment:

I am happy to do any testing.
My reason for getting involved is I am new to helping with Cpython and thought 
this may be the least intrusive way of getting started with something that 
nobody really cares about that much.
Hence, the oldest issue I see can be either completed as first envisioned or 
can be closed and I am ok with either.

--

___
Python tracker 
<https://bugs.python.org/issue2628>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

2021-10-31 Thread mike mcleod



mike mcleod  added the comment:

Hi,
I would like to help on this issue.
Let me know what can be done?

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue10483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2628] ftplib Persistent data connection

2021-10-26 Thread mike mcleod



mike mcleod  added the comment:

Hi Ethan,

Thanks, awaiting reply..

Regards,
Mike

On Tue, 26 Oct 2021 at 01:36, Ethan Furman  wrote:

>
> Ethan Furman  added the comment:
>
> We don't have a CLA from jbell.  I've sent an email asking him to do so...
> we'll see what happens.
>
> --
> nosy: +ethan.furman
>
> ___
> Python tracker 
> <https://bugs.python.org/issue2628>
> ___
>

--

___
Python tracker 
<https://bugs.python.org/issue2628>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2628] ftplib Persistent data connection

2021-10-24 Thread mike mcleod



mike mcleod  added the comment:

Hi,

I would like to help on this issue.

--
nosy: +mikecmcleod

___
Python tracker 
<https://bugs.python.org/issue2628>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45433] libpython should not be linked with libcrypt

2021-10-11 Thread Mike Gilbert



Change by Mike Gilbert :


--
keywords: +patch
pull_requests: +27176
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28881

___
Python tracker 
<https://bugs.python.org/issue45433>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45433] libpython should not be linked with libcrypt

2021-10-11 Thread Mike Gilbert



New submission from Mike Gilbert :

In https://bugs.python.org/issue44751, crypt.h was removed from Python.h. This 
would imply that libpython is not meant to expose any crypt-related symbols.

In fact, it looks like libpython does not use crypt() or crypt_r() at all. 
These are only used by cryptmodule.

In configure.ac, we have this this logic to determine if crypt_r() is available:

```
# We search for both crypt and crypt_r as one or the other may be defined
# This gets us our -lcrypt in LIBS when required on the target platform.
AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(crypt_r, crypt)

AC_CHECK_FUNC(crypt_r,
  AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#define _GNU_SOURCE  /* Required for crypt_r()'s prototype in glibc. */
#include 
]], [[
struct crypt_data d;
char *r = crypt_r("", "", );
]])],
[AC_DEFINE(HAVE_CRYPT_R, 1, [Define if you have the crypt_r() function.])],
[])
)
```

The AC_SEARCH_LIBS macro adds "-lcrypt" to LIBS, and this gets passed down to 
the link command for libpython. This is probably not intentional.

The HAVE_CRYPT_R value is used in _cryptmodule.c. setup.py performs its own 
check for libcrypt when building the module.

I think the value of LIBS should be saved before the AC_SEARCH_LIBS calls and 
restored after the AC_CHECK_FUNC call. I have tested this locally on Gentoo 
Linux, and it seems to work fine.

I will work on a pull request.

--
components: C API
messages: 403663
nosy: floppymaster
priority: normal
severity: normal
status: open
title: libpython should not be linked with libcrypt
type: compile error
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue45433>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41710] acquire(timeout) of threading.Lock and threading.Condition is affected by jumps in system time: Python should use sem_clockwait(CLOCK_MONOTONIC)

2021-10-01 Thread Mike Crowe

Mike Crowe  added the comment:

vstinner wrote:
> The glibc 2.30 adds pthread_cond_clockwait() which could be used to use
> CLOCK_MONOTONIC. But if pthread_cond_clockwait() is available (glibc 
> 2.30 or newer), it expects that pthread_condattr_setclock() is also 
> available. So I'm not sure that it's worth it to change 
> PyCOND_TIMEDWAIT().

That's correct. The only time that pthread_cond_clockwait is essential is if 
you don't know which clock you're going to need to use for the wait at the time 
of condition variable creation. (This is true for a generic condition variable 
wrapper that can be passed absolute timeouts using different clocks like the 
C++ std::condition_variable.)

However, sem_clockwait is the only way to wait on a semaphore using 
CLOCK_MONOTONIC, so it is worth using that rather than sem_timedwait if it's 
available. Similarly for pthread_mutex_clocklock.

--

___
Python tracker 
<https://bugs.python.org/issue41710>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27035] Cannot set exit code in atexit callback

2021-08-19 Thread Mike Hommey



Mike Hommey  added the comment:

> In 2.7 the only exception that _did_ change the exit code was SystemExit.

(and only if it was the last thrown exception)

--

___
Python tracker 
<https://bugs.python.org/issue27035>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27035] Cannot set exit code in atexit callback

2021-08-19 Thread Mike Hommey



Mike Hommey  added the comment:

> I think we should change the documentation to expand the parenthetical " 
> (unless SystemExit is raised)" to a complete explanation of that special case.

That would not be enough, since the case for other exceptions would still be 
ambiguous, as the described behavior ("After all exit handlers have had a 
chance to run the last exception to be raised is re-raised.") would imply the 
exit code would be altered, like when exceptions are raised in normal context. 
In 2.7 the only exception that _did_ change the exit code was SystemExit.

--
nosy: +Mike Hommey

___
Python tracker 
<https://bugs.python.org/issue27035>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41710] Timeout is affected by jumps in system time

2021-08-11 Thread Mike Crowe



Mike Crowe  added the comment:

glibc v2.30 onwards provides sem_clockwait which can wait on either 
CLOCK_MONOTONIC or CLOCK_REALTIME. I failed to notice that 
https://sourceware.org/bugzilla/show_bug.cgi?id=14717 existed until today. :(

--
nosy: +mikecrowe

___
Python tracker 
<https://bugs.python.org/issue41710>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44352] Native Windows Python builds running on Europe/Moscow TZ report wrong time from datetime.datetime.now when there is TZ environment variable also set to Europe/Moscow

2021-06-11 Thread Mike Kaganski



Mike Kaganski  added the comment:

@Eryk Sun: yes, of course you are right - but please see the date of the 
commit; I didn't know what you kindly explained me in your reply yesterday :-) 
Thank you again.

--

___
Python tracker 
<https://bugs.python.org/issue44352>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44352] Native Windows Python builds running on Europe/Moscow TZ report wrong time from datetime.datetime.now when there is TZ environment variable also set to Europe/Moscow

2021-06-10 Thread Mike Kaganski



Mike Kaganski  added the comment:

Thank you Eryk! This is a good workaround for me. I have implemented it: 
https://git.libreoffice.org/core/+/3bcaa4ba79477a21251ddaa06e0ea159196a7ffb

It looks like it's not a Python's problem; I suppose this may be closed. Thanks 
again!

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue44352>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44352] Native Windows Python builds running on Europe/Moscow TZ report wrong time from datetime.datetime.now when there is TZ environment variable also set to Europe/Moscow

2021-06-08 Thread Mike Kaganski

New submission from Mike Kaganski :

On a Windows 10 system, which TZ is set to Moscow (UTC+3), I use a native
Windows Python build (as opposed to e.g. one from Cygwin). Specifically, I
tested both custom Python build created by LibreOffice project, as well as the
Python by Python Software Foundation available from MS Store [1].

1. Open command prompt on Windows (cmd.exe).
2. Execute 'set TZ=Europe/Moscow'
3. Execute 'python'
4. In the Python prompt, execute 'import datetime'
5. Execute 'datetime.datetime.now()'

The result of this is a time that is two hours off, e.g.

> datetime.datetime(2021, 6, 8, 19, 59, 21, 925240)

instead of proper

> datetime.datetime(2021, 6, 8, 21, 59, 21, 925240)

which appears when step #2 is skipped.
Possibly Python takes both system time zone information *and* the environment
variable into account when calculating the time. I suppose it should only
consider one or the other, not both.

Note that the problem does not happen with Cygwin's Python, which works fine
with the same TZ environment variable value.

For a real-life problem that results from this, see case at [2], where unit
test is failing only from 00:00 till 02:00 on my local system, obviously
telling me that I should sleep at that time, not try to run unit tests :-)

[1] https://www.microsoft.com/en-us/p/python-38/9mssztt1n39l
[2]
https://gerrit.libreoffice.org/c/core/+/92217/2#message-f55091795e7cde9d75adc00ddb69451121b644f6

--
components: Windows
messages: 395355
nosy: mikekaganski, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Native Windows Python builds running on Europe/Moscow TZ report wrong
time from datetime.datetime.now when there is TZ environment variable also set
to Europe/Moscow
type: behavior
versions: Python 3.8

___
Python tracker
<https://bugs.python.org/issue44352>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44304] segmentation fault appeared in python 3.10.0b2

2021-06-05 Thread mike bayer



mike bayer  added the comment:

great news!   

Based on how many random factors were needed to reproduce as well as that it 
seemed to be gc related and appeared very suddenly, I had an intuition this was 
on the cpython side, thanks so much for doing this Pablo!

--

___
Python tracker 
<https://bugs.python.org/issue44304>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44304] segmentation fault appeared in python 3.10.0b2

2021-06-04 Thread mike bayer



mike bayer  added the comment:

yes, if I have time I will begin to undertake that, wanted to put it up here in 
case anyone has git bisect on speed dial for cpython.

--

___
Python tracker 
<https://bugs.python.org/issue44304>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44304] segmentation fault appeared in python 3.10.0b2

2021-06-03 Thread mike bayer



mike bayer  added the comment:

if the issue is in greenlet this can be bounced back to 
https://github.com/python-greenlet/greenlet/issues/242

--

___
Python tracker 
<https://bugs.python.org/issue44304>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44304] segmentation fault appeared in python 3.10.0b2

2021-06-03 Thread mike bayer



New submission from mike bayer :

segmentation fault related to object deallocation and traceback objects, is 
extremely difficult to reproduce and definitely appeared as of 3.10.0b2, does 
not occur in 3.10.0b1.   linux and osx platforms are affected.

The issue requires "greenlet==1.1.0" to be installed, as well as that for me to 
reproduce it I have to use sqlite3 with some special APIs also, so while this 
issue might be in greenlet, or sqlite3, I have a feeling these are all factors 
that are combining together in a very random way to reveal something that's 
actually happening in the interpreter, but in any case would be great if core 
devs can see why it happens.  The below script has been tested on various linux 
platforms, and the overall bug was revealed by an interaction in the SQLAlchemy 
test suite that's occurring in many places including OSX, linux.   As noted, a 
whole bunch of very random things are needed for me to reproduce it.


import greenlet
import sqlite3


class _ErrorContainer(object):
error = None


def _expect_raises_fn(fn):
ec = _ErrorContainer()
try:
fn()
except Exception as err:
assert str(err) == "this is a test"

# assign the exception context outside of the except
# is necessary
ec.error = err

# don't del the exception context is necessary
#del ec


def greenlet_spawn(fn, *args, **kwargs):

# spawning a greenlet is necessary
context = greenlet.greenlet(fn, greenlet.getcurrent())

# assignment to "result" is necessary
result = context.switch(*args, **kwargs)

# raising exception is necessary
raise Exception("this is a test")


class OuterConnectionWrapper:
def __init__(self, connection):
self.connection = connection

def go(self, stmt):
sqlite_connection = self.connection
cursor = sqlite_connection.cursor()
cursor.execute("select 1")
return cursor

def execute(self, stmt):
return greenlet_spawn(self.go, stmt)

def _do_close(self):
self.connection.close()
self.connection = None

def close(self):
self._do_close()


class InnerConnectionWrapper:
def __init__(self, connection):
self.connection = connection

def create_function(self, *arg, **kw):
self.connection.create_function(*arg, **kw)

def cursor(self):
return self.connection.cursor()

def close(self):
self.connection = None


class ConnectionPool:
def __init__(self):
self.conn = sqlite3.connect(":memory:")

def regexp(a, b):
return None

self.conn.create_function("regexp", 2, regexp)

def connect(self):
return InnerConnectionWrapper(self.conn)


def do_test():
pool = ConnectionPool()

def go():
c1 = pool.connect()
conn = OuterConnectionWrapper(c1)
try:
conn.execute("test")
finally:
conn.close()

_expect_raises_fn(go)


do_test()

--
components: Interpreter Core
messages: 395028
nosy: zzzeek
priority: normal
severity: normal
status: open
title: segmentation fault appeared in python 3.10.0b2
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44304>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.

2021-05-06 Thread Mike Lissner



Mike Lissner  added the comment:

>  With the fix for this bug, urlsplit silently removes (some of) those 
> characters before we can replace them, modifying the output of our 
> sanitisation code

I don't have any good solutions for 3.9.5, but going forward, this feels like 
another example of why we should just do parsing right (the way browsers do). 
That'd maintain tabs and whatnot in your output, and it'd fix the security 
issue by putting `java\nscript` into the scheme attribute instead of the path.

> One solution that presents itself to me: add a `strip_insecure_characters: 
> bool = True` parameter.

Doesn't this lose sight of what this tool is supposed to do? It's not supposed 
to have a good (new, correct) and a bad (old, obsolete) way of parsing. Woe 
unto whoever has to write the documentation for that parameter. 

Also, I should reiterate that these aren't "insecure" characters so if we did 
have a parameter for this, it'd be more like `do_rfc_3986_parsing` or maybe 
`do_naive_parsing`. The chars aren't insecure in themselves. They're fine. 
Python just gets tripped up on them.

--

___
Python tracker 
<https://bugs.python.org/issue43882>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.

2021-05-05 Thread Mike Lissner



Mike Lissner  added the comment:

> I'd wonder how to pass through valid exceptions without urlparse raising 
> something.

Oops, meant to say "valid URLs", not valid exceptions, sorry.

--

___
Python tracker 
<https://bugs.python.org/issue43882>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.

2021-05-05 Thread Mike Lissner



Mike Lissner  added the comment:

> Instead of the patches as you see them, we could've raised an exception.

In my mind the definition of a valid URL is what browsers recognize. They're 
moving towards the WHATWG definition, and so too must we. 

If we make python raise an exception when a URL has a newline in the scheme 
(e..g: "htt\np"), we'd be raising exceptions for *valid* URLs as browsers 
define them. That doesn't seem right at all to me. I'd be frustrated to have to 
catch such an exception, and I'd wonder how to pass through valid exceptions 
without urlparse raising something.


> Making the output 'sanitized' means that invalid input is converted into 
> valid output.  This goes against the principle of least surprise.

Well, not quite, right? The URLs this fixes *are* valid according to browsers. 
Browsers say these tabs and newlines are OK. 



I agree though that there's an issue with the approach of stripping input in a 
way that affects output. That doesn't seem right. 

I think the solution I'd favor (and I imagine what's coming in 43883) is to do 
this properly so that newlines are preserved in the output, but so that the 
scheme is also placed properly in the scheme attribute. 

So instead of this (from the initial report):

> In [9]: from urllib.parse import urlsplit
> In [10]: urlsplit("java\nscript:alert('bad')")
> Out[10]: SplitResult(scheme='', netloc='', path="java\nscript:alert('bad')", 
> query='', fragment='')

We get something like this:

> In [10]: urlsplit("java\nscript:alert('bad')")
> Out[10]: SplitResult(scheme='java\nscript', netloc='', path="alert('bad')", 
> query='', fragment='')

In other words, keep the funky characters and parse properly.

--

___
Python tracker 
<https://bugs.python.org/issue43882>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.

2021-05-04 Thread Mike Lissner



Mike Lissner  added the comment:

I haven't watched that Blackhat presentation yet, but from the slides, it seems 
like the fix is to get all languages parsing URLs the same as the browsers. 
That's what @orsenthil has been doing here and plans to do in 
https://bugs.python.org/issue43883.

Should we get a bug filed with requests/urllib3 too? Seems like a good idea if 
it suffers from the same problems.

--

___
Python tracker 
<https://bugs.python.org/issue43882>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43882] urllib.parse should sanitize urls containing ASCII newline and tabs.

2021-04-19 Thread Mike Lissner



Change by Mike Lissner :


--
nosy: +Mike.Lissner

___
Python tracker 
<https://bugs.python.org/issue43882>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43883] Making urlparse WHATWG conformant

2021-04-19 Thread Mike Lissner



Change by Mike Lissner :


--
nosy: +Mike.Lissner

___
Python tracker 
<https://bugs.python.org/issue43883>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43484] valid datetimes can become invalid if the timezone is changed

2021-03-19 Thread mike bayer



mike bayer  added the comment:

> 
I don't really know why it would be a "security vulnerability", but presumably 
a library could either convert their datetimes to UTC as soon as they get them 
from the user if they want to use them as UTC in the future, or they could 
simply refuse to accept any datetimes outside the range 
`datetime.datetime.MINYEAR + timedelta(hours=48) < dt.replace(tzinfo=None) < 
datetime.datetime.MAXYEAR - timedelta(hours=48)`, 

this is absolutely correct, but I'm not sure if you're aware, there's kind of a 
whole subsection of the tech community that considers anything that a user 
might do without understanding all the consequences which could in any way 
allow untrusted input to affect things to be a "security risk".  In SQLAlchemy 
i had CVEs posted because we have a method called order_by() that accepted a 
string, and the notion was, someone will write a web app that takes an 
arbitrary string as input and send it there!  CVE!   For you and me that would 
of course be crazy as this is obviously part of the SQL string being sent to 
the database,  but this is a particular programming subculture that has the 
ability to create a lot of havoc by filling up the CVE system with "Security 
Vulnerabilities" based on what many of us consider obviously wrong.

> Can you clarify why this crashes? Is it because it always returns the 
> datetime value in UTC?

it returns the datetime value in the default timezone set up for the server 
which could be UTC or a local timezone, but the idea is it's potentially 
different from the timezone that's been put in.

> I'll note that I don't actually understand the difference between 
> "abstraction layer" and "psycopg2 driver", so I may be conflating those two, 

from my POV I have always thought PostgreSQLs' TIMESTAMP WITH TIMEZONE datatype 
is nuts, and that you should only be sending UTC timestamps to a database.   
But people want to use PG's type and IMO they need to understand what they're 
doing.  thanks for the response.

--

___
Python tracker 
<https://bugs.python.org/issue43484>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43493] EmailMessage mis-folding headers of a certain length

2021-03-17 Thread Mike Glover



Mike Glover  added the comment:

Further research shows that email.parser.Parser is not handling the affected 
lines correctly -- the leading '\n ' is not being stripped from the header 
value.

Attached is the (ugly, worksforme) function I'm using to workaround this problem

--
Added file: https://bugs.python.org/file49889/foldfix.py

___
Python tracker 
<https://bugs.python.org/issue43493>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43493] EmailMessage mis-folding headers of a certain length

2021-03-14 Thread Mike Glover



New submission from Mike Glover :

The attached file demonstrates the incorrect folding behavior I'm seeing.  
Header lines of a certain total length get folded after the colon following the 
header name, which is not valid RFC.  Slightly longer or shorter lines are 
folded correctly.

Interestingly, the test file produces correct output on 3.5.2

$ python --version
Python 3.8.5

$ sudo apt install python3
...
python3 is already the newest version (3.8.2-0ubuntu2).

(yes, that difference has me scratching my head)

And yes, I realize this is not the latest release of the 3.8 branch, but it 
*is* the latest available through apt on Ubuntu 20.04 LTS, and a search of the 
issue tracker and the release notes for all of 3.8.* turned up nothing 
applicable.

--
components: email
files: header_misfolding.py
messages: 388687
nosy: barry, mglover, r.david.murray
priority: normal
severity: normal
status: open
title: EmailMessage mis-folding headers of a certain length
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49875/header_misfolding.py

___
Python tracker 
<https://bugs.python.org/issue43493>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43484] we can create valid datetime objects that become invalid if the timezone is changed

2021-03-12 Thread mike bayer



New submission from mike bayer :

So I'm pretty sure this is "not a bug" but it's a bit of a problem and I have a 
user suggesting the "security vulnerability" bell on this one, and to be honest 
I don't even know what any library would do to "prevent" this.

Basically, the datetime() object limits based on a numerical year of MINYEAR, 
rather than limiting based on an actual logical date.

So I can create an "impossible" date as follows:


d = datetime.strptime("Mon Jan  1 00:00:00 0001 +01:00", "%c %z")

or like this:

d = datetime(year=1, month=1, day=1, tzinfo=timezone(timedelta(hours=1)))

andyou can see where this is going - it can't be converted to a timezone 
that pushes the year to zero:

>>> from datetime import datetime, timezone, timedelta
>>> d = datetime(year=1, month=1, day=1, tzinfo=timezone(timedelta(hours=1)))
>>> d.astimezone(timezone.utc)
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: date value out of range

this because, after all, astimezone() is just subraction or addition and if it 
overflows past the artificial boundary, well you're out of luck.

Why's this a security problem?   ish?because PostgreSQL has a data type 
"TIMESTAMP WITH TIMEZONE" and if you take said date and INSERT it into your 
database, then SELECT it back using any Python DBAPI that returns datetime() 
objects like psycopg2, if your server is in a timezone with zero or negative 
offset compared to the given date, you get an error.  So the mischievous user 
can create that datetime for some reason and now they've broken your website 
which can't SELECT that table anymore without crashing.

So, suppose you maintain the database library that helps people send data in 
and out of psycopg2.We have, the end user's application, we have the 
database abstraction library, we have the psycopg2 driver, we have Python's 
datetime() object with MIN_YEAR, and finally we have PostgreSQL with the 
TIMEZONE WITH TIMESTAMP datatype that I've never liked.

Of those five roles, whose bug is this?I'd like to say it's the end user 
for letting untrusted input that can put unusual timezones and timestamps in 
their system.   But at the same time it's a little weird Python is letting me 
create this date that lacks the ability to convert into UTC. thanks for 
reading!

--
components: Library (Lib)
messages: 388585
nosy: zzzeek
priority: normal
severity: normal
status: open
title: we can create valid datetime objects that become invalid if the timezone 
is changed
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43484>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43180] Lists without commas?

2021-02-09 Thread Mike Siegel



New submission from Mike Siegel :

Perhaps I'm missing something here but this behavior seems unexpected to me. 
"Lists

The items of a list are arbitrary Python objects. Lists are formed by 
placing a *comma-separated* list of expressions in square brackets. (Note that 
there are no special cases needed to form lists of length 0 or 1.)"

>>> import sys
>>> print(sys.version)
3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0]
>>> test = [ "foo" "bar" ]
>>> type(test)

>>> print(test)
['foobar']
>>> import json
>>> json.dumps({ "foo": test })
'{"foo": ["foobar"]}'
>>>

In a complex dict structure this is an easy way to shoot yourself in the foot.

--
messages: 386725
nosy: mikesiegel
priority: normal
severity: normal
status: open
title: Lists without commas?

___
Python tracker 
<https://bugs.python.org/issue43180>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39457] Add an autocommit property to sqlite3.Connection with a PEP 249 compliant manual commit mode and migrate

2021-01-06 Thread mike bayer



mike bayer  added the comment:

I think this issue just discusses the naming of an attribute called 
".autocommit".  for the discussion for SQLite's unusual starting of 
transactions, that's all in two other issues:

https://bugs.python.org/issue9924 

https://bugs.python.org/issue10740

so I would encourage folks to read those discussions.  at issue is the 
limitation of SQLite that it locks the whole file for transactions, which is 
the main rationale for why SQLite is hesitant to begin a transaction.  however, 
without configurability, this means it's not compatible with SAVEPOINT or 
serializable isolation levels.   when users want to use those two features we 
have them set isolation_level=None and emit "BEGIN" on the connection directly. 
   the connection.commit() and connection.rollback() methods continue to be 
functional

--

___
Python tracker 
<https://bugs.python.org/issue39457>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39457] Add an autocommit property to sqlite3.Connection with a PEP 249 compliant manual commit mode and migrate

2021-01-05 Thread mike bayer



mike bayer  added the comment:

> Under your proposal, the first line would be changed to say 
> "autocommit=True", even though not all the code below is in autocommit mode 
> (according to the SQLite engine's definition). What's more, I could insert 
> this line of code between statements 3 and 6:

  >  print("Autocommit mode?", conn.autocommit)

As Marc-Andre indicated, this is in fact how "autocommit" behaves on other 
drivers, including:

psycopg2 - uses either connection.autocommit, or 
extensions.ISOLATION_LEVEL_AUTOCOMMIT
pymysql - uses conn.autocommit(True|False)
mysqldb - uses conn.autocommit(True|False)
cx_Oracle - uses conn.autocommit
pyodbc - uses conn.autocommit

With all of the above drivers, one can emit "BEGIN" and "COMMIT" using
connection.execute(), and within the scope of that BEGIN/COMMIT pair, the 
database "engine" itself is in a transaction.  The "driver" however remains in 
"autocommit" mode.   This mode specifically means the driver is not getting
involved in starting and stopping transactions. 

As Marc mentions, we're not supposed to be emitting "BEGIN" and "COMMIT" on 
the driver, but none of them get in the way of us doing so, and additionally
most databases support additional options for the "BEGIN/START TRANSACTION" 
phase
that aren't available in the DBAPIs so sometimes we don't have much choice at 
least for the "BEGIN" command.

Here's an example using psycopg2, where the timestamp now() will freeze
when we're in a transaction started using manual "BEGIN"/ "COMMIT", while 
.autocommit stays True, and otherwise match statement execution time if we're 
not:

>>> import psycopg2
>>> conn = psycopg2.connect(user="scott", password="tiger", 
host="localhost", database="test")
>>> conn.autocommit = True
>>> cursor = conn.cursor()
>>> cursor.execute("SELECT 1")
>>> cursor.execute("select now() = statement_timestamp()")
>>> cursor.fetchall()
[(True,)]
>>> cursor.execute("BEGIN")
>>> cursor.execute("select now() = statement_timestamp();")
>>> cursor.fetchall()
[(False,)]  # we're in a transaction
>>> conn.autocommit   # still in driver-level autocommit
True
>>> cursor.execute("COMMIT")
>>> cursor.execute("select now() = statement_timestamp();")
>>> cursor.fetchall()
[(True,)]

For SQLAlchemy we already support pysqlite's "isolation_level=None" to 
implement "autocommit" so this issue does not affect us much, but the meaning 
of the term "autocommit" at the driver level shouldn't be controversial at this 
point as there's a lot of precedent.   "connection.autocommit" does not refer 
to the current transactional state of the database, just the current preference 
set upon the driver itself.

--

___
Python tracker 
<https://bugs.python.org/issue39457>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42771] Implement interactive hotkey, Ctrl+L to clear the console in Windows.

2020-12-28 Thread Mike Miller



Mike Miller  added the comment:

I found an implementation of this for Windows in case it is needed.  Not sure 
if it is the best way to do it, as the Console API is rather clumsy.  However 
this one works to my knowledge:

https://github.com/tartley/colorama/blob/master/colorama/winterm.py#L111

--

___
Python tracker 
<https://bugs.python.org/issue42771>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42771] Implement interactive hotkey, Ctrl+L to clear the console in Windows.

2020-12-28 Thread Mike Miller



New submission from Mike Miller :

The Ctrl+L as clear-screen hotkey is supported just about everywhere, Unix and 
Windows, with the exceptions of cmd.exe and python.exe interactive mode.

As the legacy cmd.exe can be easily replaced, that leaves python.exe.  Likely 
needs to be configured via readline or its analog used under Windows.

Documenting it would be good too.  Am happy to help, write, or test.

--
components: Windows
messages: 383917
nosy: mixmastamyk, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Implement interactive hotkey, Ctrl+L to clear the console in Windows.
type: enhancement

___
Python tracker 
<https://bugs.python.org/issue42771>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

2020-12-20 Thread mike bayer



Change by mike bayer :


--
nosy: +zzzeek

___
Python tracker 
<https://bugs.python.org/issue16535>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42555] math function sqrt() not working in 3.9

2020-12-03 Thread mike dalrymple



New submission from mike dalrymple :

Downloaded Python 3.9.0
Documentation indicates:
math.sqrt(x)
Return the square root of x.

When I use in IDLE shell 3.9.0, I receive error:
>>> sqrt(25)
Traceback (most recent call last):
  File "", line 1, in 
sqrt(25)
NameError: name 'sqrt' is not defined

What is the problem?

--
assignee: terry.reedy
components: IDLE
messages: 382411
nosy: mikeuser_01, terry.reedy
priority: normal
severity: normal
status: open
title: math function sqrt() not working in 3.9
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42555>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42341] xml.dom.minidom parsing omits Text nodes in top level

2020-11-12 Thread Mike Frysinger



New submission from Mike Frysinger :

$ python3
Python 3.8.5 (default, Aug  2 2020, 15:09:07) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from xml.dom import minidom

# Lets parse a simple XML file with comment & text nodes in the top level.
>>> dom = minidom.parseString('>> encoding="UTF-8"?>\n\n\n\n\n\n\n')

# Where did those newlines get to outside of  ?
>>> dom.toxml()
'\n\n\n'

# No Text nodes in the root list :(.
>>> dom.childNodes
[, , ]

# But they all exist fine under .
>>> dom.childNodes[2].childNodes
[, , , 
, ]

--
components: XML
messages: 380872
nosy: vapier
priority: normal
severity: normal
status: open
title: xml.dom.minidom parsing omits Text nodes in top level
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue42341>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23436] xml.dom.minidom.Element.ownerDocument is hidden

2020-11-12 Thread Mike Frysinger



Change by Mike Frysinger :


--
title: xml.dom.minidom.Element.ownerDocument is hiden -> 
xml.dom.minidom.Element.ownerDocument is hidden

___
Python tracker 
<https://bugs.python.org/issue23436>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42253] xml.dom.minidom.rst missing standalone documentation

2020-11-12 Thread Mike Frysinger



Change by Mike Frysinger :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
title: xml.dom.minidom.rst missed informations -> xml.dom.minidom.rst missing 
standalone documentation

___
Python tracker 
<https://bugs.python.org/issue42253>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42173] Drop Solaris support

2020-10-30 Thread Mike La Spina


Mike La Spina  added the comment:

I suspect the download stats are not a full indication of usage on “SunOS” 
based on download stats. The bits are preinstalled. I do use it frequently via 
Illumos. Please support the platform.

Regards,
Mike

--
nosy: +michael.laspina

___
Python tracker 
<https://bugs.python.org/issue42173>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41629] class not set defining 'X' as . Was classcell propagated to type.new?

2020-08-24 Thread mike bayer



New submission from mike bayer :

This is likely related or a dupe of https://bugs.python.org/issue29270, but the 
error message is different.  I'm posting this to confirm it's the same issue, 
or not, and to at least provide a google result for people who also see this 
error as 29270 seems to imply this might not be fixable.

Like 29270, it involves the fact that the interpreter seems to be looking at my 
super() call inside of a method without actually calling it, and then getting 
upset about __classcell__:



from typing import NamedTuple


class X(NamedTuple):
a: str
b: str

# comment this out to remove the issue
def foo(self):
return super(X, self)


and that's it!  on my interpreter:

Python 3.8.3 (default, May 23 2020, 16:34:37) 
[GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] on linux

I get:

$ python test3.py 
Traceback (most recent call last):
  File "test3.py", line 4, in 
class X(NamedTuple):
RuntimeError: __class__ not set defining 'X' as . Was 
__classcell__ propagated to type.__new__?

The most surprising thing is that this seems extremely basic and google is not 
finding this error message for me.

--
components: Interpreter Core
messages: 375863
nosy: zzzeek
priority: normal
severity: normal
status: open
title: __class__ not set defining 'X' as . Was 
__classcell__ propagated to type.__new__?
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue41629>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-08 Thread mike bayer



mike bayer  added the comment:

I tested "cancellation", shutting down the DB connection mid query.  Because 
the greenlet is only in the middle and not at the endpoints, it propagates the 
exception and there does not seem to be anything different except for the 
greenlet sequence in the middle, which is also clear:

https://gist.github.com/zzzeek/9e0d78eff14b3bbd5cf12fed8b02bce6

the first comment on the gist has the stack trace produced.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-08 Thread mike bayer



mike bayer  added the comment:

as far as cancellation, I gather you're referring to what in gevent / greenlet 
is the GreenletExit exception.  Sure, that thing is a PITA.   Hence we're all 
working to provide asyncio frontends and networking backends so that the 
effects of cancellation I (handwavy handwavy) believe would work smoothly as 
long as the middle part is done right.   cancellation is likely a more 
prominent issue with HTTP requests and responses because users are hitting 
their browser stop buttons all the time.  With databases this typically is 
within the realm of network partitioning or service restarts, or if the driver 
is screwing up in some way which with the monkeypatching thing is more likely, 
but "cancellation" from a database perspective is not the constant event that I 
think it would be in an HTTP perspective.


> I think I either disagree or am missing something :-). Certainly for both 
> edgedb and urllib3, when they're running in sync mode, they end up using 
> synchronous network APIs at the "bottom", and it works fine.

OK it took me a minute to understand what you're saying, which is, if we are 
doing the coroutine.send() thing you illustrated below, we're not in an event 
loop anyway so we can just call blocking code.   OK I did not understand that.  
I haven't looked at the coroutine internals through all of this (which is part 
of my original assertion that I should not have been the person proposing this 
whole greenlet thing anyway :) ).

Why did urllib3 write unasync?  https://pypi.org/project/unasync/strictly 
so they can have a python 2 codebase and that's it?   

SQLAlchemy goes python 3 only in version 2.0.  I did bench the coro example 
against a non-coro example and it's 3x slower likely due to the StopIteration 
but as mentioned earlier if this is only once per front-to-back then it would 
not amount to anything in context.   Still, the risk factor of a rewrite like 
that, where risk encompasses just all the dumb mistakes and bugs that would be 
introduced by rewriting everything, does not seem worth it.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop



mike bayer  added the comment:

slight correction: it is of course possible to use gevent with a database 
driver without monkeypatching, as I wrote my own gevent benchmarks using 
psycogreen.  I think what I'm getting at is that it's a good thing if async 
DBAPIs could target asyncio explicitly rather than having to write different 
gevent/eventlet specific things, and that tools like SQLAlchemy can allow for 
greenlet style coding against those DBAPIs without one having to install/run 
the whole gevent event loop.   Basically I like the greenlet style of coding 
but I would be excited to skip the gevent part, never do any monkeypatching 
again, and also have other parts of the app doing asyncio work with other kinds 
of services. this is about interoperability.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop



mike bayer  added the comment:

> Oh, I thought the primary problem for SQLAlchemy supporting async is that the 
> ORM needs to do IO from inside __getattr__ methods. So I assumed that the 
> reason you were so excited about greenlets was that it would let you use 
> await_() from inside those __getattr__ calls, which would involve exposing 
> your use of greenlets as part of your public API.


The primary problem is people want to execute() a SQL statement using await, 
and then they want to use a non-blocking database driver (basically asyncpg, 
I'm not sure there are any others, maybe there's one for MySQL also) on the 
back.Tools like aiopg have provided partial SQLAlchemy-like front-ends to 
accomplish this but they can't do ORM support, not because the ORM has lazy 
loading, but just to do explicit operations like query.all() or session.flush() 
that can sometimes require a lot of front-to-back database operations to 
complete which would be very involved to rewrite all that code using 
async/await.

Then there's the secondary problem of ORMs doing lazy loading, which is what 
you refer towards as "IO inside __getattr__ methods".   SQLAlchemy is not 
actually as dependent on lazy loading as other ORMs as we support a wide range 
of ways to "eagerly" load data up front.  With the SQLAlchemy 2.0-style ORM API 
that has a clear spot for "await" to occur, they can call "await 
session.execute(select(SomeObject))" and get a whole traversible graph of 
things loaded up front.We even have a loader called "raiseload" that is 
specifically anti-lazy loading, it's a loader that raises an error if you try 
to access something that wasn't explicitly loaded already.  So for a lot of 
cases we are already there.

But then, towards your example of "something.b = x", or more commonly in ORMS a 
get operation like "something.b" emitting SQL, the extension I'm building will 
very likely include some kind of feature that they can do this with an explicit 
call.  At the moment with the preliminary code that's in there, this might look 
like:

   await greenlet_spawn(getattr, something, "b")

not very pretty at the moment but that general idea.   

But the thing is, greenlet_spawn() can naturally apply to anything.  So it 
remains to be seen both how I would want to present this notion, as well as if 
people are going to be interested in it or not, but as a totally extra thing 
beyond the "await session.execute()" API that is the main thing, someone could 
do something like this:

   await greenlet_spawn(my_business_orm_method)

and then in "my_business_orm_method()", all the blocking style ORM things that 
async advocates warn against could be happening in there. I'm certainly not 
going to tell people they have to be doing that, but I dont think I should 
discourage it either, because if the above business method is written 
"reasonably" (see next paragraph), there really is no problem introduced by 
implicit IO.

By "written reasonably" I'm referring to the fact that in this whole situation, 
90% of everything people are doing here are in the context of HTTP services.   
The problem of, "something.a now creates state that other tasks might see" is 
not a real "problem" that is solved by using IO-only explicit context 
switching.  This is because in a CRUD-style application, "something" is not 
going to be a process-local yet thread-global object that had to be created 
specifically for the application (there's things like the database connection 
pool and some registries that the ORM uses, but those problems are taken care 
of and aren't specific to one particular application). There is certainly 
going to be global mutable state with the CRUD/HTTP application which is the 
database itself.  Event based programming doesn't save you from concurrency 
issues here because any number of processes maybe accessing the database at the 
same time.  There are well-established concurrency patterns one uses wit
 h relational databases, which include first and foremost transaction 
isolation, but also things like compare-and-swap, "select for update", ensuring 
MVCC is turned on (SQL Server), table locks, etc.  These techniques are 
independent of the concurrency pattern used within the application, and they 
are arguably better suited to blocking-style code in any case because on the 
database side we must emit our commands within a transaction serially in any 
case.   The major convenient point of "async" that we can fire off a bunch of 
web service requests in parallel does not apply to the CRUD-style business 
methods within our web service request because we can only do things in our 
ACID transaction one at a time.

The problem of "something.a" emitting IO needs to be made sane against other 
processes also view

[issue22239] asyncio: nested event loop



mike bayer  added the comment:

> With greenlets OTOH, it becomes possible for another task to observe 
> someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally 
> invoked an await_().

let me try this one more time.Basically if someone wrote this:

async def do_thing():
   someobj.a =1
   await do_io_setattr(someobj, "b", 2)

then in the async approach, you can again say, assuming "someobj" is global, 
that another task can observe "someobj.a == 1" without "someobj.b == 2".I 
suppose you are making the argument that because there's an "await" keyword 
there, now everything is OK because the reader of the code knows there's a 
context switch.

Whether or not one buys that, the point of my approach is that SQLAlchemy 
itself *will* publish async methods.  End user code *will not* ever context 
switch to another task without them explicitly calling using an await.  That 
SQLAlchemy internally is not using this coding style, whether or not that leads 
to new kinds of bugs, there are new kinds of bugs no matter what kind of code a 
library uses, I don't think this hurts the user community.  The community is 
hurting *A LOT* right now because asyncio is intentionally non-compatible with 
the traditional blocking approach that is not only still prevalent it's one 
that a lot of us think is *easier* to work with.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop



mike bayer  added the comment:

> With greenlets OTOH, it becomes possible for another task to observe 
> someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally 
> invoked an await_(). Any operation can potentially invoke a context switch. 
> So debugging greenlets code is roughly as hard as debugging full-on 
> multithreaded code, and much harder than debugging async/await code.

I would invite you to look more closely at my approach.   The situation you 
describe above applies to a library like gevent, where IO means a context 
switch that can go anywhere.  My small recipe never breaks out of the asyncio 
event loop, and it only context switches within the scope of a single 
coroutine, not to any arbitrary coroutine.   So I don't think the above issue 
applies.

Additionally, we are here talking about *libraries* that are independently 
developed and tested distinct from end-user code.If there's a bug in 
SQLAlchemy, the end user isn't the person debugging that.   arguments over "is 
async or sync easier to debug" are IMO pretty subjective and at this point they 
are not relevant to what sync-based libraries should be doing.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

yes so if you have async/await all internal, are you saying you can make that 
work for synchronous code *without* running the event loop?  that is, some kind 
of container that just does the right thing?  my concern with that would still 
be performance.When asyncio was based on yield and exception throws, that 
was a lot of overhead to add to functions and that was what my performance 
testing some years back showed.   w/ async/await I'm sure things have been 
optimized, but in general when i have function a() -> b() -> c(), I am trying 
to iron as much Python overhead as I possibly can out of that and I'd be 
concerned that the machinery to work through async/await would add latency.   
additionally if it was async/await internally but then i need to access the 
majority of Python DBAPIs that are sync, I need a thread pool anyway, right?  
which is also another big barrier to jump over.

It seems you were involved with urllib3's approach to use a code rewriter 
rather than a runtime approach based on the discussion at 
https://github.com/urllib3/urllib3/issues/1323 , but it's not clear if Python 2 
compatibility was the only factor or if the concern of "writing a giant shim" 
was also.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

>  This recipe was one of the reasons why I added `loop.set_task_factory` 
> method to the spec, so that it's possible to implement this in an *existing* 
> event loop like uvloop. But ultimately asyncio is flexible enough to let 
> users use their own event loop which can be compatible with greenlets among 
> other improvements.

Right, when I sought to look at this, I know that my users want to use the 
regular event loop in asyncio or whatever system they are using.


> Ultimately, asyncio will probably never ship with greenlets integration 
> enabled by default, but we should definitely make it possible (if there are 
> some limitations now).  It doesn't seem to me that nested event loops are 
> needed for this, right?

So right, the approach I came up with does not need nested event loops and it's 
my vague understanding that nesting event loops is more difficult to debug, 
because you have these two separate loops handing off to each other.

What is most striking about my recipe is that it never even leaves the default 
event loop.  Originally my first iteration when I was trying to get it working, 
I had a separate thread going on, as it seemed intuitive that "of course you 
need a thread to bridge async and sync code" but then I erased the "Thread()" 
part around it and then it just worked anyway.   Like it's simple enough that 
shipping this as a third party library is almost not even worth it, you can 
just drop this in wherever.   If different libraries had their own drop-in of 
this, they would even work together.   greenlet is really like totally 
cheating.  

the philosophical thing here is, usually in my many twitter debates on the 
subject, folks get into how they like the explicit async and await keywords and 
they like that IO is explicit.   So I'm seeking to keep these people happy and 
give then "async def execute(sql)", and use an async DB driver, but then the 
library that these folks are using is internally not actually explicit IO.  But 
they don't have to see that, I've done all the work of the "implicit IO" stuff 
and in a library it's not the same kind of problem anyway. I think this is 
a really critical technique to have so that libraries that mediate between a 
user-facing facade and TCP based backends no longer have to make a hard choice 
about if they are to support sync vs. async (or async with an optional sync 
facade around it).

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

hey there,

I seem to have two cents to offer so here it is.An obscure issue in the 
Python bug tracker is probably not the right place for this so consider this as 
an early draft of something that maybe I'll talk about more elsewhere.

> This basically divides code into two islands - async and non-async

yes, this is the problem, and at the bottom of this apparently somewhat ranty 
comment is a solution, and the good news is that it does not require Python or 
asyncio be modified.  My concern is kind of around how it is that everyone has 
been OK with the current state of affairs for so long, why it is that "asyncio 
is fundamentally incompatible with library X" is considered to be acceptable, 
and also how easy it was to find a workaround, this is not something I would 
have expected to come up with.  Kind of like you don't expect to invent Velcro 
or windshield wipers.

asyncio's approach is what those of us in the library/framework community call 
"explicit async", you have to mark functions that will be doing IO and the 
points at which IO occurs must also be marked.Long ago it was via callback 
functions, then asyncio turned it into decorators and yields, and finally 
pep492 turned it into async/await, and it is very nicely done.  It is of course 
a feature of asyncio that writing out async/await means your code can in theory 
be clearer as to where IO occurs and all that, and while I don't totally buy 
that myself, I'm of course in favor of that style of coding being available, it 
definitely has its own kind of self-satisfaction built in when you do it.  
That's all great.

But as those of us in the library/framework community also know, asyncio's 
approach essentially means, libraries like Flask, Django, my own SQLAlchemy, 
etc. are all automatically "non-workable" with the asyncio approach; while 
these libraries can certainly have asyncio endpoints added to them, the task as 
designed is not that simple, since to go from an asyncio endpoint all the way 
through library code that doesn't care about async and then down into a 
networking library that again has asyncio endpoints, the publishing of "async" 
and the "await" or yield approach must be wired all the way through every 
function and method.  This is all despite that when you're not at the 
endpoints, the points at which IO occurs is fully predictable such that 
libraries like gevent don't need you to write it.   So we are told that 
libraries have to have full end-to-end rewrites of all their code to work this 
way, or otherwise maintain two codebases, or something like that.

The side effect of this is that a whole bunch of library and framework authors 
now get to create all new libraries and frameworks, which do exactly the same 
thing as all the existing libraries and frameworks, except they sprinkle the 
"async/await" keywords throughout middle tiers as required.  Vague claims of 
"framework X is faster because it's async" appear, impossible to confirm as it 
is unknown how much of their performance gains come from the "async" aspect and 
how much of it is that they happened to rewrite a new framework from scratch in 
a completely different way (hint: it's the latter).

Or in other cases, as if to make it obvious how much the "async/await" keywords 
come down to being more or less boilerplate for the "middle" parts of 
libraries, the urllib3 project wrote the "unasync" project [1] so that they can 
simply maintain two separate codebases, one that has "async/await" and  the 
other which just search-and-replaced them out.

SQLAlchemy has not been "replaced" by this trend as asyncio database libraries 
have not really taken off in Python, and there are very few actual async 
drivers.   Some folks have written SQLAlchemy-async libraries that use 
SQLAlchemy's expression system while they have done the tedious, redundant and 
impossible-to-maintain work of replicating enough of SQLAlchemy's execution 
internals such that a modest "sqlalchemy-like" experience with asyncio can be 
reproduced. But these libraries are closed out from all of the fixes and 
improvements that occur to SQLAlchemy itself, as well as that these systems 
likely target a smaller subset of SQLAlchemy's behaviors and features in any 
case.They certainly can't get the ORM working as the ORM runs lots of SQL 
executions internally, all of which would have to propagate their "asyncness" 
outwards throughout hundreds of functions.

The asyncpg project, one of the few asyncio database drivers that exists, notes 
in its FAQ "asyncpg uses asynchronous execution model and API, which is 
fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true  
because SQLAlchemy works just fine with gevent and eventlet, with no 
architectural changes at all.  Us

[issue41071] from an int to a float , why



mike stern  added the comment:

i am calm

but sometimes I am wondering if the philosophy behind python is really like 
they claimed to make it very simple. Well I see a lot of confusions sometimes , 
and this is one


From: report=bugs.python@roundup.psfhosted.org 
 on behalf of Tim Peters 

Sent: Monday, June 22, 2020 5:59 PM
To: rskir...@hotmail.com 
Subject: [issue41071] from an int to a float , why

Tim Peters  added the comment:

Mike, read that exchange again. You originally wrote

"print(2 / 2) gives 2.0 instead of 2"

but you didn't _mean_ that. You meant to say it "gives 1.0 instead of 1", or 
you meant something other than "2 / 2").  In Python 3,

>>> print(2 / 2)
1.0

Which is what Serhiy said it does.

For the rest, read the PEP again after you calm down. In particular,

"Classic division will remain the default in the Python 2.x series; true 
division will be standard in Python 3.0."

Also all true.

--

___
Python tracker 
<https://bugs.python.org/issue41071>
___

--

___
Python tracker 
<https://bugs.python.org/issue41071>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41071] from an int to a float , why



mike stern  added the comment:

sorry but that article was about version 2.2 in 2000

Created:11-Mar-2001
Python-Version: 2.2

where is the one for 3.7


From: report=bugs.python@roundup.psfhosted.org 
 on behalf of Tim Peters 

Sent: Monday, June 22, 2020 5:20 PM
To: rskir...@hotmail.com 
Subject: [issue41071] from an int to a float , why

Tim Peters  added the comment:

Read the PEP Serhiy already linked to:

https://www.python.org/dev/peps/pep-0238/

This was a deliberate change to how "integer / integer" works, introduced with 
Python 3.

--
nosy: +tim.peters
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41071>
___

--

___
Python tracker 
<https://bugs.python.org/issue41071>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41071] from an int to a float , why



mike stern  added the comment:

I appreciate your answer
I just didn't like how she answered earlier saying

I cannot reproduce. 2 / 2 gives 1.0 to me.

which is not true

thanks anyway, now i am gonna have to do some reading


From: report=bugs.python@roundup.psfhosted.org 
 on behalf of Tim Peters 

Sent: Monday, June 22, 2020 5:20 PM
To: rskir...@hotmail.com 
Subject: [issue41071] from an int to a float , why

Tim Peters  added the comment:

Read the PEP Serhiy already linked to:

https://www.python.org/dev/peps/pep-0238/

This was a deliberate change to how "integer / integer" works, introduced with 
Python 3.

--
nosy: +tim.peters
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41071>
___

--

___
Python tracker 
<https://bugs.python.org/issue41071>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41071] from an int to a float , why



mike stern  added the comment:

Not satisfied with that reply

you can't just decide to close the ticket without even giving a reasonable 
answer, or even try that on 2.7 or 3.7 to see if it is true what I said.
Besides, that is not a convincing answer, that is actually no answer at all!!!

I made my research and this is what is it 
https://stackoverflow.com/questions/183853/what-is-the-difference-between-and-when-used-for-division

in python 2.7 
print((20 / 3)) > 6
but
print((20.0 / 3)) or print((20 / 3.0)) or print((20.0 / 3.0)) > 
6.667
however 
print((20 / 2)) > 10 # integer number


in python 3.7
print((20 / 3)) > 6.667
not only that but even
print((20 / 2)) > 10.0  # float number
unless you use //
print((20 // 2)) > 10

can someone really explain this ?

--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue41071>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41071] from an int to a float , why