[issue47143] Add types.copy_class() which updates closures
Lumír Balhar added the comment: Do you think it's a good idea to start a PR with a copy of the implementation from attrs for Python 3.11? We can then add tests for the new function and also some for dataclasses where this new function is needed and try to find all corner cases. -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46404] 3.11a4: a small attrs regression
Lumír Balhar added the comment: We have the same problem reported in attrs here in dataclasses and because it's not tested the way to manipulate __closure__ cells changes frequently. My plan is to implement something similar to this into dataclasses: https://github.com/python-attrs/attrs/blob/5c040f30e3e4b3c9c0f27c8ac6ff13d604c1818c/src/attr/_make.py#L895-L916 Basically, when a new dataclass is created (with slots=True), look for references to the original class and fix them. This fixes the problem reported to attrs in dataclasses and when we fix it and add some tests for it, the future behavior should be more stable. What do you think? -- ___ Python tracker <https://bugs.python.org/issue46404> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46404] 3.11a4: a small attrs regression
Lumír Balhar added the comment: In my opinion, we should keep it simple for attrs and dataclasses to fix closure cells when a class is replaced and therefore it seems to be correct to have it mutable as it currently is in 3.11. My plan is to implement the fix for dataclasses and some tests for these use cases so the behavior should not change in the future. For attrs it means one more branch in the code now but much simpler code when the support for older releases gets dropped. Any other opinions? -- ___ Python tracker <https://bugs.python.org/issue46404> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46404] 3.11a4: a small attrs regression
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue46404> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45460] distutils.sysconfig.get_python_lib() does not respect sysconfig/distutils install schemes
Change by Lumír Balhar : ___ Python tracker <https://bugs.python.org/issue45460> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45413] Add install scheme for virtual environments
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue45413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44351] distutils.sysconfig.parse_makefile() regression in Python 3.10
Change by Lumír Balhar : -- keywords: +patch pull_requests: +25223 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26637 ___ Python tracker <https://bugs.python.org/issue44351> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44351] distutils.sysconfig.parse_makefile() regression in Python 3.10
Lumír Balhar added the comment: Thanks for the report. The regression is caused by the fact that the old implementation of parse_makefile in distutils.sysconfig was using feature-rich class TextFile which handles all the functionalities you reported as broken - stripping of comments, joining lines, and more. The new implementation of parse_makefile in sysconfig just reads all the lines without any processing. The TextFile class is deprecated together with the whole distutils. The class is not used outside of distutils modules, only in these functions: distutils.extension:read_setup_file distutils.sdist:read_template + its tests in distutils/tests/test_text_file.py If the functionality is something we want to preserve, we should find a new home for the class and then we can use it in the sysconfig module. -- ___ Python tracker <https://bugs.python.org/issue44351> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44055] NamedTemporaryFile opened twice on Windows
Lumír Balhar added the comment: I understand, thanks for the explanation. I'd like to improve the documentation to make it more clear that the file actually can be reopened on Windows when you use delete=False but I cannot find the right words :/ What about: That name can be retrieved from the name attribute of the returned file-like object. If delete is true (the default), the file is deleted as soon as it is closed. Whether the name can be used to open the file a second time, while the named temporary file is still open with delete set to true, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later). -- ___ Python tracker <https://bugs.python.org/issue44055> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44055] NamedTemporaryFile opened twice on Windows
New submission from Lumír Balhar :
Hello.
The documentation about tempfile.NamedTemporaryFile[0] contains:
That name can be retrieved from the name attribute of the returned file-like
object. Whether the name can be used to open the file a second time, while the
named temporary file is still open, varies across platforms (it can be so used
on Unix; it cannot on Windows NT or later).
But after some testing on Windows, it seems that there is no problem in opening
a temporary file for the second time. A simple reproducer:
from tempfile import NamedTemporaryFile
# Open the file for the first time
tmp_file = NamedTemporaryFile(mode="w+", delete=False)
tmp_file.write("foo")
tmp_file.seek(0)
content_original = tmp_file.read()
print("Original content:", content_original)
# Open the file for the second time
with open(tmp_file.name, mode="w+") as file:
file.write("bar")
file.seek(0)
content = file.read()
print("Updated content:", content)
The output is:
Original content: foo
Updated content: bar
So the question is: do I misunderstand the documentation? Or is there any
automatic magic handling this on Windows?
I'm not a windows user and I've found this accidentally when preparing a
Windows CI job for a project.
[0]
https://docs.python.org/3/library/tempfile.html?highlight=namedtemporaryfile#tempfile.NamedTemporaryFile
--
assignee: docs@python
components: Documentation, Windows
messages: 393082
nosy: docs@python, frenzy, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: NamedTemporaryFile opened twice on Windows
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue44055>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43976] Introduce mechanism to allow Python distributors to add custom site install schemes
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue43976> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43752] [sqlite3] Fetching an empty value from date column raises ValueError
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue43752> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43831] sqlite: convert_timestamp raises ValueError for empty columns
New submission from Lumír Balhar :
Hello.
I've discovered this issue when I was debugging a test failure in IPython. See
this issue for more details (not needed):
https://github.com/ipython/ipython/issues/12906
I'm testing this on Fedora Linux with Python 3.10.0a7 from our RPM package with
SQLite 3.35.4.
I have a very simple SQLite database:
# sqlite3 test.sqlite
SQLite version 3.35.4 2021-04-02 15:20:15
Enter ".help" for usage hints.
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE sessions (session integer
primary key autoincrement, start timestamp,
end timestamp, num_cmds integer, remark text);
INSERT INTO sessions VALUES(1,'2021-04-13 09:44:58.903345',NULL,NULL,'');
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('sessions',1);
COMMIT;
When I query it without special converters, it works well:
# python3
>>> import sqlite3
>>> conn = sqlite3.connect('test.sqlite')
>>> c = conn.cursor()
>>> c.execute("SELECT * from sessions where session == 1").fetchone()
(1, '2021-04-13 09:44:58.903345', None, None, '')
>>>
but with detect_types=sqlite3.PARSE_DECLTYPES, it fails to parse the date and
time:
# python3
>>> import sqlite3
>>> conn = sqlite3.connect('test.sqlite', detect_types=sqlite3.PARSE_DECLTYPES)
>>> c = conn.cursor()
>>> c.execute("SELECT * from sessions where session == 1").fetchone()
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.10/sqlite3/dbapi2.py", line 67, in convert_timestamp
datepart, timepart = val.split(b" ")
ValueError: not enough values to unpack (expected 2, got 1)
>>>
With help of pdb in ipython, I've discovered that the value convert_timestamp
gets to parse is an empty bytestring:
In [5]: c.execute("SELECT * from sessions where session == 1").fetchone()
---
ValueErrorTraceback (most recent call last)
in
> 1 c.execute("SELECT * from sessions where session == 1").fetchone()
/usr/lib64/python3.10/sqlite3/dbapi2.py in convert_timestamp(val)
65
66 def convert_timestamp(val):
---> 67 datepart, timepart = val.split(b" ")
68 year, month, day = map(int, datepart.split(b"-"))
69 timepart_full = timepart.split(b".")
ValueError: not enough values to unpack (expected 2, got 1)
> /usr/lib64/python3.10/sqlite3/dbapi2.py(67)convert_timestamp()
65
66 def convert_timestamp(val):
---> 67 datepart, timepart = val.split(b" ")
68 year, month, day = map(int, datepart.split(b"-"))
69 timepart_full = timepart.split(b".")
ipdb> val
b''
ipdb>
Is anything in my database wrong? It seems that the content of the start column
is correct and covert_timestamp should be able to parse it. Is it possible that
the troublemaker here is the empty column `end` of type timestamp?
Answer to my own question: yes, the issue here is that the column `end` is of
type timestamp and it's empty. If I update it with a date and time, everything
works:
# sqlite3 test.sqlite
sqlite> update sessions set end='2021-04-14 09:44:58.903345' where session = 1;
sqlite> select * from sessions;
1|2021-04-13 09:44:58.903345|2021-04-14 09:44:58.903345||
sqlite>
# python3
>>> import sqlite3
>>> conn = sqlite3.connect('test.sqlite', detect_types=sqlite3.PARSE_DECLTYPES)
>>> c = conn.cursor()
>>> c.execute("SELECT * from sessions where session == 1").fetchone()
(1, datetime.datetime(2021, 4, 13, 9, 44, 58, 903345), datetime.datetime(2021,
4, 14, 9, 44, 58, 903345), None, '')
So, the final question is whether this is correct behavior. I believe that
columns without content should not be passed to converters.
--
components: Library (Lib)
messages: 390953
nosy: frenzy
priority: normal
severity: normal
status: open
title: sqlite: convert_timestamp raises ValueError for empty columns
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue43831>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42988] [security] CVE-2021-3426: Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue42988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41282] Deprecate and remove distutils
Change by Lumír Balhar : -- pull_requests: +23332 pull_request: https://github.com/python/cpython/pull/24549 ___ Python tracker <https://bugs.python.org/issue41282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43228] Regression in function builtins
Lumír Balhar added the comment:
I'm not an expert nor an author but this might help:
Cloudpickle offers extended possibilities for pickling but uses the standard
pickle module for unpickling:
>>> import pickle, cloudpickle
>>> cloudpickle.load is pickle.load
True
>>> cloudpickle.loads is pickle.loads
True
So, the question here is why the new Python version cannot handle cloudpickle
output anymore.
>>> def f():
... return len("")
...
>>> pickle.dumps(f)
b'\x80\x04\x95\x12\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x01f\x94\x93\x94.'
>>> cloudpickle.dumps(f)
b'\x80\x05\x95\xa9\x01\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\r_builtin_type\x94\x93\x94\x8c\nLambdaType\x94\x85\x94R\x94(h\x02\x8c\x08CodeType\x94\x85\x94R\x94(K\x00K\x00K\x00K\x00K\x02KCC\x08t\x00d\x01\x83\x01S\x00\x94N\x8c\x04\x94\x86\x94\x8c\x03len\x94\x85\x94)\x8c\x07\x94\x8c\x01f\x94K\x01C\x02\x00\x01\x94))t\x94R\x94}\x94(\x8c\x0b__package__\x94N\x8c\x08__name__\x94\x8c\x08__main__\x94uNNNt\x94R\x94\x8c\x1ccloudpickle.cloudpickle_fast\x94\x8c\x12_function_setstate\x94\x93\x94h\x18}\x94}\x94(h\x15h\x0f\x8c\x0c__qualname__\x94h\x0f\x8c\x0f__annotations__\x94}\x94\x8c\x0e__kwdefaults__\x94N\x8c\x0c__defaults__\x94N\x8c\n__module__\x94h\x16\x8c\x07__doc__\x94N\x8c\x0b__closure__\x94N\x8c\x17_cloudpickle_submodules\x94]\x94\x8c\x0b__globals__\x94}\x94u\x86\x94\x86R0.'
It seems to me that cloudpickle adds also __globals__ to the final output and
pickle is no longer able to restore it.
--
___
Python tracker
<https://bugs.python.org/issue43228>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43228] Regression in function builtins
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue43228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41282] Deprecate and remove distutils
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue41282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24908] sysconfig.py and distutils.sysconfig.py disagree on directory spelling on Windows
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue24908> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39825] EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows)
Change by Lumír Balhar : -- nosy: +frenzy ___ Python tracker <https://bugs.python.org/issue39825> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38112] Compileall improvements
Change by Lumír Balhar : -- nosy: +frenzy nosy_count: 4.0 -> 5.0 pull_requests: +19475 pull_request: https://github.com/python/cpython/pull/20174 ___ Python tracker <https://bugs.python.org/issue38112> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40495] compileall: option to hardlink duplicate optimization levels bytecode cache files
Change by Lumír Balhar : -- keywords: +patch pull_requests: +19214 stage: -> patch review pull_request: https://github.com/python/cpython/pull/19901 ___ Python tracker <https://bugs.python.org/issue40495> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40495] compileall: option to hardlink duplicate optimization levels bytecode cache files
Lumír Balhar added the comment: I forgot to mention that I am working on PR which should be ready soon because the implementation is already done and tested in compileall2. -- ___ Python tracker <https://bugs.python.org/issue40495> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40495] compileall: option to hardlink duplicate optimization levels bytecode cache files
New submission from Lumír Balhar : We would like to include a possibility of hardlink deduplication of identical pyc files to compileall module in Python 3.9. We've discussed the change [0] and tested it in Fedora RPM build system via implementation in the compileall2 module [1]. The discussion [0] contains a lot of details so I mention here only the key features: * the deduplication can be enabled only if multiple optimization levels are processed at once * it generates a pyc file (optimization level 0) as usual but if it finds that optimized files (optimization levels 1 and 2) have the same content, it uses hardlinks (os.link) to prevents duplicates * the deduplication is disabled by default We believe that this might be handy for more Pythonistas. In our case, this functionality lowers the installation size of Python 3.9 from 125 MiB to 103 MiB. [0] https://discuss.python.org/t/compileall-option-to-hardlink-duplicate-optimization-levels-bytecode-cache-files/3014 [1] https://github.com/fedora-python/compileall2 -- components: Library (Lib) messages: 368022 nosy: frenzy priority: normal severity: normal status: open title: compileall: option to hardlink duplicate optimization levels bytecode cache files type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40495> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38112] Compileall improvements
Change by Lumír Balhar : -- keywords: +patch pull_requests: +15638 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16012 ___ Python tracker <https://bugs.python.org/issue38112> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21258] Add __iter__ support for mock_open
Change by Lumír Balhar <frenzy.madn...@gmail.com>: -- nosy: +frenzy ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue21258> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29144] Implicit namespace packages in Python 3.6
Lumír Balhar added the comment: Thank you for the replies. I created an issue [0] on setuptools GitHub (without reply yet) because I also think that setuptools should install pkg_resources-style __init__.py files. [0] https://github.com/pypa/setuptools/issues/1097 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29144> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29144] Implicit namespace packages in Python 3.6
Lumír Balhar added the comment: Hello. I've tried the build of python-moksha-common on Fedora rawhide [0] and Fedora 25 [1]. Fedora rawhide: Python 2.7.13 and 3.6.1, setuptools 36.2.0 Fedora 25: Python 2.7.13 and 3.5.3, setuptools 25.1.1 Source tarball of moksha.common contains pkg_resources-style __init__.py file and the setuptools skipped installation of this file in all (four) cases. It seems that setuptools do the same in all versions but the problem comes with Python 3.6 which is more strict than Python 3.5 in behavior related to namespace packages. So, when one part of namespace package doesn't contain __init__.py file and other part contains pkg_resources-style __init__.py file: - in Python 2.7 and 3.5 everything works - in Python 3.6 submodules cannot import each other The question is whether it is a bug/feature in Python 3.6 and setuptools should install pkg_resources-style __init__.py or not. What do you think? [0] https://koji.fedoraproject.org/koji/taskinfo?taskID=20606746 [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=20606838 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29144> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29144] Implicit namespace packages in Python 3.6
Lumír Balhar added the comment: Hello. I've met this issue again in moksha project [0] where namespace package is used. - module moksha.common is installed via RPM to site-packages and there isn't file moksha/__init__.py so the implicit way to create namespace package is used there. - moksha.hub module is built in a buildroot and there is moksha/__init__.py file so this module uses different way which causes that import of moksha.common is failing. We can delete moksha/__init__.py from buildroot of moksha.hub but then we cannot run tests with `python setup.py test`. Error message is: error in moksha.hub setup command: Distribution contains no modules or packages for namespace package 'moksha' The second possibility is ship moksha/__init__.py file in python-moksha-common RPM. The question is: Why there isn't moksha/__init__.py file in RPM even when the source tarball contains it? Because of setuptools! Setuptools skips installation of __init__.py file when namespace package is recognized with the message: Skipping installation of /builddir/build/BUILDROOT/python-moksha-common-1.2.5-1.fc27.x86_64/usr/lib/python3.6/site-packages/moksha/__init__.py (namespace package) Questions summary: - Why there is such big difference between Python 3.5 and Python 3.6? - Why does setuptools skip installation of __init__.py file but it needs them to run tests of namespace subpackage? Thank you! Lumír [0] https://github.com/mokshaproject/moksha -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29144> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29144] Implicit namespace packages in Python 3.6
New submission from Lumír Balhar: Hello. I've found a really strange difference between Python 3.5 and 3.6 related to namespace packages and I cannot find any note in changelogs. I've created a simple bash script which can reproduce my issue using virtual environments: http://pastebin.com/j3fXMtR4 This script creates a virtual environment, installs marrow.util from PyPI which is one part of namespace package and creates the other part (marrow.mailer) in the local folder. Then script tries to import marrow.util and marrow.mailer - and there is the difference. In Python 3.5 - I can import marrow.util (installed via pip) but I cannot import local marrow.mailer module. I think that this is right behavior. In Python 3.6 - I cannot import marrow.util (installed via pip) but I can import local marrow.mailer module. Python versions are 3.5.2 (installed from Fedora repositories) and 3.6.0 (compiled manually). Why there is such a big difference? Am I missing something? I tried to ask on IRC, stackoverflow etc. but without success. Could you please explain the reason for this behavior to me? Thank you and have a nice day. Lumír Bash script attached -- files: test_namespace_package.sh messages: 284587 nosy: frenzy priority: normal severity: normal status: open title: Implicit namespace packages in Python 3.6 type: behavior versions: Python 3.6 Added file: http://bugs.python.org/file46129/test_namespace_package.sh ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29144> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
