[issue29753] [Linux] ctypes packs bitfields Incorrectly
Charles Machalow added the comment: Maybe we need to add a __packing__ option to specify how packing should work and default to legacy behavior. Then allow users to specify if they want similar behavior cross-os. Otherwise changing this does change packing for existing users and can lead to breakages. What exactly was the hit regression here? On Sun, Jul 11, 2021, 10:47 AM miss-islington wrote: > > miss-islington added the > comment: > > > New changeset 42da46ed522157b057d73e6b623615ef6427999e by Miss Islington > (bot) in branch '3.10': > bpo-29753: revert 0d7ad9f (GH-19850) (GH-27085) > > https://github.com/python/cpython/commit/42da46ed522157b057d73e6b623615ef6427999e > > > -- > > ___ > Python tracker > <https://bugs.python.org/issue29753> > ___ > -- ___ Python tracker <https://bugs.python.org/issue29753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41312] add !p to pprint.pformat() in str.format() an f-strings
Charles Machalow added the comment: In terms of multiple parameters, I propose adding a method to control the defaults used by !p. Though the defaults would work more than well enough for basic log and print usage. -- ___ Python tracker <https://bugs.python.org/issue41312> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41312] add !p to pprint.pformat() in str.format() an f-strings
Charles Machalow added the comment: One of the key things for ppformat here is to format long spanning dicts/lists to multiple lines, that look easy to read in a log. I feel as though that feature/usefulness outweigh potential indentation weirdness. A lot of the usage would probably be towards the end of an fstring. -- ___ Python tracker <https://bugs.python.org/issue41312> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41312] add !p to pprint.pformat() in str.format() an f-strings
Charles Machalow added the comment: Fair enough. Didn't really know that list existed. Sent this there. Awaiting moderator approval. Thanks. -- ___ Python tracker <https://bugs.python.org/issue41312> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41312] add !p to pprint.pformat() in str.format() an f-strings
New submission from Charles Machalow :
Right now in str.format(), we have !s, !r, and !a to allow us to call str(),
repr(), and ascii() respectively on the given expression.
I'm proposing that we add a !p conversion to have pprint.pformat() be called to
convert the given expression to a 'pretty' string.
Calling
```
print(f"My dict: {d!p}")
```
is a lot more concise than:
```
import pprint
print(f"My dict: {pprint.pformat(d)}")
```
We may even be able to have a static attribute stored to change the various
default kwargs of pprint.pformat().
--
components: IO, Library (Lib)
messages: 373738
nosy: Charles Machalow
priority: normal
severity: normal
status: open
title: add !p to pprint.pformat() in str.format() an f-strings
type: enhancement
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue41312>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33498] pathlib.Path wants an rmtree method
Charles Machalow added the comment: I'm disappointed to see this closed. For new (and old) users, it makes complete sense to have an rmtree method on the Path object itself. If I'm using pathlib, I try not to delegate to os. for file operations, since it also tends to seem more OO and clean to stick with the object methods. Same thought process for shutil for rmtree. We already have things like owner() and group() that explicitly go import grp, pwd to do their job on Path objects. Why is shutil special with regards to using another piece of the standard library from pathlib? To me this request falls in the same boat as those functions; they were likely added for convenience and since it made sense. I believe the very same holds true for the request to add rmtree(). -- nosy: +Charles Machalow ___ Python tracker <https://bugs.python.org/issue33498> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40972] Add a recurse flag to Path.rmdir()
New submission from Charles Machalow : I think it would make sense to add a recurse flag to the Path.rmdir() method. It would default to False (to allow for current behavior). If set to True, the method would act very similarly to shutil.rmtree() in that it would delete all files in the directory and then delete the directory itself. I understand that this behavior doesn't really line up with os.rmdir(), though I think it makes sense to allow this type of method to appear on the Path object for easy deletion of a directory with things still inside (while looking more object-oriented at the same time) If people think this makes sense, I may be able to provide a PR to just delegate to shutil.rmtree for the recurse=True operation. Thanks for thoughts. -- components: Library (Lib) messages: 371487 nosy: Charles Machalow priority: normal severity: normal status: open title: Add a recurse flag to Path.rmdir() type: enhancement versions: Python 3.10, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40972> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39920] Pathlib path methods do not work with Window Dos devices
New submission from Charles Machalow : I ran the following as admin in the Python interpreter (on Windows): >>> d = pathlib.Path(r'\\.\PHYSICALDRIVE0') >>> print(d) \\.\PHYSICALDRIVE0\ >>> d.exists() Traceback (most recent call last): File "", line 1, in File "C:\Python37\lib\pathlib.py", line 1318, in exists self.stat() File "C:\Python37\lib\pathlib.py", line 1140, in stat return self._accessor.stat(self) PermissionError: [WinError 31] A device attached to the system is not functioning: '.\\PHYSICALDRIVE0\\' >>> d.is_char_device() Traceback (most recent call last): File "", line 1, in File "C:\Python37\lib\pathlib.py", line 1403, in is_char_device return S_ISCHR(self.stat().st_mode) File "C:\Python37\lib\pathlib.py", line 1140, in stat return self._accessor.stat(self) PermissionError: [WinError 31] A device attached to the system is not functioning: '.\\PHYSICALDRIVE0\\' >>> d.is_block_device() Traceback (most recent call last): File "", line 1, in File "C:\Python37\lib\pathlib.py", line 1390, in is_block_device return S_ISBLK(self.stat().st_mode) File "C:\Python37\lib\pathlib.py", line 1140, in stat return self._accessor.stat(self) PermissionError: [WinError 31] A device attached to the system is not functioning: '.\\PHYSICALDRIVE0\\' I think that exists(), is_char_device(), and is_block_device() should be able to work on Windows in some form or fashion. At least without a traceback. -- messages: 363796 nosy: Charles Machalow priority: normal severity: normal status: open title: Pathlib path methods do not work with Window Dos devices type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue39920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29753] Ctypes Packing Bitfields Incorrectly - Linux
Charles Machalow <csm10...@gmail.com> added the comment: All of Python is implicitly cross platform. If something isn't actually cross platform, it should be mentioned explicitly in the documentation. For example see the mmap documentation, it explicitly say on Unix it does X, on Windows it does Y. We should be as explicit as possible for ctypes to say on Windows we expect packing like X (GCC) and on Linux we expect packing like Y (VC++). -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue29753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29753] Ctypes Packing Bitfields Incorrectly - Linux
Charles Machalow <csm10...@gmail.com> added the comment: Antti, is there a place in the ctypes documentation that explicitly says ctypes is not meant to be used cross-platform? If not, shouldn't that be mentioned? I think ultimately ctypes should default to standard OS/compiler behavior, but should allow the flexibility for one to override for cross-platform interchangeability. In the C++ code here, the code is explicitly checking if Windows (or GCC) to choose behavior. In theory, that could be changed to allow runtime-choice for packing behavior. Of course, that is probably best for a feature issue. For this one, I'd just like to have the behavior on Linux match GCC, as that is the definitive bug here as our example has shown. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue29753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12528] Implement configurable bitfield allocation strategy
Charles Machalow added the comment: Was browsing and found this. This option would be very useful as it could help avoid a current bug I've had to deal with : https://bugs.python.org/issue29753. My use case works with data/structs from another device all-together, so I can't control it's packing. However since GCC/Linux builds currently have the specified bug, I can't get around it. This option would have allowed me to just specify the MSVC strategy, which in general is what I want to not have to have OS-specific behavior. With the bug in mind, this could have almost be a bug-workaround for me. -- nosy: +Charles Machalow ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29753] Ctypes Packing Incorrectly - Linux
Charles Machalow added the comment:
Some more debug with print statements in the c code seems to confirm my
suspicion:
bitsize, pfield_size, bitofs, dict->size prints were added just above the if
chain to determine fieldtype and fieldtype was just after that if chain. That
code looks something like this:
printf("bitsize: %d\n" , (int)bitsize);
printf("pfield_size: %u\n", (size_t)*pfield_size);
printf("bitofs: %d\n", (int)*pbitofs);
printf("dict->size: %d\n", (int)dict->size);
if (bitsize /* this is a bitfield request */
&& *pfield_size /* we have a bitfield open */
#ifdef MS_WIN32
/* MSVC, GCC with -mms-bitfields */
&& dict->size * 8 == *pfield_size
#else
/* GCC */
&& dict->size * 8 <= *pfield_size
#endif
&& (*pbitofs + bitsize) <= *pfield_size) {
/* continue bit field */
fieldtype = CONT_BITFIELD;
#ifndef MS_WIN32
} else if (bitsize /* this is a bitfield request */
&& *pfield_size /* we have a bitfield open */
&& dict->size * 8 >= *pfield_size
&& (*pbitofs + bitsize) <= dict->size * 8) {
/* expand bit field */
fieldtype = EXPAND_BITFIELD;
#endif
} else if (bitsize) {
/* start new bitfield */
fieldtype = NEW_BITFIELD;
*pbitofs = 0;
*pfield_size = dict->size * 8;
} else {
/* not a bit field */
fieldtype = NO_BITFIELD;
*pbitofs = 0;
*pfield_size = 0;
}
printf("Fieldtype: %d\n--\n", fieldtype);
And the run with the custom-built Python 2.7.13:
>>> from test import *
bitsize: 0
pfield_size: 0
bitofs: 255918304
dict->size: 2
Fieldtype: 0
--
bitsize: 9
pfield_size: 0
bitofs: 0
dict->size: 2
Fieldtype: 1
--
bitsize: 1
pfield_size: 16
bitofs: 9
dict->size: 2
Fieldtype: 2
--
bitsize: 1
pfield_size: 16
bitofs: 10
dict->size: 2
Fieldtype: 2
--
bitsize: 1
pfield_size: 16
bitofs: 11
dict->size: 2
Fieldtype: 2
--
bitsize: 1
pfield_size: 16
bitofs: 12
dict->size: 2
Fieldtype: 2
--
bitsize: 3
pfield_size: 16
bitofs: 13
dict->size: 2
Fieldtype: 2
--
bitsize: 10
pfield_size: 16
bitofs: 16
dict->size: 4
Fieldtype: 3
--
bitsize: 20
pfield_size: 32
bitofs: 26
dict->size: 4
Fieldtype: 1
--
bitsize: 2
pfield_size: 32
bitofs: 20
dict->size: 4
Fieldtype: 2
--
10
>>> MyStructure.P
>>> MyStructure.T
>>> MyStructure.R
>>> MyStructure.P
>>> MyStructure.L
>>> MyStructure.Pro
>>> MyStructure.R
>>> MyStructure.T
>>> MyStructure.C
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29753>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29753] Ctypes Packing Incorrectly - Linux
Charles Machalow added the comment:
Took a quick look at the c code for this. The area at fault appears to be this
section in cfield.c:
#ifndef MS_WIN32
} else if (bitsize /* this is a bitfield request */
&& *pfield_size /* we have a bitfield open */
&& dict->size * 8 >= *pfield_size
&& (*pbitofs + bitsize) <= dict->size * 8) {
/* expand bit field */
fieldtype = EXPAND_BITFIELD;
#endif
The problem seems to be after the end of the 2nd c_uint16, it seems to think
that the next 10 bytes should extend that c_uint16 to a c_uint32 instead of
taking the type as the beginning of a new c_uint32. So I guess it is making the
structure something like this:
("P", c_uint16),
("L", c_uint32, 9),
("Pro", c_uint32, 1),
("G", c_uint32, 1),
("IB", c_uint32, 1),
("IR", c_uint32, 1),
("R", c_uint32, 3),
("T", c_uint32, 10),
# And now this needs an extra 6 bits of padding to fill
the c_uint32
("C", c_uint32, 20),
("R2", c_uint32, 2)
# And now this needs an extra 10 bits of padding to fill
the c_uint32.
I guess that is how we get to 10... instead of the expected 8 bytes. I don't
believe that this behavior is desired nor really makes logical sense.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29753>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29753] Ctypes Packing Incorrectly - Linux
New submission from Charles Machalow:
There appears to be a bug related to sizing/packing of ctypes Structures on
Linux. I'm not quite sure how, but this structure:
class MyStructure(Structure):
_pack_ = 1
_fields_= [
("P", c_uint16),# 2 Bytes
("L", c_uint16, 9),
("Pro", c_uint16, 1),
("G", c_uint16, 1),
("IB", c_uint16, 1),
("IR", c_uint16, 1),
("R", c_uint16, 3), # 4 Bytes
("T", c_uint32, 10),
("C", c_uint32, 20),
("R2", c_uint32, 2) # 8 Bytes
]
Gives back a sizeof of 8 on Windows and 10 on Linux. The inconsistency makes it
difficult to have code work cross-platform.
Running the given test.py file will print out the size of the structure on your
platform.
Tested with Python 2.7.6 and Python 3.4.3 (builtin to Ubuntu 14.04), Python
2.7.13, (built from source) both on Ubuntu 14.04. On Linux all Python builds
were 32 bit.
On Windows I tried with 2.7.7 (both 32 and 64 bit).
I believe on both platforms it should return a sizeof 8.
--
components: ctypes
files: test.py
messages: 289193
nosy: Charles Machalow, amaury.forgeotdarc, belopolsky, meador.inge
priority: normal
severity: normal
status: open
title: Ctypes Packing Incorrectly - Linux
type: behavior
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file46708/test.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29753>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25858] Structure field size/ofs __str__ wrong with large size fields
Changes by Charles Machalow <csm10...@gmail.com>: Removed file: http://bugs.python.org/file41298/ctypesBug.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25858> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25858] Structure field size/ofs __str__ wrong with large size fields
Changes by Charles Machalow <csm10...@gmail.com>: Added file: http://bugs.python.org/file41299/ctypesBug.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25858> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25858] Structure field size/ofs __str__ wrong with large size fields
New submission from Charles Machalow:
Large sized fields in Structures lead to incorrect string representations
because it is assuming that because the size is so large, it must be a bit
field instead of a byte field.
class bugStruct(Structure):
_pack_ = 1
_fields_ = [
("correct", c_uint8 * 65535),
("wrongSizeAndOffset", c_uint8 * 99),
]
print(str(bugStruct.correct))
# Correct
print(str(bugStruct.wrongSizeAndOffset))
# Incorrect
# Should be:
To get the math for this do the following on an incorrect size/offset:
size = (bits << 16) + ofs.split(":")[1]
ofs = ofs.split(":")[0]
Though this isn't really safe because the field may actually be bit-sized
--
components: ctypes
files: ctypesBug.py
messages: 256359
nosy: Charles Machalow
priority: normal
severity: normal
status: open
title: Structure field size/ofs __str__ wrong with large size fields
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file41297/ctypesBug.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25858>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25858] Structure field size/ofs __str__ wrong with large size fields
Charles Machalow added the comment: Adding file with code to reproduce. -- Added file: http://bugs.python.org/file41298/ctypesBug.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25858> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25858] Structure field size/ofs __str__ wrong with large size fields
Changes by Charles Machalow <csm10...@gmail.com>: Removed file: http://bugs.python.org/file41297/ctypesBug.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25858> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
