Type annotation pitfall

[Robert Latest via Python-list]

Hi all,

this just caused me several hours of my life until I could whittle it down to
this minimal example. Simple question: Why is the x member of object "foo"
modified by initializing "bar"?

Obviously, initializing foo with None doesn't set foo.x at all. So I guess x
stays a class property, not an instance property. And instantiating bar just
added an item to the class property but didn't instantiate a new set. So
basically, neither foo nor bar ever had their "own" x, right?

Oooohh, dangerous! Never use mutable types in type hint, unless it's in
explicit dataclasses (which automatically add an appropriate __init__()?)

Now I must fine-comb all my code for more of these.

class Foo():
x : set = set()

def __init__(self, s):
if s:
self.x.add(s)

foo = Foo(None)
print(foo.x) # prints 'set()'
bar = Foo('abc')
print(foo.x) # prints '{'abc'}

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[dn via Python-list]

On 24/09/2021 14.07, Stefan Ram wrote:
> dn  writes:
>> With that, why not code it as Python expressions, and include the module?
> 
>   This might create a code execution vulnerability if such 
>   files are exchanged between multiple parties.


The OP's spec, as quoted earlier(!), reads:

"It's my own research, so I can give myself the data in any format that
I like."

Whither "files are exchanged" and/or "multiple parties"? Are these
anticipations of problems that may/won't ever apply? aka YAGNI.

Concern about such an approach *is* warranted.

However, the preceding question to be considered during the design-stage
is: 'does such concern apply?'. The OP describes full and unique agency.
Accordingly, "KISS"!

NB my personal choice would likely be JSON or YAML, but see reservations
(eg @Chris) - and with greater relevance: shouldn't we consider the OP's
'learning curve'?
(such deduced only from OP's subsequent reactions/responses 'here' -
with any and all due apologies)
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Flush / update GUIs in PyQt5 during debugging in PyCharm

[Mohsen Owzar]

Hi Guys
I've written a GUI using PyQt5 and in there I use StyleSheets (css) for the 
buttons and labels to change their background- and foreground-colors and their 
states as well.
Because my program doesn't function correctly, I try to debug it in my IDE 
(PyCharm).
The problem is that during debugging, when I change some attributes of a button 
or label, let say its background-color, I can not see this modification of the 
color until the whole method or function is completed.
I believe that I have seen somewhere during my searches and googling that one 
can flush or update the GUI after each step/action is done.
But until now I couldn't manage it and I don't know where I have to invoke 
flush/update command in PyCharm.
If anyone has done this before and knows about it, I would very appreciate 
seeing his solution.

Regards
Mohsen
-- 
https://mail.python.org/mailman/listinfo/python-list

When should I use "parent=None" in __ini__ and "parent" in super()

[Mohsen Owzar]

Hi Guys
I'm writing since almost one-year codes in Python, using TKinter and PyQt5.
I'm somehow able to writes GUIs in both of them.
But since I'm using more Pyqt5 and using classes with initialization and 
super() constructs, and also I saw lots of videos and examples of coding them, 
I still don’t know exactly how and when should I use the parent in __init()__ 
and super() construct.
For example, the following lines use "parent=None" in the __init__ construct 
and "parent" in the super() construct. And sometimes I do not see any of them 
in these two constructs.

class MyClass1(QWidget):
def __init__(self, name, parent=None):
super(MyClass1, self).__init__(parent)
print(self.parent().test)

And sometimes without "parent" like the following:

class MyClass2(QWidget):
def __init__(self, name):
super(MyClass2, self).__init__()
print(self.parent().test)

Could anyone explain this to me when and for which action I have to use the 
first one "MyClass1" and when the second one "MyClass2"?

Regards
Mohsen
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45275] Make argparse print description of subcommand when invoke help doc on subcommand

[Chuanlong Du]

New submission from Chuanlong Du :

I have command-line script `blog` written using argparse. It supports 
subcommands. When I check the help doc of a subcommand, e.g., using `blog 
convert -h`, it prints the help doc of the subcommand `blog convert` but 
doesn't print the description of the subcommand `blog convert`. A screenshot is 
attached. It is quite often that I know a command-line application have certain 
subcommands but I forget exactly what they do. It would be very helpful if 
`blog subcmd -h` prints the description of the subcmd so that users do not have 
to call `blog -h` again to check exactly what the subcommand does.

--
components: Library (Lib)
files: Selection_011.png
messages: 402541
nosy: longendu
priority: normal
severity: normal
status: open
title: Make argparse print description of subcommand when invoke help doc on 
subcommand
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9
Added file: https://bugs.python.org/file50301/Selection_011.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: XML Considered Harmful

[Chris Angelico]

On Fri, Sep 24, 2021 at 1:44 PM Dan Stromberg  wrote:
>
>
> On Thu, Sep 23, 2021 at 8:12 PM Chris Angelico  wrote:
>>
>> One good hybrid is to take a subset of Python syntax (so it still
>> looks like a Python script for syntax highlighting etc), and then
>> parse that yourself, using the ast module. For instance, you can strip
>> out comments, then look for "VARNAME = ...", and parse the value using
>> ast.literal_eval(), which will give you a fairly flexible file format
>> that's still quite safe.
>
>
> Restricting Python with the ast module is interesting, but I don't think I'd 
> want to bet my career on the actual safety of such a thing.  Given that Java 
> bytecode was a frequent problem inside web browsers, imagine all the 
> messiness that could accidentally happen with a subset of Python syntax from 
> untrusted sources.
>
> ast.literal_eval might be a little better - or a list of such, actually.

Uhh, I specifically mention literal_eval in there :) Simple text
parsing followed by literal_eval for the bulk of it is a level of
safety that I *would* bet my career on.

> Better still to use JSON or ini format - IOW something designed for the 
> purpose.

It all depends on how human-editable it needs to be. JSON has several
problems in that respect, including some rigidities, and a lack of
support for comments. INI format doesn't have enough data types for
many purposes. YAML might be closer, but it's not for every situation
either.

That's why we have options.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Dan Stromberg]

On Thu, Sep 23, 2021 at 8:12 PM Chris Angelico  wrote:

> One good hybrid is to take a subset of Python syntax (so it still
> looks like a Python script for syntax highlighting etc), and then
> parse that yourself, using the ast module. For instance, you can strip
> out comments, then look for "VARNAME = ...", and parse the value using
> ast.literal_eval(), which will give you a fairly flexible file format
> that's still quite safe.
>

Restricting Python with the ast module is interesting, but I don't think
I'd want to bet my career on the actual safety of such a thing.  Given that
Java bytecode was a frequent problem inside web browsers, imagine all the
messiness that could accidentally happen with a subset of Python syntax
from untrusted sources.

ast.literal_eval might be a little better - or a list of such, actually.

Better still to use JSON or ini format - IOW something designed for the
purpose.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Chris Angelico]

On Fri, Sep 24, 2021 at 12:22 PM Stefan Ram  wrote:
>
> dn  writes:
> >With that, why not code it as Python expressions, and include the module?
>
>   This might create a code execution vulnerability if such
>   files are exchanged between multiple parties.
>
>   If code execution vulnerabilities and human-readability are
>   not an issue, then one could also think about using pickle.
>
>   If one ignores security concerns for a moment, serialization into
>   a text format and subsequent deserialization can be a easy as:
>
> |>>> eval( str( [1, (2, 3)] ))
> |[1, (2, 3)]
>

One good hybrid is to take a subset of Python syntax (so it still
looks like a Python script for syntax highlighting etc), and then
parse that yourself, using the ast module. For instance, you can strip
out comments, then look for "VARNAME = ...", and parse the value using
ast.literal_eval(), which will give you a fairly flexible file format
that's still quite safe.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[dn via Python-list]

On 22/09/2021 07.22, Michael F. Stemper wrote:
> On 21/09/2021 13.49, alister wrote:
>> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:
>>
>>> On the prolog thread, somebody posted a link to:
>>> 

Given the source, shouldn't one take any criticism of Python (or Java)
with at least the proverbial grain of salt!


>>> One thing that it tangentially says is "XML is not the answer."

"tangential" as in 'spinning off'?


...

> It's my own research, so I can give myself the data in any format that I
> like.
...
With that, why not code it as Python expressions, and include the module?
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +26923
pull_request: https://github.com/python/cpython/pull/28538

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: XML Considered Harmful

[Jon Ribbens via Python-list]

On 2021-09-23, Stefan Ram  wrote:
>   The real problem with CSV is that there is no CSV.
>
>   This is not a specific data language with a specific
>   specification. Instead it is a vague designation for
>   a plethora of CSV dialects, which usually dot not even
>   have a specification.

Indeed. For example, at least at some points in its history,
Excel has been unable to import CSV written by itself, because
its importer was incompatible with its own exporter.

>   Compare this with XML. XML has a sole specification managed
>   by the W3C.

Other well-defined formats are also available ;-)
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[Eryk Sun]

Eryk Sun  added the comment:

See bpo-21822 from 2014, which I think was the first report of this issue.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message

[Daniel Hillier]

Daniel Hillier  added the comment:

Thanks Łukasz!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset d672dd34f0bc3addeaf1789d4183e3a37ab110d5 by Miss Islington (bot) 
in branch '3.9':
bpo-38623: Add note about site module (site-packages) (GH-16974) (GH-28537)
https://github.com/python/cpython/commit/d672dd34f0bc3addeaf1789d4183e3a37ab110d5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks, Peter! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10, Python 3.11 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 87f97fe5e6434da51246d70af9e2cd7d63c29fba by Miss Islington (bot) 
in branch '3.10':
bpo-38623: Add note about site module (site-packages) (GH-16974) (GH-28536)
https://github.com/python/cpython/commit/87f97fe5e6434da51246d70af9e2cd7d63c29fba


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[gaborjbernat]

Change by gaborjbernat :


--
nosy: +gaborjbernat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: XML Considered Harmful

[Chris Angelico]

On Fri, Sep 24, 2021 at 7:11 AM Eli the Bearded <*@eli.users.panix.com> wrote:
>
> In comp.lang.python, Christian Gollwitzer   wrote:
> > Am 22.09.21 um 16:52 schrieb Michael F. Stemper:
> >> On 21/09/2021 19.30, Eli the Bearded wrote:
> >>> Yes, CSV files can model that. But it would not be my first choice of
> >>> data format. (Neither would JSON.) I'd probably use XML.
> >> Okay. 'Go not to the elves for counsel, for they will say both no
> >> and yes.' (I'm not actually surprised to find differences of opinion.)
>
> Well, I have a recommendation with my answer.
>
> > It's the same as saying "CSV supports images". Of course it doesn't, its
> > a textfile, but you could encode a JPEG as base64 and then put this
> > string into the cell of a CSV table. That definitely isn't what a sane
> > person would understand as "support".
>
> I'd use one of the netpbm formats instead of JPEG. PBM for one bit
> bitmaps, PGM for one channel (typically grayscale), PPM for three
> channel RGB, and PAM for anything else (two channel gray plus alpha,
> CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to
> map to CSV since it is a three channel format (YCbCr), where the
> channels are typically not at the same resolution. Usually Y is full
> size and the Cb and Cr channels are one quarter size ("4:2:0 chroma
> subsampling"). The unequal size of the channels does not lend itself
> to CSV, but I can't say it's impossible.
>

Examine prior art, and I truly do mean art, from Matt Parker:

https://www.youtube.com/watch?v=UBX2QQHlQ_I

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Add a context manager which suppresses keyboard interruption.

That's not enough technically. This can be any signal handler that raises an 
exception, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

RE: XML Considered Harmful

[Avi Gross via Python-list]

What you are describing Stephen, is what I meant by emulating a relational 
database with tables.

And, FYI, There is no guarantee that two authors with the same name will not be 
assumed to be the same person.

Besides the lack of any one official CSV format, there are oodles of features I 
have seen that are normally external to the CSV. For example, I have often read 
in data from a CSV or similar, where you could tell the software to consider a 
blank or 999 to mean NA and what denotes a line in the file to be ignored as a 
comment and whether a separator is a space or any combination of whitespace and 
what quotes something so say you can hide a comma and how to handle escapes and 
whether to skip blank lines and more.

Now a really good design might place some metadata into the file that can be 
used to set defaults for things like that or incorporate them into the format 
unambiguously. It might calculate the likely data type for various fields and 
store that in the metadata. So even if you stored rectangular data in a CSV 
file, perhaps the early lines would be in some format that can be read as 
comments and supply some info like the above.

Are any of the CSV variants more like that?

-Original Message-
From: Python-list  On 
Behalf Of Stefan Ram
Sent: Thursday, September 23, 2021 5:43 PM
To: python-list@python.org
Subject: Re: XML Considered Harmful

"Avi Gross"  writes:
>But scientific papers seemingly allow oodles of authors and any time 
>you update the data, you may need yet another column.

  You can use three CSV files: papers, persons, and authors:

  papers.csv

1, "Is the accelerated expansion evidence of a change of signature?"

  persons.csv

1, Marc Mars

  authors.csv

1, 1

  I.e., paper 1 is authored by person 1.

  Now, when we learn that José M. M. Senovilla also is a
  co-author of "Is the accelerated expansion evidence of a
  forthcoming change of signature?", we do only have to add
  new rows, no new colums.

  papers.csv

1, "Is the accelerated expansion evidence of a change of signature?"

  persons.csv

1, "Marc Mars"
2, "José M. M. Senovilla"

  authors.csv

1, 1
1, 2

  The real problem with CSV is that there is no CSV.

  This is not a specific data language with a specific
  specification. Instead it is a vague designation for
  a plethora of CSV dialects, which usually dot not even
  have a specification. Compare this with XML. XML has
  a sole specification managed by the W3C.


--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45026] More compact range iterator

[Łukasz Langa]

Łukasz Langa  added the comment:

Good point benchmarking small iterations too, Dennis. I missed that. 

Agreed then, GH-27986 looks like a winner.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[miss-islington]

Change by miss-islington :


--
pull_requests: +26921
pull_request: https://github.com/python/cpython/pull/28536

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 55b45bf707c6c8084db259fe2f8aa08e84ea0d99 by Peter Bittner in 
branch 'main':
bpo-38623: Add note about site module (site-packages) (GH-16974)
https://github.com/python/cpython/commit/55b45bf707c6c8084db259fe2f8aa08e84ea0d99


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38623] Python documentation should mention how to find site-packages

[miss-islington]

Change by miss-islington :


--
pull_requests: +26922
pull_request: https://github.com/python/cpython/pull/28537

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am not sure that it can be solved at Python level.

Possible solutions:

* Add a Lock method (or just a builtin function) which acquires and immediately 
releases the lock, without possibility to interrupt.

if lock._blink(block, timeout):
self._stop()

* Add a context manager which suppresses keyboard interruption.

with suppress_interrupt():
if not lock._blink(block, timeout):
return
self._stop()

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks, Daniel! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11 -Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 91099e25446b214725f8e2bec6f90553260c by Daniel Hillier in 
branch 'main':
bpo-39359: [zipfile] add missing "pwd: expected bytes, got str" exception 
(GH-18031)
https://github.com/python/cpython/commit/91099e25446b214725f8e2bec6f90553260c


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38415] @asynccontextmanager decorated functions are not callable like @contextmanager

[Łukasz Langa]

Łukasz Langa  added the comment:

Look, not even 23 months to land this thing!

Thanks, Jason! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11 -Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38415] @asynccontextmanager decorated functions are not callable like @contextmanager

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 86b833badd3d6864868404ead2f8c7994d24f85c by Jason Fried in branch 
'main':
bpo-38415: Allow using @asynccontextmanager-made ctx managers as decorators 
(GH-16667)
https://github.com/python/cpython/commit/86b833badd3d6864868404ead2f8c7994d24f85c


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

RE: XML Considered Harmful

[Avi Gross via Python-list]

Can we agree that there are way more general ways to store data than
anything currently in common use and that in some ways, CSV and cousins like
TSV are a subset of the others in a sense? There are trees and arbitrary
graphs and many complex data structures often encountered while a program is
running as in-memory objects. Many are not trivial to store.

But some are if all you see is table-like constructs including matrices and
data.frames.

I mean any rectangular data format with umpteen rows and N columns can
trivially be stored in many other formats and especially when it allows some
columns to have NA values. The other format would simply have major
categories that contain components with one per column, and if missing,
represents an NA. Is there any reason JSON or XML cannot include the
contents of any CSV with headers and without loss of info?

Going the other way is harder. Note that a data.frame type of structure
often imposes restrictions on a CSV and requires everything in a column to
be of the same type, or coercible to a common type. (well, not always true
as in using list columns in R.)  But given some arbitrary structure in XML,
can you look at all possible labels and if it is not too complex, make a CSV
with one or more columns for every possible need? It can be a problem if say
a record for an Author allows multiple actual co-authors. Normal books may
let you get by with multiple columns (mostly containing an NA) with names
like author1, author2, author3, ...

But scientific papers seemingly allow oodles of authors and any time you
update the data, you may need yet another column. And, of course, processing
data where many columns have the same meaning is a bit of a pain. Data
structures can also often be nested multiple levels and at some point, CSV
is not a reasonable fit unless you play database games and make multiple
tables you can store and retrieve to make complex queries, as in many
relational database systems. Yes, each such table can be a CSV.

But if you give someone a hammer, they tend to stop using thumbtacks or
other tools. The real question is what kind of data makes good sense for an
application. If a nice rectangular format works, great. Even if not, the
Author problem above can fairly easily be handled by making the author
column something like a character string you compose as "Last1, First1;
Last2, First2; Last3, First3" and that fits fine in a CSV but can be taken
apart in your software if looking for any book by a particular author. Not
optimal, but a workaround I am sure is used.

But using the most abstract and complex storage method is very often
overkill and unless you are very good at it, may well be a fairly slow and
even error-prone way to solve a problem.

-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Thursday, September 23, 2021 9:27 AM
To: Python 
Subject: Re: XML Considered Harmful

On Thu, Sep 23, 2021 at 10:55 PM Mats Wichmann  wrote:
>
> On 9/22/21 10:31, Dennis Lee Bieber wrote:
>
> >   If you control both the data generation and the data 
> > consumption, finding some format  ...
>
> This is really the key.  I rant at people seeming to believe that csv 
> is THE data interchange format, and it's about as bad as it gets at 
> that, if you have a choice.  xml is noisy but at least (potentially) 
> self-documenting, and ought to be able to recover from certain errors.
> The problem with csv is that a substantial chunk of the world seems to 
> live inside Excel, and so data is commonly both generated in csv so it 
> can be imported into excel and generated in csv as a result of 
> exporting from excel, so the parts often are *not* in your control.
>
> Sigh.

The only people who think that CSV is *the* format are people who habitually
live in spreadsheets. People who move data around the internet, from program
to program, are much more likely to assume that JSON is the sole format. Of
course, there is no single ultimate data interchange format, but JSON is a
lot closer to one than CSV is.

(Or to be more precise: any such thing as a "single ultimate data
interchange format" will be so generic that it isn't enough to define
everything. For instance, "a stream of bytes" is a universal data
interchange format, but that's not ultimately a very useful claim.)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue15870] PyType_FromSpec should take metaclass as an argument

[Josh Haberman]

Josh Haberman  added the comment:

> It's better to pass the metaclass as a function argument, as with bases. I'd 
> prefer adding a new function that using a slot.

Bases are available both as a slot (Py_tp_bases) and as an argument 
(PyType_FromSpecWithBases).  I don't see why this has to be an either/or 
proposition.  Both can be useful.

Either would satisfy my use case.  I'm constructing N such classes, so the spec 
won't be statically initialized anyway and the initialization issues on Windows 
don't apply.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: XML Considered Harmful

[Michael F. Stemper]

On 23/09/2021 12.51, Eli the Bearded wrote:

Am 22.09.21 um 16:52 schrieb Michael F. Stemper:

On 21/09/2021 19.30, Eli the Bearded wrote:

Yes, CSV files can model that. But it would not be my first choice of
data format. (Neither would JSON.) I'd probably use XML.

Okay. 'Go not to the elves for counsel, for they will say both no
and yes.' (I'm not actually surprised to find differences of opinion.)


Well, I have a recommendation with my answer.


Sorry, didn't mean that to be disparaging.

--
Michael F. Stemper
This post contains greater than 95% post-consumer bytes by weight.
--
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Michael F. Stemper]

On 22/09/2021 17.37, Dennis Lee Bieber wrote:

On Wed, 22 Sep 2021 09:52:59 -0500, "Michael F. Stemper"
 declaimed the following:

On 21/09/2021 19.30, Eli the Bearded wrote:

In comp.lang.python, Michael F. Stemper  wrote:



How does CSV handle hierarchical data? For instance, I have



Can CSV files model this sort of situation?





Yes, CSV files can model that. But it would not be my first choice of
data format. (Neither would JSON.) I'd probably use XML.


Okay. 'Go not to the elves for counsel, for they will say both no
and yes.' (I'm not actually surprised to find differences of opinion.)


You'd have to include a "level" (and/or data type if multiple objects
can be at the same level) field (as the first field) in CSV which
identifies how to parse the rest of the CSV data (well, technically, the
CSV module has "parsed" it -- in terms of splitting at commas, handling
quoted strings (which may contain commas which are not split points, etc.).

1-generator, name
2-fuel, name, UOM, heat-content, price
2-curve, name
3-point, X, Y
3-point, X, Y
...
2-curve, name
3-point, X, Y
3-point, X, Y


This reminds me of how my (former) employer imported data models into
our systems from the 1970s until the mid-2000s. We had 80-column records
(called "card images"), that would have looked like:

FUEL0 LIGNITETON13.610 043.581
UNIT1 COAL CREK1
UNIT2 ...

The specific columns for the start and end of each field on each record
were defined in a thousand-plus page document. (We modeled all of a
power system, not just economic data about generators.)

However, this doesn't seem like it would fit too well with the csv
module, since it requires a lot more logic on the part of the consuming
program.

Interesting flashback, though.

--
Michael F. Stemper
Deuteronomy 24:17
--
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Christian Gollwitzer]

Am 22.09.21 um 16:52 schrieb Michael F. Stemper:

On 21/09/2021 19.30, Eli the Bearded wrote:

Yes, CSV files can model that. But it would not be my first choice of
data format. (Neither would JSON.) I'd probably use XML.


Okay. 'Go not to the elves for counsel, for they will say both no
and yes.' (I'm not actually surprised to find differences of opinion.)


It is wrong, CSV has no model of hierarchical data. A CSV file is a 2d 
table, just like a database table or an Excel sheet.


You can /layer/ high-dimensional data on top of a 2D table, there is the 
relational algebra theory behind this, but it is wrong (or misleading at 
best) to say that CSV can model hierarchical data.


It's the same as saying "CSV supports images". Of course it doesn't, its 
a textfile, but you could encode a JPEG as base64 and then put this 
string into the cell of a CSV table. That definitely isn't what a sane 
person would understand as "support".


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Eli the Bearded]

In comp.lang.python, Christian Gollwitzer   wrote:
> Am 22.09.21 um 16:52 schrieb Michael F. Stemper:
>> On 21/09/2021 19.30, Eli the Bearded wrote:
>>> Yes, CSV files can model that. But it would not be my first choice of
>>> data format. (Neither would JSON.) I'd probably use XML.
>> Okay. 'Go not to the elves for counsel, for they will say both no
>> and yes.' (I'm not actually surprised to find differences of opinion.)

Well, I have a recommendation with my answer.

> It's the same as saying "CSV supports images". Of course it doesn't, its 
> a textfile, but you could encode a JPEG as base64 and then put this 
> string into the cell of a CSV table. That definitely isn't what a sane 
> person would understand as "support".

I'd use one of the netpbm formats instead of JPEG. PBM for one bit
bitmaps, PGM for one channel (typically grayscale), PPM for three
channel RGB, and PAM for anything else (two channel gray plus alpha,
CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to
map to CSV since it is a three channel format (YCbCr), where the
channels are typically not at the same resolution. Usually Y is full
size and the Cb and Cr channels are one quarter size ("4:2:0 chroma
subsampling"). The unequal size of the channels does not lend itself
to CSV, but I can't say it's impossible.

But maybe you meant the whole JFIF or Exif JPEG file format base64
encoded with no attempt to understand the image. That sort of thing
is common in JSON, and I've seen it in YAML, too. It wouldn't surprise
me if people do that in CSV or XML, but I have so far avoided seeing
that. I used that method for sticking a tiny PNG in a CSS file just
earlier this month. The whole PNG was smaller than the typical headers
of an HTTP/1.1 request and response, so I figured "don't make it a
separate file".

Elijah
--
can at this point recegnize a bunch of "magic numbers" in base64


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Free OCR package in Python and selecting appropriate widget for the GUI

[Mohsen Owzar]

DFS schrieb am Mittwoch, 22. September 2021 um 09:41:42 UTC+2:
> On 9/22/2021 1:54 AM, Mohsen Owzar wrote: 
> > DFS schrieb am Mittwoch, 22. September 2021 um 05:10:30 UTC+2: 
> >> On 9/21/2021 10:38 PM, Mohsen Owzar wrote: 
> >>> DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: 
>  On 9/21/2021 4:36 AM, Mohsen Owzar wrote: 
> > Hi Guys 
> > Long time ago I've written a program in Malab a GUI for solving Sudoku 
> > puzzles, which worked not so bad. 
> > Now I try to write this GUI with Python with PyQt5 or TKinter. 
> > First question is: 
> > Is there any free OCR software, packages or code in Python, which I can 
> > use to recognize the given digits and their positions in the puzzle 
> > square. 
> > Second: 
> > Because, I can not attach a picture to this post, I try to describe my 
> > picture of my GUI. 
>  Draw your GUI in PyQt designer or other graphics tool, then upload a 
>  screenshot of it to imgur, then post the link to the picture. 
> >>> Thanks, for your answer. 
> >>> But, what is "imgur"? 
> >>> I'm not so familiar with handling of pictures in this group. 
> >>> How can I call "imgur" or how can I get there? 
> >>> 
> >>> Regards 
> >>> Mohsen 
> >> www.imgur.com 
> >> 
> >> It's a website you can upload image files or screenshots to. Then you 
> >> can copy a link to your picture and post the link here. 
> > I have already posted the link, but I can not see it anywhere. 
> > Now, I post it again: 
> > https://imgur.com/a/Vh8P2TE 
> > I hope that you can see my two images. 
> > Regards 
> > Mohsen
> Got it. 
> 
> I haven't used tkinter. In PyQt5 designer I think you should use one 
> QTextEdit control for each square. 
> 
> 
> Each square with the small black font can be initially populated with 
> 
> 1 2 3 
> 4 5 6 
> 7 8 9 
> 
> 
> 
> https://imgur.com/lTcEiML 
> 
> 
> 
> some starter python code (maybe save as sudoku.py) 
> 
> = 
> from PyQt5 import Qt, QtCore, QtGui, QtWidgets, uic 
> from PyQt5.Qt import * 
> from PyQt5.QtCore import * 
> from PyQt5.QtGui import * 
> from PyQt5.QtWidgets import * 
> 
> #objects 
> app = QtWidgets.QApplication([]) 
> frm = uic.loadUi("sudoku.ui") 
> 
> 
> #grid = a collection of squares 
> grids = 1 
> 
> #squares = number of squares per grid 
> squares = 9 
> 
> #fill the squares with 1-9 
> def populateSquares(): 
> for i in range(grids,grids+1): 
> for j in range(1,squares+1): 
> widget = frm.findChild(QtWidgets.QTextEdit, "txt{}_{}".format(i,j)) 
> widget.setText("1 2 3 4 5 6 7 8 9") 
> 
> #read data from squares 
> def readSquares(): 
> for i in range(grids,grids+1): 
> for j in range(1,squares+1): 
> print("txt%d_%d contains: %s" % 
> (i,j,frm.findChild(QtWidgets.QTextEdit, 
> "txt{}_{}".format(i,j)).toPlainText())) 
> 
> 
> #connect pushbuttons to code 
> frm.btnPopulate.clicked.connect(populateSquares) 
> frm.btnReadContents.clicked.connect(readSquares) 
> 
> #show main form 
> frm.show() 
> 
> #initiate application 
> app.exec() 
> = 
> 
> 
> 
> 
> 
> .ui file (ie save as sudoku.ui) 
> = 
> 
>  
>  
> MainWindow 
>  
>  
>  
> 0 
> 0 
> 325 
> 288 
>  
>  
>  
> Sudoku 
>  
>  
>  
>  
>  
> 32 
> 22 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> background-color: rgb(255, 255, 127); 
>  
>  
> QFrame::StyledPanel 
>  
>  
> QFrame::Sunken 
>  
>  
> Qt::ScrollBarAlwaysOff 
>  
>  
> true 
>  
>  
>  
>  
>  
> 114 
> 22 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> background-color: rgb(255, 255, 127); 
>  
>  
> QFrame::StyledPanel 
>  
>  
> QFrame::Sunken 
>  
>  
> Qt::ScrollBarAlwaysOff 
>  
>  
> true 
>  
>  
>  
>  
>  
> 196 
> 22 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> background-color: rgb(255, 255, 127); 
>  
>  
> QFrame::StyledPanel 
>  
>  
> QFrame::Sunken 
>  
>  
> Qt::ScrollBarAlwaysOff 
>  
>  
> true 
>  
>  
>  
>  
>  
> 32 
> 86 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> background-color: rgb(255, 255, 127); 
>  
>  
> QFrame::StyledPanel 
>  
>  
> QFrame::Sunken 
>  
>  
> Qt::ScrollBarAlwaysOff 
>  
>  
> true 
>  
>  
>  
>  
>  
> 114 
> 86 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> background-color: rgb(255, 255, 127); 
>  
>  
> QFrame::StyledPanel 
>  
>  
> QFrame::Sunken 
>  
>  
> Qt::ScrollBarAlwaysOff 
>  
>  
> true 
>  
>  
>  
>  
>  
> 196 
> 86 
> 83 
> 65 
>  
>  
>  
>  
> Courier 
> 12 
> 50 
> false 
>  
>  
>  
> false 
>  
>  
> color: rgb(0, 0, 127); 
> 

téléchargement python

[olivier Perchet]

   Envoye `a partir de [1]Courrier pour Windows





   [2][IMG] Garanti sans virus. [3]www.avast.com

References

   Visible links
   1. https://go.microsoft.com/fwlink/?LinkId=550986
   2. 
https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=emailclient
   3. 
https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=emailclient
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45101] Small inconsistency in usage message between the python and shell script versions of python-config

[Kien Dang]

Kien Dang  added the comment:

Thanks. I didn't know about the `pkg-config` thing. I just checked with 
pkg-config on macOS

`pkg-config --help` prints usage instructions to stdout

pkg-config followed by invalid options prints output to stderr, for example 
`pkg-config --asdf` prints `Unknown option --asdf` to stderr.

The PR I submitted does something similar, making both python and shell script 
versions of `python-config` to print out the usage instruction message to 
stdout when --help is provided and stderr when given invalid arguments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 55576893b31452ba739e01189424cd62cf11ed20 by Miss Islington (bot) 
in branch '3.10':
bpo-43760: Document PyThreadState.use_tracing removal (GH-28527) (GH-28529)
https://github.com/python/cpython/commit/55576893b31452ba739e01189424cd62cf11ed20


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +pablogsal
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +26920
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28532

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[STINNER Victor]

STINNER Victor  added the comment:

For curious people, see also bpo-44422 "Fix threading.enumerate() reentrant 
call": another example of race condition recently fixed in the threading 
module. But it's unrelated to this bug ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[STINNER Victor]

Change by STINNER Victor :


Added file: https://bugs.python.org/file50300/threading_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45274] Race condition in Thread._wait_for_tstate_lock()

[STINNER Victor]

New submission from STINNER Victor :

Bernát Gábor found an interesting bug on Windows. Sometimes, when a process is 
interrupted on Windows with CTRL+C, its parent process hangs in thread.join():
https://twitter.com/gjbernat/status/1440949682759426050

Reproducer:

* Install https://github.com/gaborbernat/tox/tree/2159
* Make an empty folder and put the above tox.ini in it
* Invoke python -m tox and once that command is sleeping press CTRL+C (the app 
should lock there indefinitely).

tox.ini:
---

[testenv]
skip_install = true
commands = python -c 'import os; print(f"start {os.getpid()}"); import time; 
time.sleep(100); print("end");'
---

Source: https://gist.github.com/gaborbernat/f1e1aff0f2ee514b50a92a4d019d4d13

I tried to attach the Python process in Python: there is a single thread, the 
main thread which is blocked in thread.join(). You can also see it in the 
faulthandler traceback.

I did a long analysis of the _tstate_lock and I checked that thread really 
completed. Raw debug traces:

== thread 6200 exit ==
 
thread_run[pid=3984, thread_id=6200]: clear
PyThreadState_Clear[pid=3984, thread_id=6200]: on_delete()
release_sentinel[pid=3984, thread_id=6200]: enter
release_sentinel[pid=3984, thread_id=6200]: release(obj=01C1122669C0, 
lock=01C110BBDA00)
release_sentinel[pid=3984, thread_id=6200]: exit
PyThreadState_Clear[pid=3984, thread_id=6200]: on_delete()--
 
== main thread is calling join() but gets a KeyboardInterrupt ==
 
[pid=3984, thread_id=8000] Lock.acquire() -> ACQUIRED
Current thread 0x1f40 (most recent call first):
  File "C:\vstinner\python\3.10\lib\threading.py", line 1118 in 
_wait_for_tstate_lock
  File "C:\vstinner\python\3.10\lib\threading.py", line 1092 in join
  File "C:\vstinner\env\lib\site-packages\tox\session\cmthread_run[pid=3984, 
thread_id=6200]: exit
d\run\common.py", line 203 in execute
  File "C:\vstinner\env\lib\site-packages\tox\session\cmd\run\sequential.py", 
line 20 in run_sequential
  File "C:\vstinner\env\lib\site-packages\tox\session\cmd\legacy.py", line 104 
in legacy
  File "C:\vstinner\env\lib\site-packages\tox\run.py", line 49 in main
  File "C:\vstinner\env\lib\site-packages\tox\run.py", line 23 in run
  File "C:\vstinner\env\lib\site-packages\tox\__main__.py", line 4 in 
  File "C:\vstinner\python\3.10\lib\runpy.py", line 86 in _run_code
  File "C:\vstinner\python\3.10\lib\runpy.py", line 196 in _run_module_as_main
_wait_for_tstate_lock[pid=3984, current thread_id=8000, self thread_id=6200]: 
EXC: KeyboardInterrupt(); acquired? None
 
== main thread calls repr(thread) ==
 
ROOT: [3984] KeyboardInterrupt - teardown started
_wait_for_tstate_lock[pid=3984, current thread_id=8000, self thread_id=6200]: 
acquire(block=False, timeout=-1): lock obj= 0x1c1122669c0
  File "C:\vstinner\python\3.10\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
  File "C:\vstinner\python\3.10\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
  File "C:\vstinner\env\lib\site-packages\tox\__main__.py", line 4, in 
run()
  File "C:\vstinner\env\lib\site-packages\tox\run.py", line 23, in run
result = main(sys.argv[1:] if args is None else args)
  File "C:\vstinner\env\lib\site-packages\tox\run.py", line 49, in main
result = handler(state)
  File "C:\vstinner\env\lib\site-packages\tox\session\cmd\legacy.py", line 104, 
in legacy
return run_sequential(state)
  File "C:\vstinner\env\lib\site-packages\tox\session\cmd\run\sequential.py", 
line 20, in run_sequential
return execute(state, max_workers=1, has_spinner=False, live=True)
  File "C:\vstinner\env\lib\site-packages\tox\session\cmd\run\common.py", line 
217, in execute
print(f'join {thread}')
  File "C:\vstinner\python\3.10\lib\threading.py", line 901, in __repr__
self.is_alive() # easy way to get ._is_stopped set when appropriate
  File "C:\vstinner\python\3.10\lib\threading.py", line 1181, in is_alive
self._wait_for_tstate_lock(False)
  File "C:\vstinner\python\3.10\lib\threading.py", line 1113, in 
_wait_for_tstate_lock
traceback.print_stack()
_wait_for_tstate_lock[pid=3984, current thread_id=8000, self thread_id=6200]: 
failed to acquire 0x1c1122669c0
_wait_for_tstate_lock[pid=3984, current thread_id=8000, self thread_id=6200]: 
exit 0x1c1122669c0
join 
 
== main thread calls thread.join()... which hangs ==
 
_wait_for_tstate_lock[pid=3984, current thread_id=8000, self thread_id=6200]: 
acquire(block=True, timeout=-1): lock obj= 0x1c1122669c0
  File "C:\vstinner\python\3.10\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
  File "C:\vstinner\python\3.10\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
  File "C:\vstinner\env\lib\site-packages\tox\__main__.py", line 4, in 
run()
  File "C:\vstinner\env\lib\site-packages\tox\run.py", line 23, in run
result = main(sys.argv[1:] if args is None else args)
  File 

[issue15870] PyType_FromSpec should take metaclass as an argument

[Steve Dower]

Steve Dower  added the comment:

But at least if it's available as a slot then a module is *able* to use it with 
limited ABI going backwards. A new function doesn't allow that.

And yeah, it means they have to write more complex code than just a static 
array definition, but people are willing and able to do it.

I don't think there's anything about the current FromSpec that suggests all the 
parameters have to be interpreter-agnostic. And the design does suggest that 
we'll add more later without having to break the ABI (otherwise it would be a 
regular struct).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15870] PyType_FromSpec should take metaclass as an argument

[Petr Viktorin]

Petr Viktorin  added the comment:

- specs/slots are (usually) constant & static; pointers to types are not always 
C constant expressions (on Windows, if they come from a different DLL)
- specs/slots are (usually) shared across all subinterpreters; types are 
specific to a single interpreter

It's better to pass the metaclass as a function argument, as with bases. I'd 
prefer adding a new function that using a slot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44309] Add support for yescrypt in crypt.

[David Mandelberg]

Change by David Mandelberg :


--
nosy: +dseomn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15870] PyType_FromSpec should take metaclass as an argument

[Josh Haberman]

Josh Haberman  added the comment:

> Passing the metaclass as a slot seems like the right idea for this API, 
> though I recall there being some concern about the API (IIRC, mixing function 
> pointers and data pointers doesn't work on some platforms?)

PyType_Slot is defined as a void* (not a function pointer): 
https://github.com/python/cpython/blob/8492b729ae97737d22544f2102559b2b8dd03a03/Include/object.h#L223-L226

So putting a PyTypeObject* into a slot would appear to be more kosher than 
function pointers.

Overall, a slot seems like a great first approach.  It doesn't require any new 
functions, which seems like a plus.  If the any linking issues a la tp_base are 
seen, a new function could be added later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42969] pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful

[Jeremy Maitin-Shepard]

Jeremy Maitin-Shepard  added the comment:

In general, I view hanging threads as the least bad thing to do when faced with 
either acquiring the GIL or not returning at all.  There is a lot of existing 
usage of Python that currently poses a risk of random crashes and memory 
corruption while Python is exiting, and I would like to fix that.

However, I would certainly recommend that code using the Python C API attempt 
to avoid threads getting to that state in the first place.  I added a "finalize 
block" mechanism to that PR which is intended to provide a way to attempt to 
acquire the GIL in a way that ensures the GIL won't get hung.  I would welcome 
feedback on that.  A common use case for that API might be a non-Python created 
thread that wants to invoke some sort of asynchronous callback handler via 
Python APIs.

For Python daemon threads that you control, you can avoid them hanging by 
registering an atexit function that signals them to exit and then waits until 
they do.

vsteinner: Regarding processing the Windows messages, I updated the PR to 
include a link to the MSDN documentation that led me to think it was a good 
idea.

vstinner: As for random code outside of Python itself that is using 
`PyThread_exit_thread`: although I suppose there are legitimate use cases for 
`pthread_exit` and `_endthreadex`, these functions are only safe with the 
cooperation of the entire call stack of the thread.  Additionally, while 
`pthread_exit` and `_endthreadex` have similar behavior for C code, they don't 
have the same behavior for C++ code, and that difference seems like a likely 
source of problems.  Finally, I would say Python itself does not guarantee that 
its call stacks safely cooperate with `pthread_exit` (at the very least, there 
are sure to be memory leaks).  Therefore, I think Python should not encourage 
its use by leaving it as a non-deprecated public API.  If a user wishes to 
terminate a thread, they can invoke `pthread_exit` or `_endthreadex` directly, 
ideally without any Python functions in the call stack, and can refer to the 
documentation of those functions to understand the implications.

gps: The reasons I believe hanging the thread is better than `pthread_exit`:
- `pthread_exit` essentially throws an exception.  In theory that means you 
could do proper cleanup via C++ destructors and/or re-throwing catch blocks.  
But in practice existing extension code is not designed to do that, and it 
would be quite a complex task to modify it to do proper cleanup, and on Windows 
the cleanup wouldn't run anyway.
- Additionally, throwing an exception means if there is a `noexcept` function 
in the call stack, the program terminates.  We would need to document that you 
aren't allowed to call Python APIs if there is a `noexcept` function in the 
call stack.  If you have a `catch(...)` in the call stack, then you may 
inadvertently catch the exception and return control back to Python at a point 
that assumes it owns the GIL, which will cause all sorts of havoc.  We would 
likewise need to document that you can't have a non-rethrowing `catch(...)` 
block in the call stack (I believe pybind11 has some of those).
- Throwing an exception also means C++ destructors run.  pybind11 has a smart 
pointer type that holds a `PyObject` and whose destructor calls `Py_DECREF`.  
That causes a crash when `pthread_exit` unwinds the stack, since the thread 
doesn't own the GIL.

Those are the additional problems specific to `pthread_exit`.  As gps noted, 
there is the additional problem of memory corruption common to both 
`pthread_exit` and `_endthreadex`:
- Data structures that are accessible from other threads may contain pointers 
to data on the thread's stack.  For example, with certain types of 
locks/signalling mechanisms it is common to store a linked list node on the 
stack that as then added to a list of waiting threads.  If we destroy the 
thread stack without proper cleanup (and that proper cleanup definitely won't 
happen with `_endthreadex`, and probably in most cases still won't happen with 
`pthread_exit`), the data structure has now become corrupted.

I don't think hanging the thread really increases the risk of deadlock over the 
status quo.  In theory someone could have a C++ destructor that does some 
cleanup that safely pevents deadlock, but that is not portable to Windows, and 
I expect that properly implemented `pthread_exit`-safe code is extremely rare.

I think we would want to ensure that Python itself is implemented in such a way 
as to not deadlock if Python-created threads with only Python functions in the 
call stack hang.  Mostly that would amount to not holding mutexes while calling 
functions that may transitively attempt to acquire the GIL (or release and then 
re-acquire the GIL).  That is probably a good practice for avoiding deadlock 
even when not finalizing.

We would also want to document that external code using the Python API should 
be safe from deadlock if a 

[issue45256] Remove the usage of the C stack in Python to Python calls

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Though I wouldn't like to lose the ability to extract the Python stack by 
> inspecting native memory alone.

Don't worry about it, I am personally making sure that keeps being possible. it 
will need some changes in the tools, but not any more that any other changes 
between minor versions of the interpreter

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15870] PyType_FromSpec should take metaclass as an argument

[Sebastian Berg]

Sebastian Berg  added the comment:

I can make a PR from the patch (and add the `Py_tp_metaclass` slot if desired) 
with a basic test here, if that is what is blocking things.

Fixing the type and size of the allocation (as the patch does) would allow me 
to give people a way to create a new NumPy DType dynamically.  I only need the 
user to call my initialization function as soon as the type was created (with 
`PyType_FromSpec` or otherwise).
(And I can avoid any internal acrobatics creating the type for the user; this 
stuff tends to be super confusing even if the principles are fairly straight 
forward...)

Happy to pursue other avenues, but I am not clear which...


> IIRC, mixing function pointers and data pointers doesn't work on some 
> platforms?

... I guess it is too late to do some weird thing like (not sure it would be 
reasonable or is valid anyway though):

typedef union {
void *pdata;
void (*pfunc)(void);
} slot_value;

I am a bit interested in it, because I want to use a `FromSpec` API in NumPy 
and it would be nice to be sure I can grow it to include data without too much 
hassle.  But the easier thing may just be to add one or two `void *reserved` 
slot to the spec struct that must be NULL for now...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45272] 'os.path' should not be a frozen module

[Eric Snow]

Eric Snow  added the comment:

The matter here boils down to the design of _imp.is_frozen() [1].  It only 
checks to see if the given module name shows up in the list of frozen modules 
in Python/frozen.c.  This broke things when I froze os and posixpath/ntpath.

The simplest solution was to include os.path in the list of modules in 
frozen.c.  The better solution would be to have _imp.is_frozen() check the 
module in sys.modules.


[1] see find_frozen() in Python/import.c

--
nosy: +barry, brett.cannon, jaraco, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45273] OS-specific frozen modules are built, even on other OSes.

[Eric Snow]

Eric Snow  added the comment:

FYI, currently os.path is also frozen and *is* OS-specific.  However, that's a 
separate matter.  See bpo-45272.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45273] OS-specific frozen modules are built, even on other OSes.

[Eric Snow]

New submission from Eric Snow :

The list of frozen modules in Python/frozen.c is generated by 
Tools/scripts/freeze_modules.py.  Currently we freeze both posixpath and 
ntpath, even though for startup we only need one of the two (depending on the 
OS).  In this case both modules are available on all platforms (and only 
os.path differs), so we might be okay to leave both frozen.  The cost isn't 
high.

However, we may need to accommodate freezing a module only on a subset of 
supported OSes:

* if the extra cost (to the size of the executable) is too high
* if there's an OS-specific module that is only available on that OS

In that case, we'd need to generate the appropriate ifdefs in Python/frozen.c.  
We can't just exclude the module during generation since frozen.c is committed 
to the repo.

--
assignee: eric.snow
components: Interpreter Core
messages: 402514
nosy: eric.snow, steve.dower
priority: normal
severity: normal
stage: needs patch
status: open
title: OS-specific frozen modules are built, even on other OSes.
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45271] Add a 'find' method (a'la str) to list

[Raymond Hettinger]

Raymond Hettinger  added the comment:

This is one bit of symmetry that we shouldn't pursue.  The find() methods have 
been a recurring source of bugs because of failing to check for a -1 result but 
having that be a valid index into the sequence.

Also, the sequence APIs are long established.  If find() were really needed, we 
would have felt the pain and added it long ago.

As Steve says, you can take this to python-ideas, but my recommendation is to 
not do it all.

--
nosy: +rhettinger
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44687] io.BufferedReader:peek() closes underlying file, breaking peek/read expectations

[Steve Dower]

Steve Dower  added the comment:

Seems like it would be better to not check the file open/closed state on 
peek/read when there is still buffered data?

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15870] PyType_FromSpec should take metaclass as an argument

[Steve Dower]

Steve Dower  added the comment:

Passing the metaclass as a slot seems like the right idea for this API, though 
I recall there being some concern about the API (IIRC, mixing function pointers 
and data pointers doesn't work on some platforms?) that mean it'll be 
deprecated in the future.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40346] Add random.BaseRandom to ease implementation of subclasses

[Matt Bogosian]

Matt Bogosian  added the comment:

Thanks! Where's that documented? (Apologies if I missed it.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42969] pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful

[Gregory P. Smith]

Gregory P. Smith  added the comment:

A PR adding a `Py_SetThreadExitCallback(func)` API guaranteeing the callback is 
called before `pthread_exit()` would allow anyone wanting to deal with their 
deadlocks to register `abort()` or `while(1) pause();` as their exit callback.  
I'd be okay with that.  Anyone care to make a PR for that?

What should it do when SetThreadExitCallback has already been called?  Is that 
an error?  Are the callbacks chained?  In which order?  If someone passes 
nullptr does that undo it (please no!).  It is process global state that many 
things could wind up having an opinion on each with their own reason to require 
theirs to be the only one.  I vote for returning an error if a callback has 
already been set.  And not allowing unsetting a callback.

What we'd do internally at work is always guarantee our codebase's early 
application startup code (because we have such a thing) calls that to setup 
whichever exit callback we deem appropriate for everyone instead of today's 
default deadlock potential.

On pausing... agreed, it doesn't feel _comfortable_.  To me when faced with a 
known potential deadlock situation the only comfortable thing is to abort() as 
a process dying is always more useful than process hanging (or worse, partially 
hanging).

Sleeping on the problem, I don't really understand how `while(1) pause();` is 
significantly different than `pthread_exit()` when it comes to deadlocks, as it 
seems like an instantly terminated thread having state that needs cleanup 
should lead to a similar outcome as a thread with stuff needing cleanup that is 
now stuck in an infinite pause loop.  Neither of them is going to cleanup 
whatever (presumably a lock they hold) that leads something else to deadlock?  
I guess the difference is that the thread stack  memory is at least not 
released back for potential reuse while other threads and pointers could still 
be referring to it when pausing as opposed to a pthread_exit?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40346] Add random.BaseRandom to ease implementation of subclasses

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Matt, your example works in 3.11. In earlier versions you need to override the 
__new__ method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45244] pip not installed with fresh python3.8.10 installation

[Steve Dower]

Steve Dower  added the comment:

Can you attach the log files stored in your %TEMP% as a zip file? They should 
all start with "Python".

Be aware that they may contain personal information, such as usernames or 
machine names. Feel free to scrub those first if necessary.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45272] 'os.path' should not be a frozen module

[Steve Dower]

New submission from Steve Dower :

I noticed that Python/frozen.c includes posixpath as 'os.path'.

This is not correct, and shouldn't be necessary anyway, because os.path is just 
an attribute in "os" and not a concrete module (see Lib/os.py#L95 for the bit 
that makes it importable, and Lib/os.py#L61 and Lib/os.py#L81 for the imports).

--
assignee: eric.snow
messages: 402506
nosy: eric.snow, steve.dower
priority: normal
severity: normal
stage: needs patch
status: open
title: 'os.path' should not be a frozen module
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45213] Frozen modules are looked up using a linear search.

[Steve Dower]

Steve Dower  added the comment:

Could we add a compile-time requirement (most likely checked in tests, rather 
than at build) that _PyImport_FrozenModules be sorted? Then we can at least 
bail out of the loop early, and one day someone could make it a binary search.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45256] Remove the usage of the C stack in Python to Python calls

[Steve Dower]

Steve Dower  added the comment:

I fully support implementing the core idea of Stackless Python :)

I spent a whole EuroPython a couple of years back discussing the idea with 
(apparently) everyone except Mark.

Though I wouldn't like to lose the ability to extract the Python stack by 
inspecting native memory alone. That is very useful for debugging, particularly 
when you've only got a crash dump. So provided all the code objects are only an 
indirection or two away from a local (or better yet, parameter) value, it 
should be fine.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Terry J. Reedy]

Terry J. Reedy  added the comment:

You added the same file twice.  Once was enough.

To repeat: IDLE has no knowledge of color chooser.  The Custom Colors bar is 
Windows specific.  There is no such thing on macOS.


PS: when responding by email, please delete the quoted message.  It is 
redundant and therefore noise when posted on the web page.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45271] Add a 'find' method (a'la str) to list

[Steve Dower]

Steve Dower  added the comment:

Strings are already special in that str.index() and str.find() both find 
substrings, while list.index() only finds a single element.

If .find() also searched for a substring of the list, I think this could be 
helpful. Even more so if it used an efficient algorithm (bearing in mind that 
the arbitrary comparisons between elements - something else that doesn't exist 
in strings - would make this complicated).

This is probably something to bring up on the python-ideas mailing list first, 
anyway. Symmetry is not a sufficient reason in itself to add new APIs - often 
the asymmetry exists because the "missing" one is only there for 
legacy/compatibility reasons, not because it is a good API.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Pramod]

Pramod  added the comment:

I understand that, but is it possible to keep only unique colors in the
Custom Color palette, and when user chooses the exact same color to be in
the Custom Color palette it would return a pop up like "Color already in
Custom Color palette".

It would be better if the color strings which IDLE receives, are checked if
present already in Custom Colors, if not it then passes it to tkinter and
tk.

On Thu, 23 Sept 2021 at 21:08, Terry J. Reedy 
wrote:

>
> Terry J. Reedy  added the comment:
>
> IDLE has nothing to do with the operation of the color chooser.  IDLE
> calls a tkinter function that calls a tk functions that calls the OS Window
> Manager color chooser.  The choice you make is returned to IDLE as a color
> string that IDLE can later pass to tkinter and tk.  What you see on
> Windows, in your image is the MS Windows 10 color chooser.  The macOS color
> chooser is *completely* different.
>
> The Windows Custom Color bar presumes that you make a small set of
> distinguishable custom colors.  The main use in IDLE would be if you create
> a custom backgound color that you want to use for multiple
> foreground/background pairs.  The use in Windows would be if you have a
> custom palette that you want to use for multiple applications.
>
> --
> resolution:  -> third party
> stage:  -> resolved
> status: open -> closed
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Pramod]

Pramod  added the comment:

Sorry for adding my file again. Can you please state the reason for
removing the file Mr. Terry J Reedy , was it because I haven't signed the
Contributor Agreement at that time? But now that I have signed it, can I
add thehttps://bugs.python.org/file50298/python bug 1.png file again?

On Thu, 23 Sept 2021 at 20:58, Terry J. Reedy 
wrote:

>
> Change by Terry J. Reedy :
>
>
> Removed file: https://bugs.python.org/file50297/python bug 1.png
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Terry J. Reedy]

Terry J. Reedy  added the comment:

IDLE has nothing to do with the operation of the color chooser.  IDLE calls a 
tkinter function that calls a tk functions that calls the OS Window Manager 
color chooser.  The choice you make is returned to IDLE as a color string that 
IDLE can later pass to tkinter and tk.  What you see on Windows, in your image 
is the MS Windows 10 color chooser.  The macOS color chooser is *completely* 
different.

The Windows Custom Color bar presumes that you make a small set of 
distinguishable custom colors.  The main use in IDLE would be if you create a 
custom backgound color that you want to use for multiple foreground/background 
pairs.  The use in Windows would be if you have a custom palette that you want 
to use for multiple applications.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Terry J. Reedy]

Change by Terry J. Reedy :


Removed file: https://bugs.python.org/file50297/python bug 1.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Pramod]

Change by Pramod :


Added file: https://bugs.python.org/file50298/python bug 1.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom Colors" adds same colour multiple times

[Pramod]

New submission from Pramod :

When customising my Python IDLE (in Options>Configure IDLE>Highlights>Choose 
color for:), if I choose a color from the palette and click on the button "Add 
to Custom Colors" multiple times, all my existing colors are replaced by a 
single color one by one, instead I feel it would be better if Python gave a 
popup saying something like "the selected color already exists in Custom 
colors".

--
assignee:  -> terry.reedy
components: +IDLE
nosy: +terry.reedy
title: Clicking "Add to Custom C -> Clicking "Add to Custom Colors" adds same 
colour multiple times
type:  -> behavior
versions: +Python 3.9
Added file: https://bugs.python.org/file50297/python bug 1.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45271] Add a 'find' method (a'la str) to list

[Dávid Nemeskey]

Change by Dávid Nemeskey :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45271] Add a 'find' method (a'la str) to list

[Dávid Nemeskey]

New submission from Dávid Nemeskey :

There is an unjustified asymmetry between `str` and `list`, as far as lookup 
goes. Both have an `index()` method that returns the first index of a value, or 
raises a `ValueError` if it doesn't exist. However, only `str` has the `find` 
method, which returns -1 if the value is not in the string.

I think it would make sense to add `find` to `list` as well. For starters, it 
would make the API between the two sequence types more consistent. More 
importantly (though it depends on the use-case), `find` is usually more 
convenient than `index`, as one doesn't have to worry about handling an 
exception. As a bonus, since `list` is mutable, it allows one to write code 
such as

if (idx := lst.find(value)) == -1:
lst.append(value)
call_some_function(lst[idx])

, making the method even more useful as it is in `str`.

--
messages: 402497
nosy: nemeskeyd
priority: normal
severity: normal
status: open
title: Add a 'find' method (a'la str) to list

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45270] Clicking "Add to Custom C

[Pramod]

Change by Pramod :


--
nosy: pramodsarathy17
priority: normal
severity: normal
status: open
title: Clicking "Add to Custom C

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 11.0 -> 12.0
pull_requests: +26919
pull_request: https://github.com/python/cpython/pull/28529

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f4ccb79d52ee726d58bbb038ea98b4deec52001d by Victor Stinner in 
branch 'main':
bpo-43760: Document PyThreadState.use_tracing removal (GH-28527)
https://github.com/python/cpython/commit/f4ccb79d52ee726d58bbb038ea98b4deec52001d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40346] Add random.BaseRandom to ease implementation of subclasses

[Matt Bogosian]

Matt Bogosian  added the comment:

I landed here after investigating this surprising result:

  # test_case.py
  from random import Random
  from typing import Sequence, Union
  
  _RandSeed = Union[None, int, Sequence[int]]
  
  class MyRandom(Random):
def __init__(
  self,
  seed: _RandSeed = None,
):
  if seed is not None and not isinstance(seed, int):
seed = sum(seed)
  super().__init__(seed)
 
  MyRandom([1, 2])

Output:

  python ./test_case.py
  Traceback (most recent call last):
File "/…/./test_case.py", line 16, in 
  
  MyRandom([1, 2])
  TypeError: unhashable type: 'list'

In my observation, the Random class aspires to be an interface (and default 
implementation), but doesn't really live up to those aspirations. (See also 
https://github.com/python/typeshed/issues/6063.) I suspect nudging Random 
closer to its claims was the point of this proposal. I'm kind of sad it (or 
something like it) was rejected in favor of a process that will probably take 
years. Is there a reason not to do both, meaning heal what lives in the 
standard library now to live up to its own marketing *and* work toward a better 
interface in the future?

--
nosy: +posita

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: XML Considered Harmful

[Chris Angelico]

On Thu, Sep 23, 2021 at 10:55 PM Mats Wichmann  wrote:
>
> On 9/22/21 10:31, Dennis Lee Bieber wrote:
>
> >   If you control both the data generation and the data consumption,
> > finding some format  ...
>
> This is really the key.  I rant at people seeming to believe that csv is
> THE data interchange format, and it's about as bad as it gets at that,
> if you have a choice.  xml is noisy but at least (potentially)
> self-documenting, and ought to be able to recover from certain errors.
> The problem with csv is that a substantial chunk of the world seems to
> live inside Excel, and so data is commonly both generated in csv so it
> can be imported into excel and generated in csv as a result of exporting
> from excel, so the parts often are *not* in your control.
>
> Sigh.

The only people who think that CSV is *the* format are people who
habitually live in spreadsheets. People who move data around the
internet, from program to program, are much more likely to assume that
JSON is the sole format. Of course, there is no single ultimate data
interchange format, but JSON is a lot closer to one than CSV is.

(Or to be more precise: any such thing as a "single ultimate data
interchange format" will be so generic that it isn't enough to define
everything. For instance, "a stream of bytes" is a universal data
interchange format, but that's not ultimately a very useful claim.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Mats Wichmann]

On 9/22/21 10:31, Dennis Lee Bieber wrote:


If you control both the data generation and the data consumption,
finding some format  ...


This is really the key.  I rant at people seeming to believe that csv is 
THE data interchange format, and it's about as bad as it gets at that, 
if you have a choice.  xml is noisy but at least (potentially) 
self-documenting, and ought to be able to recover from certain errors. 
The problem with csv is that a substantial chunk of the world seems to 
live inside Excel, and so data is commonly both generated in csv so it 
can be imported into excel and generated in csv as a result of exporting 
from excel, so the parts often are *not* in your control.


Sigh.

--
https://mail.python.org/mailman/listinfo/python-list

[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There are several common idioms which do not work well with shared dictionaries.

1. Class attributes as defaults. If most instances of the class have the 
default value for some attribute, it can be set as the class attribute. It 
saves memory because most instances will have smaller dict. But if the first 
instance has such attribute, it cancels shared dict for all following instances 
without this attribute.

2. cached_property (and analogs). The cached instance attributes can be added 
in arbitrary order, canceling shared dict for this instance and maybe for all 
instances created later.

3. Some classes delete attributes in close() or __exit__() methods to avoid 
reference loops and to release resources earlier. Since shared dicts do not 
support deletion, such closed objects have now larger size than non-closed 
objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41629] __class__ not set defining 'X' as

[Marc Mueller]

Change by Marc Mueller :


--
nosy: +cdce8p

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Mark Shannon]

Mark Shannon  added the comment:

Raymond,

Only split dicts need the extra field.

Classes where many instances do not have exactly the same set of attributes may 
be more common than you think.
There are many reasons why some attributes may be added conditionally.

PR 28520 actually makes the dictionary code a bit simpler, as we don't need to 
maintain the invariant that value arrays cannot have holes. Maintaining an 
order is simple and cheap:

order = (order<<4) | insertion_index

There are pros and cons to both schemes: PR 28520 and the current 
implementation.

The problem with the current scheme is that it only works well for classes 
where all instances are initialized with exactly the same attributes, and in 
the same order.

The PR 28520 scheme can handle those cases where order and key set differ a 
bit, but has a maximum size of 16 before the dict must be combined.


We need as many instances as possible to have split dictionaries, to get 
https://github.com/faster-cpython/ideas/issues/72 working well as it will make 
the cost of not sharing even greater, relatively.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45256] Remove the usage of the C stack in Python to Python calls

[Mark Shannon]

Mark Shannon  added the comment:

I've trying to do this since about 2011 :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41203] Replace references to OS X in documentation with macOS

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you Patrick.

We had several PRs for fixing macOS references here and there, they were closed 
in favor of this larger PR.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44958] [sqlite3] only reset statements when needed

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Explicitly resetting statements when we're done with them removes the 
performance regression; SQLite works more efficient when we keep the number of 
non-reset statements low.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41203] Replace references to OS X in documentation with macOS

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset f1d5cdef57fea961646c52db7a60d1f110e0eaa6 by Miss Islington (bot) 
in branch '3.9':
bpo-41203: Replace Mac OS X and OS X with macOS (GH-28515) (GH-28524)
https://github.com/python/cpython/commit/f1d5cdef57fea961646c52db7a60d1f110e0eaa6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41203] Replace references to OS X in documentation with macOS

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 1493e1a40d04a048cce9c5080ec47478a4941054 by Miss Islington (bot) 
in branch '3.10':
bpo-41203: Replace Mac OS X and OS X with macOS (GH-28515) (GH-28523)
https://github.com/python/cpython/commit/1493e1a40d04a048cce9c5080ec47478a4941054


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42969] pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful

[STINNER Victor]

STINNER Victor  added the comment:

I'm not comfortable with PR 28525 which always hang threads which attempt to 
acquire the GIL after Python exited.

I would prefer to keep the current behavior by default, but give the ability to 
applications embedding Python to decide what to do.

With my Py_SetThreadExitCallback(func) idea, PyThread_exit_thread() would call 
func() and then pthread_exit(0). Applications can hang threads, log a message, 
call abort(), or whatever else.

I'm not comfortable with writing a portable function to "hang a thread". For 
example, I don't understand why PR 28525 processes Windows messages on hang 
threads.

Well, it's a complex problem :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29921] datetime validation is stricter in 3.6.1 than previous versions

[Patrick Decat]

Patrick Decat  added the comment:

pywin32 project has moved from sourceforge to github.

https://sourceforge.net/p/pywin32/bugs/748/ is now at 
https://github.com/mhammond/pywin32/issues/748

pywin32 issue is supposed to be resolved since pywin32 b222

See:

https://github.com/mhammond/pywin32/issues/748#issuecomment-359223029

https://github.com/mhammond/pywin32/commit/07202650d21e8fa7f3053ff1d4665363315cefce

https://github.com/mhammond/pywin32/blob/b222/CHANGES.txt#L24-L26

--
nosy: +Patrick Decat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45256] Remove the usage of the C stack in Python to Python calls

[Christian Tismer]

Christian Tismer  added the comment:

Hey guys, you know that you are about to implement the core idea of Stackless 
Python, right? :-D

--
nosy: +Christian.Tismer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41137] pdb uses the locale encoding for .pdbrc

[STINNER Victor]

STINNER Victor  added the comment:


New changeset ec7ffa4b5b262369f726a54e145e9c03eaeb4c1a by Victor Stinner in 
branch 'main':
bpo-41137: Reorganize What's New in Python 3.11 (GH-28518)
https://github.com/python/cpython/commit/ec7ffa4b5b262369f726a54e145e9c03eaeb4c1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45267] New install Python 3.9.7 install of Sphinx Document Generator fails

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The error in the attached file says that one of the dependencies could not be 
found, in particular the "packaging" library.

This doesn't look like a bug in CPython or pip, but more a general support 
question. I kindly ask you to visit one of the python forums or mail lists (for 
example https://discuss.python.org).

--
nosy: +ronaldoussoren
resolution:  -> not a bug
stage:  -> resolved
status: open -> pending
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45269] c_make_encoder() has uncovered error: "argument 1 must be dict or None"

[Nikita Sobolev]

New submission from Nikita Sobolev :

Looks like we never test error that `markers` are limited to `None` and `dict` 
types. Here: 
https://github.com/python/cpython/blob/main/Modules/_json.c#L1252-L1255

Coverage report: 
https://app.codecov.io/gh/python/cpython/blob/master/Modules/_json.c line: 1252

I will submit a unit test for it today.

Related: https://bugs.python.org/issue6986

--
components: Tests
messages: 402482
nosy: sobolevn
priority: normal
severity: normal
status: open
title: c_make_encoder() has uncovered error: "argument 1 must be dict or None"
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[STINNER Victor]

STINNER Victor  added the comment:

I created PR 28527 to document PyThreadState.use_tracing removal and explain 
how to port existing code to Python 3.10.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +26918
pull_request: https://github.com/python/cpython/pull/28527

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45116] Performance regression 3.10b1 and later on Windows: Py_DECREF() not inlined in PGO build

[neonene]

neonene  added the comment:

3.10rc2 Python/ceval.c
1306: #define DISPATCH() \
1307: { \
1308: if (trace_info.cframe.use_tracing OR_DTRACE_LINE OR_LLTRACE) { \
1309: goto tracing_dispatch; \

Among the 44 pgo-tests, only test_patma.TestTracing hits the condition above. 
On Windows, it seems that skipping it tightens the profile of PR28475 a bit. 
Additional tests such as test_threading(.ThreadTests.test_frame_tstate_tracing) 
might also cause some amount of variation or vice versa.

3.10rc2 x64 PGO: 1.00
+ PR28475 
  with TestTracing : 1.05x faster (slow  3, fast 46, same  9)
  without  : 1.06x faster (slow  5, fast 52, same  1)

  with TestTracing : 1.00
  without  : 1.01x faster (slow 19, fast 27, same 12)

(Details: PR28475_skip1test_bench.txt)


Does test_patma.TestTracing need training for match-case performance?

--
Added file: https://bugs.python.org/file50296/PR28475_skip1test_bench.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45268] use multiple "in" in one expression?

[Dennis Sweeney]

Dennis Sweeney  added the comment:

This is the expected behavior, documented here: 
https://docs.python.org/3/reference/expressions.html#comparisons


That page says:

* The comparison operators are  "<" | ">" | "==" | ">=" | "<=" | "!=" | 
"is" ["not"] | ["not"] "in"

* "Comparisons can be chained arbitrarily"

* "Note that a op1 b op2 c doesn’t imply any kind of comparison between a 
and c, so that, e.g., x < y > z is perfectly legal (though perhaps not pretty)."


So I'll close this for now.

I think it would be hard to change this behavior without introducing a needless 
backwards-incompatibility, but if you have a proposal, you could bring it up on 
the Python-Ideas mailing list.

--
nosy: +Dennis Sweeney
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com