[issue42238] Deprecate suspicious.py?

[Julien Palard]

Julien Palard  added the comment:


New changeset b878b3af0b3a9e3ab3ffcaf90a4066dfb3bb7cac by Julien Palard in 
branch 'main':
bpo-42238: [doc] moving from rstlint.py to sphinx-lint. (GH-31097)
https://github.com/python/cpython/commit/b878b3af0b3a9e3ab3ffcaf90a4066dfb3bb7cac


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45393] help() on operator precedence has confusing entries "await" "x" and "not" "x"

[Zackery Spytz]

Zackery Spytz  added the comment:

I have created a patch for this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45393] help() on operator precedence has confusing entries "await" "x" and "not" "x"

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 5.0 -> 6.0
pull_requests: +29415
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31246

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to solve the given problem?

[Christian Gollwitzer]

Am 10.02.22 um 07:40 schrieb NArshad:


Assume that there is a pattern of feeding for a special fish in a day (10 hours 
a day) as below:
150100303030202010  
  55
Today, the fish is fed in the second hour 60 unit instead of 100 unit 
Accidently. Implement some methods to distribute the remaining 40 unit in the 
rest of the day and propose the new patterns. Try to keep the distribution 
similar to the current feeding pattern.
Note: pay attention that the total feeding amounts should be fix in a day.


This is not a Python problem, it's a math problem and most probably a 
homework problem. Actually the question already tells you how to solve 
it. There are 40 units of fish-food left and you should distribute them 
proportionally to the rest of the day. Sum up the numbers from 3rd to 
last, add 40, and then distribute this to proportionally to each day.


You'll end up with fractinoal numbers in the general case, so you'll 
have to find a method to fairly distribute the units, if you wan to 
stick with integers.


You can also check out the various algorithms for distributing seats in 
a parliament, it is almost the same problem.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

How to solve the given problem?

[NArshad]

Assume that there is a pattern of feeding for a special fish in a day (10 hours 
a day) as below:
   150100303030202010   
 55
Today, the fish is fed in the second hour 60 unit instead of 100 unit 
Accidently. Implement some methods to distribute the remaining 40 unit in the 
rest of the day and propose the new patterns. Try to keep the distribution 
similar to the current feeding pattern. 
Note: pay attention that the total feeding amounts should be fix in a day.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21910] [doc] File protocol should document if writelines must handle generators sensibly

[Stanley]

Change by Stanley :


--
keywords: +patch
pull_requests: +29414
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31245

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Change by Raymond Hettinger :


Added file: https://bugs.python.org/file50615/instrument_issubset.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Change by Raymond Hettinger :


Removed file: https://bugs.python.org/file50614/instrument_issubset.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Change by Raymond Hettinger :


Removed file: https://bugs.python.org/file50613/instrument_issubset.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've run a few more experiments and this looks like a net win more often than 
not.  Go ahead and submit a PR so we can evaluate it further.

--
Added file: https://bugs.python.org/file50614/instrument_issubset.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Raymond Hettinger  added the comment:

We care more about the running speed than the memory usage.  Since sets only 
store pointers to data, they are typically smaller than the data they refer to.

I've attached some instrumentation code for running experiments to verify the 
workload under various scenarios.

--
Added file: https://bugs.python.org/file50613/instrument_issubset.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45923] Improve performance of sys.settracing based tools.

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher
nosy_count: 3.0 -> 4.0
pull_requests: +29413
pull_request: https://github.com/python/cpython/pull/31244

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Thanks Anthony for the report!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 9b23f8f78fdb0d7eba016616ae7a97abbfc65aa6 by Pablo Galindo Salgado 
in branch '3.10':
[3.10] bpo-46707: Avoid potential exponential backtracking in some syntax 
errors (GH-31241). (GH-31242)
https://github.com/python/cpython/commit/9b23f8f78fdb0d7eba016616ae7a97abbfc65aa6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Change by Raymond Hettinger :


--
versions:  -Python 3.10, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46705] Memory optimization for set.issubset

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +29412
pull_request: https://github.com/python/cpython/pull/31242

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset b71dc71905ab674ccaa4a56230d17a28f61c325c by Pablo Galindo Salgado 
in branch 'main':
bpo-46707: Avoid potential exponential backtracking in some syntax errors 
(GH-31241)
https://github.com/python/cpython/commit/b71dc71905ab674ccaa4a56230d17a28f61c325c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs

[Gregory Beauregard]

Gregory Beauregard  added the comment:

I wrote a PR that fixes the underlying issue here, but I'm leaving it as a 
draft while the discussion plays out. I think the stuff currently in the patch 
should be okay regardless of the discussion decision, because the underlying 
issue is that P.args and P.kwargs didn't pass typing._type_check, which is 
needed for reasons unrelated to Annotated (such as if they were stringified and 
had get_type_hints called on them).

If the result of the discussion is that we need to start supporting limitations 
on where Annotated is used, I'll add another PR that introduces support for 
this pattern in whatever locations that are decided it shouldn't be allowed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +29411
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31241

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[anthony shaw]

Change by anthony shaw :


--
versions: +Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46707] Parser hanging on stacked { tokens

[anthony shaw]

New submission from anthony shaw :

Providing an (invalid) input to the parser causes an exponentially-slow DoS to 
the Python executable in 3.10.

e.g.

python3.10 -c "{:"

takes ~2 seconds

python3.10 -c ":"

takes ~22 seconds

Tested this all the way up to 
d```{{{ef f():y

which took over an hour

--
components: Parser
keywords: 3.10regression
messages: 412972
nosy: anthonypjshaw, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Parser hanging on stacked { tokens
type: crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[MRAB]

On 2022-02-10 01:37, Jen Kris via Python-list wrote:

I'm using Python 3.8 so I tried your second choice:

pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

but pSents is 0x0.  pSentMod and pListItem are valid pointers.


'PyObject_CallFunction' looks like a good one to use:

"""PyObject* PyObject_CallFunction(PyObject *callable, const char 
*format, ...)


Call a callable Python object callable, with a variable number of C 
arguments. The C arguments are described using a Py_BuildValue() style 
format string. The format can be NULL, indicating that no arguments are 
provided.

"""

[snip]

What I do is add comments to keep track of what objects I have 
references to at each point and whether they are new references or could 
be NULL.


For example:

pName = PyUnicode_FromString("nltk.corpus");
//> pName+?

This means that 'pName' contains a reference, '+' means that it's a new 
reference, and '?' means that it could be NULL (usually due to an 
exception, but not always) so I need to check it.


Continuing in this vein:

pModule = PyImport_Import(pName);
//> pName+? pModule+?

pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
//> pName+? pModule+? pSubMod+?
pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
//> pName+? pModule+? pSubMod+? pFidMod+?
pSentMod = PyObject_GetAttrString(pSubMod, "sents");
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+?

pFileIds = PyObject_CallObject(pFidMod, 0);
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+?

pListItem = PyList_GetItem(pFileIds, listIndex);
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem?

pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem? pListStrE+?


As you can see, there's a lot of leaked references building up.

Note how after:

pListItem = PyList_GetItem(pFileIds, listIndex);

the addition is:

//> pListItem?

This means that 'pListItem' contains a borrowed (not new) reference, but 
could be NULL.


I find it easiest to DECREF as soon as I no longer need the reference 
and remove a name from the list as soon I no longer need it (and 
DECREFed where).


For example:

pName = PyUnicode_FromString("nltk.corpus");
//> pName+?
if (!pName)
goto error;
//> pName+
pModule = PyImport_Import(pName);
//> pName+ pModule+?
Py_DECREF(pName);
//> pModule+?
if (!pModule)
goto error;
//> pModule+

I find that doing this greatly reduces the chances of getting the 
reference counting wrong, and I can remove the comments once I've 
finished the function I'm writing.

--
https://mail.python.org/mailman/listinfo/python-list

[issue46704] Parser API not checking for null-terminator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The contract of that interface is to receive null-terminated strings, so I am a 
bit clueless on how this could receive a non-null terminated string.

Notice that calling PyRun_StringFlags with a non-null terminated string is out 
of contract.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46704] Parser API not checking for null-terminator

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg412970

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46704] Parser API not checking for null-terminator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The contract of that interface is to receive null-terminated strings, so I am a 
bit clueless on how this could receive a non-null terminated string.

Notice that calling CompileCode with a non-null terminated string is out of 
contract.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46704] Parser API not checking for null-terminator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Could you please provide an example input to reproduce this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46703] boolean operation issue (True == False == False)

[Eryk Sun]

Eryk Sun  added the comment:

> True == False == False is really True == False and False == False 
> wich is False and True which is False

Moreover, since the left-hand comparison is `True == False`, which evaluates to 
False, the right-hand comparison doesn't even get evaluated. 

In the following example, print(3) doesn't get called because the left-hand 
comparison, `None != None`, is False:

>>> print(1) != print(2) == print(3)
1
2
False

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[Jen Kris via Python-list]

I'll do that and post back tomorrow.  The office is closing and I have to leave 
now (I'm in Seattle).  Thanks again for your help.  


Feb 9, 2022, 17:40 by songofaca...@gmail.com:

> On Thu, Feb 10, 2022 at 10:37 AM Jen Kris  wrote:
>
>>
>> I'm using Python 3.8 so I tried your second choice:
>>
>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>
>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>
>
> It means exception happened.
> If you are writing Python/C function, return NULL (e.g. `if (pSents ==
> NULL) return NULL`)
> Then Python show the exception and traceback for you.
>
> -- 
> Inada Naoki  
>

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Avi Gross via Python-list]

Actually, you may want a dynamic page. Pages may sometimes be delivered from 
some cache along the way perhaps within your own company firewall if someone 
else recently accessed them.
Consider a page that returns the current time or like the following asks for an 
arithmetical calculation to add 5 and 6
https://search.yahoo.com/search?fr=mcafee=E211US1249G0=%3D5%2B6
Not a great example, and it may not work for you, but there likely is a site 
that can exist that accepts a request for some formatted calculation that you 
can specify dynamically. You can compare a date/time returned to being accurate 
within some seconds and asking for n*m can obviously be validated.
Of course, once such a ploy is well known, it can be intercepted and sent 
proper results and still fool you into assuming you had full internet access. 
So a site that you can send to and receive back using a cryptographic method 
might be better.



-Original Message-
From: Abdur-Rahmaan Janhangeer 
To: Python 
Sent: Wed, Feb 9, 2022 12:24 pm
Subject: Re: Best way to check if there is internet?

> This won't work if you're behind a captive portal: every URL you
try to get will return successfully, but the content will be
the captive portal page.

Yes agree a pretty common scenario if you are moving around ...
The solution is to test against a webpage you know the content
will always be the same. Thanks for pointing out!

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C API PyObject_Call segfaults with string

[Inada Naoki]

On Thu, Feb 10, 2022 at 10:37 AM Jen Kris  wrote:
>
> I'm using Python 3.8 so I tried your second choice:
>
> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>
> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>

It means exception happened.
If you are writing Python/C function, return NULL (e.g. `if (pSents ==
NULL) return NULL`)
Then Python show the exception and traceback for you.

-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46706] AxelRacer

[Dennis Sweeney]

New submission from Dennis Sweeney :

I'm closing this -- if you found a bug in Python, please be sure to describe 
thoroughly what bug you found, steps to reproduce the bug, and what behavior 
you expected.

--
nosy: +Dennis Sweeney
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[Jen Kris via Python-list]

I'm using Python 3.8 so I tried your second choice:

pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

but pSents is 0x0.  pSentMod and pListItem are valid pointers.  


Feb 9, 2022, 17:23 by songofaca...@gmail.com:

> // https://docs.python.org/3/c-api/call.html#c.PyObject_CallNoArgs
> // This function is only for one arg. Python >= 3.9 is required.
> pSents = PyObject_CallOneArg(pSentMod, pListItem);
>
> Or
>
> // https://docs.python.org/3/c-api/call.html#c.PyObject_CallFunctionObjArgs
> // This function can call function with multiple arguments. Can be
> used with Python <3.9 too.
> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>
> On Thu, Feb 10, 2022 at 10:15 AM Jen Kris  wrote:
>
>>
>> Right you are.  In that case should I use Py_BuildValue and convert to tuple 
>> (because it won't return a tuple for a one-arg), or should I just convert 
>> pListStr to tuple?  Thanks for your help.
>>
>>
>> Feb 9, 2022, 17:08 by songofaca...@gmail.com:
>>
>> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris  wrote:
>>
>>
>> Thanks for your reply.
>>
>> I eliminated the DECREF and now it doesn't segfault but it returns 0x0. Same 
>> when I substitute pListStrE for pListStr. pListStr contains the string 
>> representation of the fileid, so it seemed like the one to use. According to 
>> http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, 
>> PyBuildValue "builds a tuple only if its format string contains two or more 
>> format units" and that doc contains examples.
>>
>>
>> Yes, and PyObject_Call accept tuple, not str.
>>
>>
>> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>>
>>
>> Feb 9, 2022, 16:52 by songofaca...@gmail.com:
>>
>> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>>  wrote:
>>
>>
>> I have everything finished down to the last line (sentences = 
>> gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, 
>> but it segfaults. The fileid is a string -- the first fileid in this corpus 
>> is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>>
>> HERE.
>> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
>> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>>
>> Why do you encode pListStrE?
>> Why don't you use just pListStrE?
>>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>>
>> c_args must tuple, but you passed a unicode object here.
>> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
>> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
>> may not be constructed correctly. I also tried it with "O" and it doesn't 
>> segfault but it returns 0x0.
>>
>> I'm new to using the C API. Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> Bests,
>>
>> --
>> Inada Naoki 
>>
>>
>>
>> --
>> Inada Naoki 
>>
>
>
> -- 
> Inada Naoki  
>

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46705] Memory optimization for set.issubset

[Dennis Sweeney]

Change by Dennis Sweeney :


--
nosy: +rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46706] AxelRacer

[claude-alexandre cabana]

Change by claude-alexandre cabana :


--
components: Build
nosy: claudealexcabana
priority: normal
severity: normal
status: open
title: AxelRacer
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[Inada Naoki]

// https://docs.python.org/3/c-api/call.html#c.PyObject_CallNoArgs
// This function is only for one arg. Python >= 3.9 is required.
pSents = PyObject_CallOneArg(pSentMod, pListItem);

Or

// https://docs.python.org/3/c-api/call.html#c.PyObject_CallFunctionObjArgs
// This function can call function with multiple arguments. Can be
used with Python <3.9 too.
pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

On Thu, Feb 10, 2022 at 10:15 AM Jen Kris  wrote:
>
> Right you are.  In that case should I use Py_BuildValue and convert to tuple 
> (because it won't return a tuple for a one-arg), or should I just convert 
> pListStr to tuple?  Thanks for your help.
>
>
> Feb 9, 2022, 17:08 by songofaca...@gmail.com:
>
> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris  wrote:
>
>
> Thanks for your reply.
>
> I eliminated the DECREF and now it doesn't segfault but it returns 0x0. Same 
> when I substitute pListStrE for pListStr. pListStr contains the string 
> representation of the fileid, so it seemed like the one to use. According to 
> http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue 
> "builds a tuple only if its format string contains two or more format units" 
> and that doc contains examples.
>
>
> Yes, and PyObject_Call accept tuple, not str.
>
>
> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>
>
> Feb 9, 2022, 16:52 by songofaca...@gmail.com:
>
> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>  wrote:
>
>
> I have everything finished down to the last line (sentences = 
> gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, 
> but it segfaults. The fileid is a string -- the first fileid in this corpus 
> is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);
>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);
>
>
> Why do you encode pListStrE?
> Why don't you use just pListStrE?
>
> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
> may not be constructed correctly. I also tried it with "O" and it doesn't 
> segfault but it returns 0x0.
>
> I'm new to using the C API. Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> Bests,
>
> --
> Inada Naoki 
>
>
>
> --
> Inada Naoki 
>
>


-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46705] Memory optimization for set.issubset

[Jack Nguyen]

New submission from Jack Nguyen :

I noticed that the set.issubset cpython implementation casts its iterable 
argument to a set. In some cases, casting the whole iterable to a set is 
unnecessary (see https://bugs.python.org/issue18032). Although the latter 
suggestion is to perform early termination, my suggestion is to use the 
intersection instead.

# PyAnySet_Check coming from the cpython source code.
def issubset(self, other):
# Intersection suggestion:
if not PyAnySet_Check(other):
return len(self.intersection(other)) == len(self)
# Usual implementation for sets.
else:
return ...

The main advantage that this implementation has is its memory performance, 
using only O(min(len(self), len(other))) memory, since it never stores elements 
it does not need.

I'm assuming that set construction costs O(n) set.__contains__ calls. This 
implementation uses len(other) calls to self.__contains__ and tmp.__contains__, 
where tmp = set(other). The current implementation uses len(self) + len(other) 
calls to tmp.__contains__.

Thus, I suspect the current implementation only has a chance at running 
noticeably faster when len(self) << len(other), where it performs fewer calls 
to set.__contains__. This is, however, also where the proposed implementation 
has significantly superior memory performance.

--
components: Interpreter Core
messages: 412966
nosy: panda1200
priority: normal
severity: normal
status: open
title: Memory optimization for set.issubset
type: performance
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46704] Parser API not checking for null-terminator

[anthony shaw]

New submission from anthony shaw :

In tokenizer.c, the translate_newlines() function does a `strlen()` on the 
input string, if the string is not null-terminated, e.g. 
'\xbe' this leads to a heap-buffer-overflow. The overflow is not exploitable, 
but if there are further changes to the parser, it might be worth using a 
strlen() alternative, like strnlen().

static char *
translate_newlines(const char *s, int exec_input, struct tok_state *tok) {
int skip_next_lf = 0;
size_t needed_length = strlen(s) + 2, final_length;


This leads to a heap-buffer-overflow detected by ASAN in a simple reproducible 
example, calling PyRun_StringFlags() from the LLVM fuzzer:


fuzz_target(47084,0x11356f600) malloc: nano zone abandoned due to inability to 
preallocate reserved vm space.
Dictionary: 35 entries
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3034498392
INFO: Loaded 1 modules   (43 inline 8-bit counters): 43 [0x10a2b71e8, 
0x10a2b7213), 
INFO: Loaded 1 PC tables (43 PCs): 43 [0x10a2b7218,0x10a2b74c8), 
INFO:1 files found in ../Tests/fuzzing/corpus
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 
4096 bytes
=
==47084==ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x60203131 at pc 0x00010bd1d555 bp 0x7ff7b5da0590 sp 0x7ff7b5d9fd50
READ of size 2 at 0x60203131 thread T0
#0 0x10bd1d554 in wrap_strlen+0x184 
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x15554)
#1 0x10b12132b in translate_newlines+0x1b (Python:x86_64+0x5d32b)
#2 0x10b12071c in _PyParser_ASTFromString+0x1ac (Python:x86_64+0x5c71c)
#3 0x10b2f86de in PyRun_StringFlags+0x5e (Python:x86_64+0x2346de)
#4 0x10a25ec6b in CompileCode(char const*) fuzz_target.cpp:54
#5 0x10a25f247 in LLVMFuzzerTestOneInput fuzz_target.cpp:68
#6 0x10a27aff3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, 
unsigned long) FuzzerLoop.cpp:611
#7 0x10a27c3c4 in 
fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector >&) FuzzerLoop.cpp:804
#8 0x10a27c859 in fuzzer::Fuzzer::Loop(std::__1::vector >&) FuzzerLoop.cpp:857
#9 0x10a26aa5f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char 
const*, unsigned long)) FuzzerDriver.cpp:906
#10 0x10a298e42 in main FuzzerMain.cpp:20
#11 0x1134f44fd in start+0x1cd (dyld:x86_64+0x54fd)

0x60203131 is located 0 bytes to the right of 1-byte region 
[0x60203130,0x60203131)
allocated by thread T0 here:
#0 0x10bd58a0d in wrap__Znam+0x7d 
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x50a0d)
#1 0x10a27af02 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, 
unsigned long) FuzzerLoop.cpp:596
#2 0x10a27c3c4 in 
fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector >&) FuzzerLoop.cpp:804
#3 0x10a27c859 in fuzzer::Fuzzer::Loop(std::__1::vector >&) FuzzerLoop.cpp:857
#4 0x10a26aa5f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char 
const*, unsigned long)) FuzzerDriver.cpp:906
#5 0x10a298e42 in main FuzzerMain.cpp:20
#6 0x1134f44fd in start+0x1cd (dyld:x86_64+0x54fd)

SUMMARY: AddressSanitizer: heap-buffer-overflow 
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x15554) in wrap_strlen+0x184
Shadow bytes around the buggy address:
  0x1c0405d0: fa fa 02 fa fa fa 02 fa fa fa 02 fa fa fa 02 fa
  0x1c0405e0: fa fa 02 fa fa fa 02 fa fa fa 02 fa fa fa 02 fa
  0x1c0405f0: fa fa 03 fa fa fa 03 fa fa fa 03 fa fa fa 03 fa
  0x1c040600: fa fa 01 fa fa fa 01 fa fa fa 01 fa fa fa 01 fa
  0x1c040610: fa fa 00 00 fa fa 00 fa fa fa 00 fa fa fa 00 00
=>0x1c040620: fa fa 00 fa fa fa[01]fa fa fa fd fa fa fa fd fd
  0x1c040630: fa fa fd fa fa fa fd fa fa fa 00 fa fa fa 04 fa
  0x1c040640: fa fa 00 00 fa fa 01 fa fa fa 01 fa fa fa 01 fa
  0x1c040650: fa fa fd fa fa fa fd fa fa fa fd fd fa fa 01 fa
  0x1c040660: fa fa 00 00 fa fa 01 fa fa fa fd fa fa fa fd fa
  0x1c040670: fa fa 01 fa fa fa 06 fa fa fa 00 00 fa fa 06 fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:   00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:   fa
  Freed heap region:   fd
  Stack left redzone:  f1
  Stack mid redzone:   f2
  Stack right redzone: f3
  Stack after return:  f5
  Stack use after scope:   f8
  Global redzone:  f9
  Global init order:   f6
  Poisoned by user:f7
  Container overflow:  fc
  Array cookie:ac
  Intra object redzone:bb
  ASan internal:   fe
  Left alloca redzone: ca
  Right alloca redzone:cb
  Shadow gap:  cc
==47084==ABORTING
MS: 0 ; base unit: 


artifact_prefix='./'; Test unit written to 
./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
Base64: 
zsh: abort  ./fuzz_target -dict=../Tests/fuzzing/python.dict -only_ascii=1

--
components: Parser
messages: 

Re: C API PyObject_Call segfaults with string

[Jen Kris via Python-list]

Right you are.  In that case should I use Py_BuildValue and convert to tuple 
(because it won't return a tuple for a one-arg), or should I just convert 
pListStr to tuple?  Thanks for your help.  


Feb 9, 2022, 17:08 by songofaca...@gmail.com:

> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris  wrote:
>
>>
>> Thanks for your reply.
>>
>> I eliminated the DECREF and now it doesn't segfault but it returns 0x0.  
>> Same when I substitute pListStrE for pListStr.  pListStr contains the string 
>> representation of the fileid, so it seemed like the one to use.  According 
>> to  http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, 
>> PyBuildValue "builds a tuple only if its format string contains two or more 
>> format units" and that doc contains examples.
>>
>
> Yes, and PyObject_Call accept tuple, not str.
>
>
> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>
>>
>> Feb 9, 2022, 16:52 by songofaca...@gmail.com:
>>
>> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>>  wrote:
>>
>>
>> I have everything finished down to the last line (sentences = 
>> gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, 
>> but it segfaults. The fileid is a string -- the first fileid in this corpus 
>> is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>>
>> HERE.
>> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
>> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>>
>> Why do you encode pListStrE?
>> Why don't you use just pListStrE?
>>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>>
>> c_args must tuple, but you passed a unicode object here.
>> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
>> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
>> may not be constructed correctly. I also tried it with "O" and it doesn't 
>> segfault but it returns 0x0.
>>
>> I'm new to using the C API. Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> Bests,
>>
>> --
>> Inada Naoki 
>>
>
>
> -- 
> Inada Naoki  
>

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C API PyObject_Call segfaults with string

[Inada Naoki]

On Thu, Feb 10, 2022 at 10:05 AM Jen Kris  wrote:
>
> Thanks for your reply.
>
> I eliminated the DECREF and now it doesn't segfault but it returns 0x0.  Same 
> when I substitute pListStrE for pListStr.  pListStr contains the string 
> representation of the fileid, so it seemed like the one to use.  According to 
>  http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, 
> PyBuildValue "builds a tuple only if its format string contains two or more 
> format units" and that doc contains examples.
>

Yes, and PyObject_Call accept tuple, not str.


https://docs.python.org/3/c-api/call.html#c.PyObject_Call

>
> Feb 9, 2022, 16:52 by songofaca...@gmail.com:
>
> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>  wrote:
>
>
> I have everything finished down to the last line (sentences = 
> gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, 
> but it segfaults. The fileid is a string -- the first fileid in this corpus 
> is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);
>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);
>
>
> Why do you encode pListStrE?
> Why don't you use just pListStrE?
>
> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
> may not be constructed correctly. I also tried it with "O" and it doesn't 
> segfault but it returns 0x0.
>
> I'm new to using the C API. Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> Bests,
>
> --
> Inada Naoki 
>
>


-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46703] boolean operation issue (True == False == False)

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This is not a bug. Please check the docs on the ternary operator:

https://docs.python.org/3/reference/expressions.html#comparisons

In particular:

Comparisons can be chained arbitrarily, e.g., x < y <= z is equivalent to x < y 
and y <= z, except that y is evaluated only once (but in both cases z is not 
evaluated at all when x < y is found to be false).



THis means that

True == False == False is really True == False and False == False wich is False 
and True which is False

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46703] boolean operation issue (True == False == False)

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg412963

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46703] boolean operation issue (True == False == False)

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This is not a bug. Please check the docs on the ternary operator:

https://docs.python.org/3/reference/expressions.html#comparisons

In particular:

Comparisons can be chained arbitrarily, e.g., x < y <= z is equivalent to x < y 
and y <= z, except that y is evaluated only once (but in both cases z is not 
evaluated at all when x < y is found to be false).



THis means that

True == False == False is really True == False and True == False wich is False 
and False which is False

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[Jen Kris via Python-list]

Thanks for your reply.  

I eliminated the DECREF and now it doesn't segfault but it returns 0x0.  Same 
when I substitute pListStrE for pListStr.  pListStr contains the string 
representation of the fileid, so it seemed like the one to use.  According to  
http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue 
"builds a tuple only if its format string contains two or more format units" 
and that doc contains examples. 


Feb 9, 2022, 16:52 by songofaca...@gmail.com:

> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>  wrote:
>
>>
>> I have everything finished down to the last line (sentences = 
>> gutenberg.sents(fileid)) where I use  PyObject_Call to call gutenberg.sents, 
>> but it segfaults.  The fileid is a string -- the first fileid in this corpus 
>> is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>
> Why do you encode pListStrE?
> Why don't you use just pListStrE?
>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
>>  from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
>> may not be constructed correctly.  I also tried it with "O" and it doesn't 
>> segfault but it returns 0x0.
>>
>> I'm new to using the C API.  Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
> Bests,
>
> -- 
> Inada Naoki  
>

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46702] Specialize UNPACK_SEQUENCE

[Brandt Bucher]

Brandt Bucher  added the comment:

(I also plan on looking into an adaptive super-duper-instruction for 
UNPACK_SEQUENCE_TWO_TUPLE__STORE_FAST__STORE_FAST after the current PR has 
landed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46703] boolean operation issue (True == False == False)

[jung mo sohn]

New submission from jung mo sohn :

In python 3.6.8, 3.7.3, 3.7.4, 3.7.5, 3.7.12, 3.8.8 versions, the output is 
False as shown below.

Python 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print(True == False == False)
False

However, in the openjdk1.8 version, the output is "true" as shown below. 

public class Test {
public static void main(String[] args) throws Exception{
System.out.println(true == false == false);
}
}

> java Test
true

In my opinion, "True" seems to be correct.

--
components: Parser
messages: 412961
nosy: jmsohn.x, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: boolean operation issue (True == False == False)
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C API PyObject_Call segfaults with string

[Inada Naoki]

On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
 wrote:
>
> I have everything finished down to the last line (sentences = 
> gutenberg.sents(fileid)) where I use  PyObject_Call to call gutenberg.sents, 
> but it segfaults.  The fileid is a string -- the first fileid in this corpus 
> is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);

HERE.
PyBytes_AS_STRING() returns pointer in the pListStrE Object.
So Py_DECREF(pListStrE) makes pListStr a dangling pointer.

>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);

Why do you encode pListStrE?
Why don't you use just pListStrE?

> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>

c_args must tuple, but you passed a unicode object here.
Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue


> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x76e4e8d5 in _PyEval_EvalCodeWithName ()
>from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it 
> may not be constructed correctly.  I also tried it with "O" and it doesn't 
> segfault but it returns 0x0.
>
> I'm new to using the C API.  Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Bests,

-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

C API PyObject_Call segfaults with string

[Jen Kris via Python-list]

This is a follow-on to a question I asked yesterday, which was answered by 
MRAB.   I'm using the Python C API to load the Gutenberg corpus from the nltk 
library and iterate through the sentences.  The Python code I am trying to 
replicate is:

from nltk.corpus import gutenberg
for i, fileid in enumerate(gutenberg.fileids()):
    sentences = gutenberg.sents(fileid)
    etc

I have everything finished down to the last line (sentences = 
gutenberg.sents(fileid)) where I use  PyObject_Call to call gutenberg.sents, 
but it segfaults.  The fileid is a string -- the first fileid in this corpus is 
"austen-emma.txt."  

pName = PyUnicode_FromString("nltk.corpus");
pModule = PyImport_Import(pName);

pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
pSentMod = PyObject_GetAttrString(pSubMod, "sents");

pFileIds = PyObject_CallObject(pFidMod, 0);
pListItem = PyList_GetItem(pFileIds, listIndex);
pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
pListStr = PyBytes_AS_STRING(pListStrE);
Py_DECREF(pListStrE);

// sentences = gutenberg.sents(fileid)
PyObject *c_args = Py_BuildValue("s", pListStr);  
PyObject *NullPtr = 0;
pSents = PyObject_Call(pSentMod, c_args, NullPtr);

The final line segfaults:
Program received signal SIGSEGV, Segmentation fault.
0x76e4e8d5 in _PyEval_EvalCodeWithName ()
   from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0

My guess is the problem is in Py_BuildValue, which returns a pointer but it may 
not be constructed correctly.  I also tried it with "O" and it doesn't segfault 
but it returns 0x0. 

I'm new to using the C API.  Thanks for any help. 

Jen


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46702] Specialize UNPACK_SEQUENCE

[Brandt Bucher]

Change by Brandt Bucher :


--
keywords: +patch
pull_requests: +29410
pull_request: https://github.com/python/cpython/pull/31240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46702] Specialize UNPACK_SEQUENCE

[Brandt Bucher]

New submission from Brandt Bucher :

UNPACK_SEQUENCE already has fast paths for tuples and lists, which make up 
(literally) 99% of unpackings in the benchmark suite. What's more, two-element 
tuples make up about two-thirds of all unpackings (though I actually suspect 
it's even higher, since the unpack_sequence benchmark is definitely skewing the 
results towards 10-element lists and tuples).

These specializations are trivial to implement and result in a solid 1% 
improvement overall.

--
assignee: brandtbucher
components: Interpreter Core
messages: 412960
nosy: Mark.Shannon, brandtbucher
priority: normal
severity: normal
stage: patch review
status: open
title: Specialize UNPACK_SEQUENCE
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46691] sysconfig.get_platform() raises ValueError on macOS if '-arch' is present in CFLAGS but doesn't refer to the '-arch' compiler flag

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg412958

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46691] sysconfig.get_platform() raises ValueError on macOS if '-arch' is present in CFLAGS but doesn't refer to the '-arch' compiler flag

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report; I don't recall this coming up before. Of course, the 
easy workaround is to avoid using "-arch" in CFLAGS paths and the like, but we 
should fix this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46691] sysconfig.get_platform() raises ValueError on macOS if '-arch' is present in CFLAGS but doesn't refer to the '-arch' compiler flag

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report; I don't recall this coming up before. Of course, the 
easy workaround is to avoid using "arch" in CFLAGS paths and the like, but we 
should fix this.

--
components: +macOS
nosy: +ned.deily, ronaldoussoren
type: crash -> compile error
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36876] [subinterpreters] Global C variables are a problem

[Eric Snow]

Eric Snow  added the comment:


New changeset cb68788dcadf43b47292bab7816a5ed9efa69730 by Eric Snow in branch 
'main':
bpo-36876: Minor cleanup to c-analyzer "ignored" data.' (gh-31239)
https://github.com/python/cpython/commit/cb68788dcadf43b47292bab7816a5ed9efa69730


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

[Eric Snow]

Eric Snow  added the comment:

@Eddie, what can I do to push this forward?  FYI, in addition to the python-dev 
thread a few weeks back, I've brought the matter up with the steering council. 
[1]

Also, if we can get back to performance-neutral (currently at about 4% slower) 
then there would be a lot less controversy.  Even at 2% it may be enough.


[1] https://github.com/python/steering-council/issues/103

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46701] cannot use typographical quotation marks in bug description

[Eric V. Smith]

Eric V. Smith  added the comment:

Please report bug tracker bugs at https://github.com/python/bugs.python.org

Although to be honest I doubt this will be fixed, since we're moving to Github 
issues.

--
nosy: +eric.smith
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36876] [subinterpreters] Global C variables are a problem

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29409
pull_request: https://github.com/python/cpython/pull/31239

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46528] Simplify the VM's stack manipulations

[Brandt Bucher]

Change by Brandt Bucher :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46528] Simplify the VM's stack manipulations

[Brandt Bucher]

Brandt Bucher  added the comment:


New changeset 78ae4cc6dc949e8bc39fab25fea5efe983dc0ad1 by Brandt Bucher in 
branch 'main':
bpo-46528: Attempt SWAPs at compile-time (GH-30970)
https://github.com/python/cpython/commit/78ae4cc6dc949e8bc39fab25fea5efe983dc0ad1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs

[Gregory Beauregard]

Change by Gregory Beauregard :


--
keywords: +patch
pull_requests: +29408
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31238

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46701] cannot use typographical quotation marks in bug description

[mirabilos]

New submission from mirabilos :

When trying to use typographical quotation marks (U+201C, U+201D) in the 
Comment field trying to submit a bug here, I get a red-background error message 
saying:

Error: 'utf8' codec can't decode bytes in position 198-199: invalid 
continuation byte

--
messages: 412953
nosy: mirabilos
priority: normal
severity: normal
status: open
title: cannot use typographical quotation marks in bug description

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46700] wrong nomenclature (options vs. arguments) in argparse

[mirabilos]

New submission from mirabilos :

The argparse documentation and tutorial as well as its default option groups 
speak of "positional arguments" and "optional arguments". These are not used 
correctly, though.

Elements of the argument vector (past item #0) are distinguished as options and 
(positional) arguments.

Options are either flags (ls "-l", cmd "/c") or GNU long options ("--help"). 
They are usually optional ("[-h]") but may be mandatory (such as -o/-i/-p for 
cpio(1)). They may have option arguments (cpio(1) "-H format").

Arguments (also called positional arguments) may be mandatory ("file") or 
optional ("[file]"). They are also called operands (mostly in POSIX, not very 
common).

The argparse documentation confused the hell out of me at first because I only 
saw argument documentation and could not find option documentation…

--
components: Library (Lib)
messages: 412952
nosy: mirabilos
priority: normal
severity: normal
status: open
title: wrong nomenclature (options vs. arguments) in argparse
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46693] dataclass generated __str__ does not use overridden member __str__

[Eric V. Smith]

Eric V. Smith  added the comment:

I'll close it.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46697] _ctypes_simple_instance returns inverted logic

[Eryk Sun]

Change by Eryk Sun :


--
stage:  -> patch review
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46697] _ctypes_simple_instance returns inverted logic

[hydroflask]

hydroflask  added the comment:

I place that patch into the public domain, I claim no ownership over that 
patch. The patch is attached purely for demonstration purposes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46697] _ctypes_simple_instance returns inverted logic

[hydroflask]

New submission from hydroflask :

`_ctypes_simple_instance` in _ctypes.c returns the opposite logic of what its 
documentation claims. It is supposed to return true when the argument (a type 
object) is a direct subclass of `Simple_Type` (`_SimpleCData` in Python code). 
However it returns false instead.

No bugs have manifested from this because all of the call sites ( 
`callproc.c::GetResult`, 
`callbacks.c::_CallPythonObject`,_`ctypes.c::PyCData_get`, 
`_ctypes.c::Simple_from_outparm`) invert the return value of this function. The 
last example, `ctypes.c::Simple_from_outparm` only calls `Simple_get_value()` 
when `_ctypes_simple_instance` returns false, which makes sense because 
otherwise the invocation of `_ctypes.c::Simple_from_outparm()` could trigger an 
assertion error.

This is not just simply an issue of inverted logic because the logic isn't 
inverted in all cases. In `_ctypes_simple_instance` in the case when 
`PyCSimpleTypeObject_Check(type)` returns false, if this were supposed to be 
perfect inverted logic then the whole routine would return 1 (True) not 0. 
Fortunately, due to the way the code is structured, I don't think there is a 
case when `PyCSimpleTypeObject_Check(type)` returns false so the incorrect case 
where it returns a constant 0 is effectively dead code.

I have compiled a version of Python with the attached patch and run "make test" 
with no issues.

--
components: ctypes
files: _ctypes_simple_instance_inverted.patch
keywords: patch
messages: 412947
nosy: hydroflask
priority: normal
severity: normal
status: open
title: _ctypes_simple_instance returns inverted logic
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9
Added file: 
https://bugs.python.org/file50612/_ctypes_simple_instance_inverted.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43532] Add keyword-only fields to dataclasses

[Eric V. Smith]

Eric V. Smith  added the comment:

Thanks, Henry Schreiner!

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46696] socketmodule add Linux SO_INCOMING_CPU constant

[David CARLIER]

New submission from David CARLIER :

Adding Linux's SO_INCOMING_CPU constant for setsockopt.

--
components: +Library (Lib)
keywords: +patch
message_count: None -> 1.0
pull_requests: +29407
stage:  -> patch review
title: socketmodule add Linux SO_INCOMING_CPU constasn -> socketmodule add 
Linux SO_INCOMING_CPU constant
type:  -> enhancement
pull_request: https://github.com/python/cpython/pull/31237

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46696] socketmodule add Linux SO_INCOMING_CPU constasn

[David CARLIER]

Change by David CARLIER :


--
nosy: devnexen
priority: normal
severity: normal
status: open
title: socketmodule add Linux SO_INCOMING_CPU constasn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43532] Add keyword-only fields to dataclasses

[miss-islington]

miss-islington  added the comment:


New changeset 7445949a4399ab19fbdd5a0b0aca53a690c3251b by Miss Islington (bot) 
in branch '3.10':
bpo-43532: add version added to KW_ONLY (GH-31235)
https://github.com/python/cpython/commit/7445949a4399ab19fbdd5a0b0aca53a690c3251b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43532] Add keyword-only fields to dataclasses

[miss-islington]

Change by miss-islington :


--
pull_requests: +29406
pull_request: https://github.com/python/cpython/pull/31236

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43532] Add keyword-only fields to dataclasses

[miss-islington]

miss-islington  added the comment:


New changeset 5a3f97291eea96037cceee097ebc00bba44bc9ed by Henry Schreiner in 
branch 'main':
bpo-43532: add version added to KW_ONLY (GH-31235)
https://github.com/python/cpython/commit/5a3f97291eea96037cceee097ebc00bba44bc9ed


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46323] Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c

[hydroflask]

hydroflask  added the comment:

For reference, here are MSDN, Linux, OpenBSD, and GCC docs on alloca:

https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/alloca?view=msvc-170

https://www.man7.org/linux/man-pages/man3/alloca.3.html

https://man.openbsd.org/alloca.3

https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index-_002a_005f_005fbuiltin_005falloca

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46323] Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c

[STINNER Victor]

STINNER Victor  added the comment:

I failed to find the doc about alloca(0). It seems like the existing 
_ctypes_callproc() function *can* call alloca(0) if argtuple is empty and pIunk 
is 0 (pIunk is specific to Windows).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46323] Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c

[hydroflask]

hydroflask  added the comment:

@corona10

I really hope I am not being annoying at this point :) One final nit, in this 
line:

https://github.com/python/cpython/pull/31224/files#diff-706e65ee28911740bf638707e19578b8182e57c6a8a9a4a91105d825f95a139dR168

You do not have to check if nargs > 0. alloca() can handle the case when nargs 
== 0. The usage of alloca() in callproc.c does not check for nargs > 0 either.

Otherwise thanks for this enhancement!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43532] Add keyword-only fields to dataclasses

[Henry Schreiner]

Change by Henry Schreiner :


--
nosy: +Henry Schreiner
nosy_count: 9.0 -> 10.0
pull_requests: +29405
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31235

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Unpacking lists in a f string

[Kirill Ratkin]

Hi.

Try this:

f"foo {','.join([f'{a} {b}' for a,b in list(zip(l1,l2))])} bar"

09.02.2022 21:13, Paulo da Silva пишет:

Às 02:17 de 09/02/22, Paulo da Silva escreveu:

Hi!

Let's say I have two lists of equal length but with a variable number 
of elements. For ex.:


l1=['a','b','c']
l2=['j','k','l']

I want to build a string like this
"foo a j, b k, c l bar"

Is it possible to achieve this with f strings or any other 
simple/efficient way?


Thanks for any help/comments.


Thank you for your responses.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How do you log in your projects?

[Martin Di Paola]

- On a line per line basis? on a function/method basis?


In general I prefer logging line by line instead per function.

It is easy to add a bunch of decorators to the functions and get the 
logs of all the program but I most of the time I end up with very 
confusing logs.


There are exceptions, yes, but I prefer the line by line where the log 
should explain what is doing the code.



- Which kind of variable contents do you write into your logfiles?
- How do you decide, which kind of log message goes into which level?
- How do you prevent logging cluttering your actual code?


These three comes to the same answer: I think on whom is going to read 
the logs.


If the logs are meant to be read by my users I log high level messages,
specially before parts that can take a while (like the classic 
"Loading...").


If I log variables, those must be the ones set by the users so he/she 
can understand how he/she is controlling the behaviour of the program.


For exceptions I print the message but not the traceback. Across the 
code tag some important functions to put an extra message that will 
enhance the final message printed to the user.


https://github.com/byexamples/byexample/blob/master/byexample/common.py#L192-L238

For example:

for example in examples:
with enhance_exceptions(example, ...):
foo()

So if an exception is raised by foo(), enhance_exceptions() will attach 
to it useful information for the user from the example variable.


In the main, then I do the pretty print
https://github.com/byexamples/byexample/blob/master/byexample/byexample.py#L17-L22

If the user of the logs is me or any other developer I write more debugging 
stuff.

My approach is to not log anything and when I have to debug something 
I use a debugger + some prints. When the issue is fixed I review which 
prints would be super useful and I turn them into logs and the rest is 
deleted.



On Tue, Feb 08, 2022 at 09:40:07PM +0100, Marco Sulla wrote:

These are a lot of questions. I hope we're not off topic.
I don't know if mine are best practices. I can tell what I try to do.

On Tue, 8 Feb 2022 at 15:15, Lars Liedtke  wrote:

- On a line per line basis? on a function/method basis?


I usually log the start and end of functions. I could also log inside
a branch or in other parts of the function/method.


- Do you use decorators to mark beginnings and ends of methods/functions
in log files?


No, since I put the function parameters in the first log. But I think
that such a decorator it's not bad.


- Which kind of variable contents do you write into your logfiles? Of
course you shouldn't leak secrets...


Well, all the data that is useful to understand what the code is
doing. It's better to repeat the essential data to identify a specific
call in all the logs of the function, so if it is called
simultaneously by more clients you can distinguish them


- How do you decide, which kind of log message goes into which level?


It depends on the importance, the verbosity and the occurrences of the logs.


- How do you prevent logging cluttering your actual code?


I have the opposite problem, I should log more. So I can't answer your question.
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

[issue46528] Simplify the VM's stack manipulations

[Brandt Bucher]

Brandt Bucher  added the comment:


New changeset 46328d8ae6529db916ccaabb9247fb0327ce0c1e by Brandt Bucher in 
branch 'main':
bpo-46528: Check PyMem_Malloc for NULL (GH-30998)
https://github.com/python/cpython/commit/46328d8ae6529db916ccaabb9247fb0327ce0c1e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'

[Dennis Lee Bieber]

On Tue, 8 Feb 2022 23:46:15 -0800 (PST), NArshad 
declaimed the following:

>When I enter data using Tkinter form in an Excel file when the excel file is 
>closed there is no error but when I enter data using Tkinter form when the 
>excel is already open following error comes:
>


>PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
>

>What to do to correct this error? I have already searched on google search 
>many times but no solution was found.

Well -- at the basic level... Make sure that only ONE user/application
has access to the file at any given moment.

Excel (and spreadsheets opened by it) are single-user applications for
a reason -- to prevent the data in the file from being changed without the
application knowing about it. They are not databases designed to allow
multiple users to make changes.

YOU will have to implement transaction control over the file. At the
rudimentary level (since you can't change how Excel itself operates)
whenever you intend to save changes to the data you have to notify any user
that has the file open that they need to close/save the file; when that has
been done you have to reread the file (because they may have made changes
to the contents that you haven't seen yet), reapply any of your changes (if
still valid conditions), then save the file. You can then notify the other
users that the file is updated and can be opened by them. If you don't
reread/validate the data before applying your changes, you would wipe out
any changes made by others.

Now, if you have total control over the applications that will access
the file (which essentially means: NOBODY will use Excel itself) you could
write a client/server scheme. In this scheme you would have one process
(the server) as the only program that does anything with the spreadsheet
file and its contents. Your client programs would connect (TCP sockets most
likely) to the server process and send it "commands" (something to be
defined/documented is an interface control document); the server would
parse the commands and implement changes to the file data, and/or return
any requested data to the client. That, at least avoids the file level
conflicts. You still have to figure out how to handle the case where two
clients try to update the same record (record level locking). 

One possibility is to have every record contain a time-stamp of the
last change -- and I'm going to assume the "commands" are record based, not
cell based -- and the command protocol for update sends back the time-stamp
originally read; the server code would compare the time-stamp with what
that record has in the file -- if they are the same, update the entire
record including a new time-stamp; if different, return a "conflict" status
with the current state of the record (with time-stamp) -- the client can
then compare the new record with the stale one, make any changes, and retry
the update. NOTE: this takes care of single record conflicts, but will help
if a "transaction" has to update multiple records since there is no history
to allow "all or none succeed" logic to be implemented.

"Commands"
READREC 
returns status (no such record) and (if valid) 
specified
record including last time-stamp
WRITEREC  
returns status (update conflict and current 
record contents
with time-stamp); if recno is unused, can pass 0 for timestamp to write new
record; upon write the time-stamp is updated
UPDATE
basically same as WRITEREC
FIND  
returns list of recno for records matching the 
search (not
the records themselves)


A proper client/server database handles most of the problem of keeping
the data (file) uncorrupted... Though clients attempting to update the same
record will have one succeed, the other get an exception -- which would be
handled by: rereading the record(s) one attempted to update, confirming the
conditions for the update are still valid, and reapplying the update.
Multiple record updates within a transaction are possible.


-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unpacking lists in a f string

[Paulo da Silva]

Às 02:17 de 09/02/22, Paulo da Silva escreveu:

Hi!

Let's say I have two lists of equal length but with a variable number of 
elements. For ex.:


l1=['a','b','c']
l2=['j','k','l']

I want to build a string like this
"foo a j, b k, c l bar"

Is it possible to achieve this with f strings or any other 
simple/efficient way?


Thanks for any help/comments.


Thank you for your responses.
--
https://mail.python.org/mailman/listinfo/python-list

Re: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'

[MRAB]

On 2022-02-09 12:45, Christian Gollwitzer wrote:

Am 09.02.22 um 08:46 schrieb NArshad:

When I enter data using Tkinter form in an Excel file when the excel file is 
closed there is no error but when I enter data using Tkinter form when the 
excel is already open following error comes:



PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'



What to do to correct this error? I have already searched on google search many 
times but no solution was found.


It's impossible. Excel locks the file deliberately when it is open, so
that you can't overwrite it from a different program. Otherwise, the
file could become inconsistent.

It's the same the other way too; you can't open the file in Excel while 
Python has it open.



The correct way to handle it in the GUI is to tell the user via a
message box that the file is open and can't be written.

An alternative to writing the file directly would be that you remote
control Excel; I think it provides a DDE API:
https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9


--
https://mail.python.org/mailman/listinfo/python-list

[issue46694] isdigit/isnumeric vs int()

[Mark Dickinson]

Change by Mark Dickinson :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46694] isdigit/isnumeric vs int()

[Mark Dickinson]

Mark Dickinson  added the comment:

This is by design: int looks for characters with the Unicode Decimal (De) 
numeric type, corresponding to str.isdecimal(), rather than for the Digit (Di) 
or Numeric (Nu) numeric types.

>>> "²".isdecimal()
False

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46072] Unify handling of stats in the CPython VM

[Mark Shannon]

Change by Mark Shannon :


--
pull_requests: +29404
pull_request: https://github.com/python/cpython/pull/31234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

We discussed that and found that a lot of errors are ignored during interning 
anyway.

But it's not too late to change if you want to (sending a PR would probably be 
quicker than arguing :-).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46323] Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset d18120cd67b4297f78bfc9bf7b36774cf0bf15f2 by Dong-hee Na in branch 
'main':
bpo-46323: Reduce stack usage of ctypes python callback function. (GH-31224)
https://github.com/python/cpython/commit/d18120cd67b4297f78bfc9bf7b36774cf0bf15f2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46695] _io_TextIOWrapper_reconfigure_impl errors out too early

[mirabilos]

New submission from mirabilos :

The following is not possible:

with open('/tmp/x.ssv', 'r', newline='\n') as f:
f.readline()
# imagine a library call boundary here
if hasattr(f, 'reconfigure'):
f.reconfigure(newline='\n')

The .reconfigure() call would not do anything, but it errors out nevertheless, 
simply because it is called (from reading the 
_io_TextIOWrapper_reconfigure_impl code in Modules/_io/textio.c).

Unfortunately, I *have* to call this in my library because I have to rely on 
“newline='\n'” behaviour (the hasattr avoids erroring out on binary streams), 
and the normal behaviour of erroring out if it’s too late to change is also 
good for me.

But the behaviour of erroring out if called at all when anything has already 
been read is a problem. This can easily be solved without breaking backwards 
compatibility, as the operation is a nop.

To clarify: I wish for…

with open('/tmp/x.ssv', 'r', newline='\n') as f:
f.readline()
# imagine a library call boundary here
if hasattr(f, 'reconfigure'):
f.reconfigure(newline='\n')

… to work, but for…

with open('/tmp/x.ssv', 'r') as f:
f.readline()
# imagine a library call boundary here
if hasattr(f, 'reconfigure'):
f.reconfigure(newline='\n')

… (line 1 is the only changed one) to continue to error out.

--
components: IO
messages: 412935
nosy: mirabilos
priority: normal
severity: normal
status: open
title: _io_TextIOWrapper_reconfigure_impl errors out too early
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46694] isdigit/isnumeric vs int()

[Nonsense]

New submission from Nonsense :

When typing in "²".isdigit() or "²".isnumeric() it gives True
but when typing in int("²") it errors out:
ValueError: invalid literal for int() with base 10: '²'

--
components: Interpreter Core
messages: 412934
nosy: smtplukas.tanner.test
priority: normal
severity: normal
status: open
title: isdigit/isnumeric vs int()
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45413] Add install scheme for virtual environments

[Filipe Laíns]

Filipe Laíns  added the comment:

I don't think the proposal is incompatible with what I discussed.

I haven't been super clear on my opinions on the implementation, so let me try 
to clarify them.

- I think that we should use a static scheme, accessible on all platforms.
- If this scheme needs to be independently defined for each platform, we should 
have different variants, available on all platforms, but still keep a generic 
named one, as an alias to the platform specific scheme
- We should not be re-using/aliasing existing schemes, particularly ones that 
are prone to downstream patching

So, my proposal would be to define a single static scheme, and changing the 
interpreter path initialization logic to hardcode its paths when on virtual 
environments.
If this presents any issue, and requires the scheme to be different for 
different platforms, we should add platform specific schemes and make the main 
one an alias to the correct scheme.

Hopefully that clarifies things up a bit. We should sort it out as soon as 
possible and update the PR, I don't think the PR as-is is the best approach.

What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Correct way to setup a package with both compiled C code and Python code?

[Dieter Maurer]

Christian Gollwitzer wrote at 2022-2-8 22:43 +0100:
>Am 08.02.22 um 18:57 schrieb Dieter Maurer:
>> Christian Gollwitzer wrote at 2022-2-7 20:33 +0100:
>>> we've developed a Python pacakge which consists of both a compiled
>>> extension module and some helper functions in Python. Is there a
>>> tutorial on how to package such an extension?
>>
>> Look at "https://package.python.org;,
>> especially 
>> "https://packaging.python.org/en/latest/guides/packaging-binary-extensions/;.
>
>Thank you, but that page is more like a high-level description, it talks
>a lot about the differences between C code and Python and how it can be
>combined using SWIG etc, but I already have a working extension.

Packaging and distribution was originally handled by `distutils`;
other tools, e.g. `setuptools`, extended `distutils` and have
gained wide spread acceptance. As a consequence, `distutils`
has been deprecated in favor of those other tools.
I assume the use of `setuptools` below.

Extensions are described via the `ext_modules` parameter of
the `setup` function. Its value is a sequence of
`setuptools.Extension` instances.

As of Python 3.11, `setuptools.Extension` is still a minor enhancement
of `distutils.extension.Extension`. This is documented via
docstrings in its source.
Read this documentation and come back if questions remain.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45863] tarfile zeroes ustar header fields unnecessarily

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Joshua Root for the bug report and the fix!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45863] tarfile zeroes ustar header fields unnecessarily

[miss-islington]

miss-islington  added the comment:


New changeset b0517a12179a9fe444ca6ce1f31c307883e81c0c by Miss Islington (bot) 
in branch '3.9':
bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693)
https://github.com/python/cpython/commit/b0517a12179a9fe444ca6ce1f31c307883e81c0c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45863] tarfile zeroes ustar header fields unnecessarily

[miss-islington]

miss-islington  added the comment:


New changeset 42f87d435ebe4f2834c8fd0b4ce38f0b1627f3ed by Miss Islington (bot) 
in branch '3.10':
bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693)
https://github.com/python/cpython/commit/42f87d435ebe4f2834c8fd0b4ce38f0b1627f3ed


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46693] dataclass generated __str__ does not use overridden member __str__

[Bruce Eckel]

Bruce Eckel  added the comment:

Oops. That does in fact work. How do I remove the bug report?

*Bruce Eckel*
HappyPathProgramming.com
SummerTechForum.com
MindViewLLC.com
Blog: BruceEckel.com
EvolveWork.co
WinterTechForum.com 
OnJava8.com 
www.AtomicKotlin.com
Reinventing-Business.com 

On Wed, Feb 9, 2022 at 10:20 AM Eric V. Smith 
wrote:

>
> Eric V. Smith  added the comment:
>
> I believe dataclasses uses repr() of the members, not str(). Can you try
> using specifying __repr__ in Teacup? Just __repr__ = __str__ should work.
>
> --
> nosy: +eric.smith
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Abdur-Rahmaan Janhangeer]

> This won't work if you're behind a captive portal: every URL you
try to get will return successfully, but the content will be
the captive portal page.

Yes agree a pretty common scenario if you are moving around ...
The solution is to test against a webpage you know the content
will always be the same. Thanks for pointing out!

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Abdur-Rahmaan Janhangeer]

>  Ah, but WHEN do those browsers report that? When attempting to connect
to whatever the default "home" page has been set to? (Mine is configured to
use https://www.google.com as the default page -- if my router is down,
obviously the browser will time-out waiting for a response from Google, and
report "no network").

Yes you are right ... lol i misread the error i guess

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46693] dataclass generated __str__ does not use overridden member __str__

[Eric V. Smith]

Eric V. Smith  added the comment:

I believe dataclasses uses repr() of the members, not str(). Can you try using 
specifying __repr__ in Teacup? Just __repr__ = __str__ should work.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46693] dataclass generated __str__ does not use overridden member __str__

[Eric V. Smith]

Change by Eric V. Smith :


--
assignee:  -> eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com