date:20240624

[Python-checkins] [3.13] gh-120683: Fix an error in logging.LogRecord timestamp (GH-120709) (GH-120933)

2024-06-24 Thread serhiy-storchaka

https://github.com/python/cpython/commit/544a47212b92f52ceffbd50275c6b0c57a446a98
commit: 544a47212b92f52ceffbd50275c6b0c57a446a98
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: serhiy-storchaka 
date: 2024-06-24T07:14:26Z
summary:

[3.13] gh-120683: Fix an error in logging.LogRecord timestamp (GH-120709) 
(GH-120933)

The integer part of the timestamp can be rounded up, while the millisecond
calculation truncates, causing the log timestamp to be wrong by up to 999 ms
(affected roughly 1 in 8 million timestamps).
(cherry picked from commit 1500a23f33f5a6d052ff1ef6383d9839928b8ff1)

Co-authored-by: Serhiy Storchaka 

files:
A Misc/NEWS.d/next/Library/2024-06-18-19-18-10.gh-issue-120683.xmRez7.rst
M Lib/logging/__init__.py
M Lib/test/test_logging.py

diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
index 174b37c0ab305b..3f4144226b40ec 100644
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -340,11 +340,14 @@ def __init__(self, name, level, pathname, lineno,
 self.lineno = lineno
 self.funcName = func
 self.created = ct / 1e9  # ns to float seconds
-
 # Get the number of whole milliseconds (0-999) in the fractional part 
of seconds.
 # Eg: 1_677_903_920_999_998_503 ns --> 999_998_503 ns--> 999 ms
 # Convert to float by adding 0.0 for historical reasons. See gh-89047
 self.msecs = (ct % 1_000_000_000) // 1_000_000 + 0.0
+if self.msecs == 999.0 and int(self.created) != ct // 1_000_000_000:
+# ns -> sec conversion can round up, e.g:
+# 1_677_903_920_999_999_900 ns --> 1_677_903_921.0 sec
+self.msecs = 0.0
 
 self.relativeCreated = (ct - _startTime) / 1e6
 if logThreads:
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
index 504862ad53395e..c78e76d16ae5cf 100644
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -4648,13 +4648,18 @@ def 
test_msecs_has_no_floating_point_precision_loss(self):
 (1_677_902_297_100_000_000, 100.0),  # exactly 100ms
 (1_677_903_920_999_998_503, 999.0),  # check truncating doesn't 
round
 (1_677_903_920_000_998_503, 0.0),  # check truncating doesn't round
+(1_677_903_920_999_999_900, 0.0), # check rounding up
 )
 for ns, want in tests:
 with patch('time.time_ns') as patched_ns:
 patched_ns.return_value = ns
 record = logging.makeLogRecord({'msg': 'test'})
-self.assertEqual(record.msecs, want)
-self.assertEqual(record.created, ns / 1e9)
+with self.subTest(ns):
+self.assertEqual(record.msecs, want)
+self.assertEqual(record.created, ns / 1e9)
+self.assertAlmostEqual(record.created - int(record.created),
+   record.msecs / 1e3,
+   delta=1e-3)
 
 def test_relativeCreated_has_higher_precision(self):
 # See issue gh-102402.
diff --git 
a/Misc/NEWS.d/next/Library/2024-06-18-19-18-10.gh-issue-120683.xmRez7.rst 
b/Misc/NEWS.d/next/Library/2024-06-18-19-18-10.gh-issue-120683.xmRez7.rst
new file mode 100644
index 00..50fc9279e4bad1
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-06-18-19-18-10.gh-issue-120683.xmRez7.rst
@@ -0,0 +1,4 @@
+Fix an error in :class:`logging.LogRecord`, when the integer part of the
+timestamp is rounded up, while the millisecond calculation truncates,
+causing the log timestamp to be wrong by up to 999 ms (affected roughly 1 in
+8 million timestamps).

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (#112191)

2024-06-24 Thread kumaraditya303

https://github.com/python/cpython/commit/fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a
commit: fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a
branch: main
author: Alek Kowalczyk 
committer: kumaraditya303 
date: 2024-06-24T08:35:02Z
summary:

gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and 
implications. (#112191)

Co-authored-by: Guido van Rossum 
Co-authored-by: Kumar Aditya 
Co-authored-by: Carol Willing 

files:
M Doc/library/asyncio-eventloop.rst

diff --git a/Doc/library/asyncio-eventloop.rst 
b/Doc/library/asyncio-eventloop.rst
index 374e789e91e790..1d79f78e8e1b67 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -1155,6 +1155,14 @@ DNS
 
Asynchronous version of :meth:`socket.getnameinfo`.
 
+.. note::
+   Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous
+   versions through the loop's default thread pool executor.
+   When this executor is saturated, these methods may experience delays,
+   which higher-level networking libraries may report as increased timeouts.
+   To mitigate this, consider using a custom executor for other user tasks,
+   or setting a default executor with a larger number of workers.
+
 .. versionchanged:: 3.7
Both *getaddrinfo* and *getnameinfo* methods were always documented
to return a coroutine, but prior to Python 3.7 they were, in fact,

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.12] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120936)

2024-06-24 Thread kumaraditya303

https://github.com/python/cpython/commit/3afb856dda9fed82e3d2f75fc54d063d26a41160
commit: 3afb856dda9fed82e3d2f75fc54d063d26a41160
branch: 3.12
author: Miss Islington (bot) <[email protected]>
committer: kumaraditya303 
date: 2024-06-24T08:45:46Z
summary:

[3.12] gh-112169: Documented getaddrinfo/getnameinfo default loop executor 
usage and implications. (GH-112191) (#120936)

gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and 
implications. (GH-112191)
(cherry picked from commit fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a)

Co-authored-by: Alek Kowalczyk 
Co-authored-by: Guido van Rossum 
Co-authored-by: Kumar Aditya 
Co-authored-by: Carol Willing 

files:
M Doc/library/asyncio-eventloop.rst

diff --git a/Doc/library/asyncio-eventloop.rst 
b/Doc/library/asyncio-eventloop.rst
index ba0ee1b6c2c528..6081a052e6e9da 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -1139,6 +1139,14 @@ DNS
 
Asynchronous version of :meth:`socket.getnameinfo`.
 
+.. note::
+   Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous
+   versions through the loop's default thread pool executor.
+   When this executor is saturated, these methods may experience delays,
+   which higher-level networking libraries may report as increased timeouts.
+   To mitigate this, consider using a custom executor for other user tasks,
+   or setting a default executor with a larger number of workers.
+
 .. versionchanged:: 3.7
Both *getaddrinfo* and *getnameinfo* methods were always documented
to return a coroutine, but prior to Python 3.7 they were, in fact,

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120935)

2024-06-24 Thread kumaraditya303

https://github.com/python/cpython/commit/206028dba986f982a940377ab1cb8b8276301b82
commit: 206028dba986f982a940377ab1cb8b8276301b82
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: kumaraditya303 
date: 2024-06-24T08:55:28Z
summary:

[3.13] gh-112169: Documented getaddrinfo/getnameinfo default loop executor 
usage and implications. (GH-112191) (#120935)

gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and 
implications. (GH-112191)
(cherry picked from commit fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a)

Co-authored-by: Alek Kowalczyk 
Co-authored-by: Guido van Rossum 
Co-authored-by: Kumar Aditya 
Co-authored-by: Carol Willing 

files:
M Doc/library/asyncio-eventloop.rst

diff --git a/Doc/library/asyncio-eventloop.rst 
b/Doc/library/asyncio-eventloop.rst
index 374e789e91e790..1d79f78e8e1b67 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -1155,6 +1155,14 @@ DNS
 
Asynchronous version of :meth:`socket.getnameinfo`.
 
+.. note::
+   Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous
+   versions through the loop's default thread pool executor.
+   When this executor is saturated, these methods may experience delays,
+   which higher-level networking libraries may report as increased timeouts.
+   To mitigate this, consider using a custom executor for other user tasks,
+   or setting a default executor with a larger number of workers.
+
 .. versionchanged:: 3.7
Both *getaddrinfo* and *getnameinfo* methods were always documented
to return a coroutine, but prior to Python 3.7 they were, in fact,

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909)

2024-06-24 Thread serhiy-storchaka

https://github.com/python/cpython/commit/c38e2f64d012929168dfef7363c9e48bd1a6c731
commit: c38e2f64d012929168dfef7363c9e48bd1a6c731
branch: main
author: Serhiy Storchaka 
committer: serhiy-storchaka 
date: 2024-06-24T12:17:25+03:00
summary:

gh-119614: Fix truncation of strings with embedded null characters in Tkinter 
(GH-120909)

Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().

files:
A Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
M Lib/test/test_tcl.py
M Lib/test/test_tkinter/test_misc.py
M Modules/_tkinter.c

diff --git a/Lib/test/test_tcl.py b/Lib/test/test_tcl.py
index 443787d721d5fb..d479f7d7515d9b 100644
--- a/Lib/test/test_tcl.py
+++ b/Lib/test/test_tcl.py
@@ -73,6 +73,18 @@ def testCall(self):
 tcl.call('set','a','1')
 self.assertEqual(tcl.call('set','a'),'1')
 
+def test_call_passing_null(self):
+tcl = self.interp
+tcl.call('set', 'a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.call('set', 'a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testCallException(self):
 tcl = self.interp
 self.assertRaises(TclError,tcl.call,'set','a')
@@ -98,6 +110,18 @@ def testSetVar(self):
 tcl.setvar('a','1')
 self.assertEqual(tcl.eval('set a'),'1')
 
+def test_setvar_passing_null(self):
+tcl = self.interp
+tcl.setvar('a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.setvar('a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testSetVarArray(self):
 tcl = self.interp
 tcl.setvar('a(1)','1')
diff --git a/Lib/test/test_tkinter/test_misc.py 
b/Lib/test/test_tkinter/test_misc.py
index d9ea642881a179..b0b9ed60040443 100644
--- a/Lib/test/test_tkinter/test_misc.py
+++ b/Lib/test/test_tkinter/test_misc.py
@@ -476,6 +476,15 @@ def test_info_patchlevel(self):
 self.assertEqual(vi.micro, 0)
 self.assertTrue(str(vi).startswith(f'{vi.major}.{vi.minor}'))
 
+def test_embedded_null(self):
+widget = tkinter.Entry(self.root)
+widget.insert(0, 'abc\0def')  # ASCII-only
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), 'abc\x00def')
+widget.insert(0, '\u20ac\0')  # non-ASCII
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), '\u20ac\0abc\x00def')
+
 
 class WmTest(AbstractTkTest, unittest.TestCase):
 
diff --git 
a/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst 
b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
new file mode 100644
index 00..d518265a7fe55a
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
@@ -0,0 +1,2 @@
+Fix truncation of strings with embedded null characters in some internal
+operations in :mod:`tkinter`.
diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c
index 8a5347be7de6ad..1542d3af42f755 100644
--- a/Modules/_tkinter.c
+++ b/Modules/_tkinter.c
@@ -512,7 +512,7 @@ unicodeFromTclObj(TkappObject *tkapp, Tcl_Obj *value)
 else
 Py_UNREACHABLE();
 }
-#endif
+#endif /* USE_TCL_UNICODE */
 const char *s = Tcl_GetStringFromObj(value, &len);
 return unicodeFromTclStringAndSize(s, len);
 }
@@ -1018,7 +1018,9 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;
 }
-if (PyUnicode_IS_ASCII(value)) {
+if (PyUnicode_IS_ASCII(value) &&
+strlen(PyUnicode_DATA(value)) == 
(size_t)PyUnicode_GET_LENGTH(value))
+{
 return Tcl_NewStringObj((const char *)PyUnicode_DATA(value),
 (int)size);
 }
@@ -1033,9 +1035,6 @@ AsObj(PyObject *value)
 "surrogatepass", NATIVE_BYTEORDER);
 else
 Py_UNREACHABLE();
-#else
-encoded = _PyUnicode_AsUTF8String(value, "surrogateescape");
-#endif
 if (!encoded) {
 return NULL;
 }
@@ -1045,12 +1044,39 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;
 }
-#if USE_TCL_UNICODE
 result = Tcl_NewUnicodeObj((const Tcl_UniChar 
*)PyBytes_AS_STRING(encoded),
(int)(size / sizeof(Tcl_Un

[Python-checkins] gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835)

2024-06-24 Thread iritkatriel

https://github.com/python/cpython/commit/65a12c559cbc13c2c5a4aa65c76310bd8d2051a7
commit: 65a12c559cbc13c2c5a4aa65c76310bd8d2051a7
branch: main
author: Irit Katriel <[email protected]>
committer: iritkatriel <[email protected]>
date: 2024-06-24T10:23:38+01:00
summary:

gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835)

files:
M Include/cpython/genobject.h
M Include/internal/pycore_frame.h
M Include/internal/pycore_genobject.h
M Include/internal/pycore_interp.h
M Objects/frameobject.c
M Objects/genobject.c
M Python/bytecodes.c
M Python/ceval.c
M Python/executor_cases.c.h
M Python/frame.c
M Python/generated_cases.c.h
M Python/pylifecycle.c

diff --git a/Include/cpython/genobject.h b/Include/cpython/genobject.h
index 49e46c277d75ae..f75884e597e2c2 100644
--- a/Include/cpython/genobject.h
+++ b/Include/cpython/genobject.h
@@ -9,29 +9,7 @@ extern "C" {
 
 /* --- Generators - */
 
-/* _PyGenObject_HEAD defines the initial segment of generator
-   and coroutine objects. */
-#define _PyGenObject_HEAD(prefix)   \
-PyObject_HEAD   \
-/* List of weak reference. */   \
-PyObject *prefix##_weakreflist; \
-/* Name of the generator. */\
-PyObject *prefix##_name;\
-/* Qualified name of the generator. */  \
-PyObject *prefix##_qualname;\
-_PyErr_StackItem prefix##_exc_state;\
-PyObject *prefix##_origin_or_finalizer; \
-char prefix##_hooks_inited; \
-char prefix##_closed;   \
-char prefix##_running_async;\
-/* The frame */ \
-int8_t prefix##_frame_state;\
-PyObject *prefix##_iframe[1];   \
-
-typedef struct {
-/* The gi_ prefix is intended to remind of generator-iterator. */
-_PyGenObject_HEAD(gi)
-} PyGenObject;
+typedef struct _PyGenObject PyGenObject;
 
 PyAPI_DATA(PyTypeObject) PyGen_Type;
 
@@ -46,9 +24,7 @@ PyAPI_FUNC(PyCodeObject *) PyGen_GetCode(PyGenObject *gen);
 
 /* --- PyCoroObject --- */
 
-typedef struct {
-_PyGenObject_HEAD(cr)
-} PyCoroObject;
+typedef struct _PyCoroObject PyCoroObject;
 
 PyAPI_DATA(PyTypeObject) PyCoro_Type;
 
@@ -59,9 +35,7 @@ PyAPI_FUNC(PyObject *) PyCoro_New(PyFrameObject *,
 
 /* --- Asynchronous Generators  */
 
-typedef struct {
-_PyGenObject_HEAD(ag)
-} PyAsyncGenObject;
+typedef struct _PyAsyncGenObject PyAsyncGenObject;
 
 PyAPI_DATA(PyTypeObject) PyAsyncGen_Type;
 PyAPI_DATA(PyTypeObject) _PyAsyncGenASend_Type;
@@ -73,7 +47,6 @@ PyAPI_FUNC(PyObject *) PyAsyncGen_New(PyFrameObject *,
 
 #define PyAsyncGenASend_CheckExact(op) Py_IS_TYPE((op), &_PyAsyncGenASend_Type)
 
-
 #undef _PyGenObject_HEAD
 
 #ifdef __cplusplus
diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h
index e4eb893263c42c..bab92c771a76b1 100644
--- a/Include/internal/pycore_frame.h
+++ b/Include/internal/pycore_frame.h
@@ -307,14 +307,6 @@ _PyFrame_PushTrampolineUnchecked(PyThreadState *tstate, 
PyCodeObject *code, int
 return frame;
 }
 
-static inline
-PyGenObject *_PyFrame_GetGenerator(_PyInterpreterFrame *frame)
-{
-assert(frame->owner == FRAME_OWNED_BY_GENERATOR);
-size_t offset_in_gen = offsetof(PyGenObject, gi_iframe);
-return (PyGenObject *)(((char *)frame) - offset_in_gen);
-}
-
 PyAPI_FUNC(_PyInterpreterFrame *)
 _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func,
 PyObject *locals, PyObject* const* args,
diff --git a/Include/internal/pycore_genobject.h 
b/Include/internal/pycore_genobject.h
index 9463c822ad8669..f6d7e6d367177b 100644
--- a/Include/internal/pycore_genobject.h
+++ b/Include/internal/pycore_genobject.h
@@ -8,7 +8,49 @@ extern "C" {
 #  error "this header requires Py_BUILD_CORE define"
 #endif
 
-#include "pycore_freelist.h"
+#include "pycore_frame.h"
+
+/* _PyGenObject_HEAD defines the initial segment of generator
+   and coroutine objects. */
+#define _PyGenObject_HEAD(prefix)   \
+PyObject_HEAD   \
+/* List of weak reference. */   \
+PyObject *prefix##_weakreflist;

[Python-checkins] [3.12] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909) (GH-120939)

2024-06-24 Thread serhiy-storchaka

https://github.com/python/cpython/commit/d5a8d4b19670b930cd6cb5e18e267877ebe49233
commit: d5a8d4b19670b930cd6cb5e18e267877ebe49233
branch: 3.12
author: Miss Islington (bot) <[email protected]>
committer: serhiy-storchaka 
date: 2024-06-24T09:39:56Z
summary:

[3.12] gh-119614: Fix truncation of strings with embedded null characters in 
Tkinter (GH-120909) (GH-120939)

Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
(cherry picked from commit c38e2f64d012929168dfef7363c9e48bd1a6c731)

Co-authored-by: Serhiy Storchaka 

files:
A Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
M Lib/test/test_tcl.py
M Lib/test/test_tkinter/test_misc.py
M Modules/_tkinter.c

diff --git a/Lib/test/test_tcl.py b/Lib/test/test_tcl.py
index 55a1fa66914c36..743ff85dd19075 100644
--- a/Lib/test/test_tcl.py
+++ b/Lib/test/test_tcl.py
@@ -73,6 +73,18 @@ def testCall(self):
 tcl.call('set','a','1')
 self.assertEqual(tcl.call('set','a'),'1')
 
+def test_call_passing_null(self):
+tcl = self.interp
+tcl.call('set', 'a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.call('set', 'a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testCallException(self):
 tcl = self.interp
 self.assertRaises(TclError,tcl.call,'set','a')
@@ -98,6 +110,18 @@ def testSetVar(self):
 tcl.setvar('a','1')
 self.assertEqual(tcl.eval('set a'),'1')
 
+def test_setvar_passing_null(self):
+tcl = self.interp
+tcl.setvar('a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.setvar('a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testSetVarArray(self):
 tcl = self.interp
 tcl.setvar('a(1)','1')
diff --git a/Lib/test/test_tkinter/test_misc.py 
b/Lib/test/test_tkinter/test_misc.py
index fc12860010efba..81866993435a0c 100644
--- a/Lib/test/test_tkinter/test_misc.py
+++ b/Lib/test/test_tkinter/test_misc.py
@@ -382,6 +382,15 @@ def test_info_patchlevel(self):
 self.assertEqual(vi.micro, 0)
 self.assertTrue(str(vi).startswith(f'{vi.major}.{vi.minor}'))
 
+def test_embedded_null(self):
+widget = tkinter.Entry(self.root)
+widget.insert(0, 'abc\0def')  # ASCII-only
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), 'abc\x00def')
+widget.insert(0, '\u20ac\0')  # non-ASCII
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), '\u20ac\0abc\x00def')
+
 
 class EventTest(AbstractTkTest, unittest.TestCase):
 
diff --git 
a/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst 
b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
new file mode 100644
index 00..d518265a7fe55a
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
@@ -0,0 +1,2 @@
+Fix truncation of strings with embedded null characters in some internal
+operations in :mod:`tkinter`.
diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c
index 29a3d28c090c43..6b5fcb8a365b61 100644
--- a/Modules/_tkinter.c
+++ b/Modules/_tkinter.c
@@ -516,7 +516,7 @@ unicodeFromTclObj(TkappObject *tkapp, Tcl_Obj *value)
 else
 Py_UNREACHABLE();
 }
-#endif
+#endif /* USE_TCL_UNICODE */
 const char *s = Tcl_GetStringFromObj(value, &len);
 return unicodeFromTclStringAndSize(s, len);
 }
@@ -1024,7 +1024,9 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;
 }
-if (PyUnicode_IS_ASCII(value)) {
+if (PyUnicode_IS_ASCII(value) &&
+strlen(PyUnicode_DATA(value)) == 
(size_t)PyUnicode_GET_LENGTH(value))
+{
 return Tcl_NewStringObj((const char *)PyUnicode_DATA(value),
 (int)size);
 }
@@ -1039,9 +1041,6 @@ AsObj(PyObject *value)
 "surrogatepass", NATIVE_BYTEORDER);
 else
 Py_UNREACHABLE();
-#else
-encoded = _PyUnicode_AsUTF8String(value, "surrogateescape");
-#endif
 if (!encoded) {
 return NULL;
 }
@@ -1051,12 +1050,39 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;

[Python-checkins] [3.13] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909) (GH-120938)

2024-06-24 Thread serhiy-storchaka

https://github.com/python/cpython/commit/732c00550f20e73edeac03d9e222c4a719362649
commit: 732c00550f20e73edeac03d9e222c4a719362649
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: serhiy-storchaka 
date: 2024-06-24T09:45:45Z
summary:

[3.13] gh-119614: Fix truncation of strings with embedded null characters in 
Tkinter (GH-120909) (GH-120938)

Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
(cherry picked from commit c38e2f64d012929168dfef7363c9e48bd1a6c731)

Co-authored-by: Serhiy Storchaka 

files:
A Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
M Lib/test/test_tcl.py
M Lib/test/test_tkinter/test_misc.py
M Modules/_tkinter.c

diff --git a/Lib/test/test_tcl.py b/Lib/test/test_tcl.py
index 443787d721d5fb..d479f7d7515d9b 100644
--- a/Lib/test/test_tcl.py
+++ b/Lib/test/test_tcl.py
@@ -73,6 +73,18 @@ def testCall(self):
 tcl.call('set','a','1')
 self.assertEqual(tcl.call('set','a'),'1')
 
+def test_call_passing_null(self):
+tcl = self.interp
+tcl.call('set', 'a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.call('set', 'a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testCallException(self):
 tcl = self.interp
 self.assertRaises(TclError,tcl.call,'set','a')
@@ -98,6 +110,18 @@ def testSetVar(self):
 tcl.setvar('a','1')
 self.assertEqual(tcl.eval('set a'),'1')
 
+def test_setvar_passing_null(self):
+tcl = self.interp
+tcl.setvar('a', 'a\0b')  # ASCII-only
+self.assertEqual(tcl.getvar('a'), 'a\x00b')
+self.assertEqual(tcl.call('set', 'a'), 'a\x00b')
+self.assertEqual(tcl.eval('set a'), 'a\x00b')
+
+tcl.setvar('a', '\u20ac\0')  # non-ASCII
+self.assertEqual(tcl.getvar('a'), '\u20ac\x00')
+self.assertEqual(tcl.call('set', 'a'), '\u20ac\x00')
+self.assertEqual(tcl.eval('set a'), '\u20ac\x00')
+
 def testSetVarArray(self):
 tcl = self.interp
 tcl.setvar('a(1)','1')
diff --git a/Lib/test/test_tkinter/test_misc.py 
b/Lib/test/test_tkinter/test_misc.py
index d9ea642881a179..b0b9ed60040443 100644
--- a/Lib/test/test_tkinter/test_misc.py
+++ b/Lib/test/test_tkinter/test_misc.py
@@ -476,6 +476,15 @@ def test_info_patchlevel(self):
 self.assertEqual(vi.micro, 0)
 self.assertTrue(str(vi).startswith(f'{vi.major}.{vi.minor}'))
 
+def test_embedded_null(self):
+widget = tkinter.Entry(self.root)
+widget.insert(0, 'abc\0def')  # ASCII-only
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), 'abc\x00def')
+widget.insert(0, '\u20ac\0')  # non-ASCII
+widget.selection_range(0, 'end')
+self.assertEqual(widget.selection_get(), '\u20ac\0abc\x00def')
+
 
 class WmTest(AbstractTkTest, unittest.TestCase):
 
diff --git 
a/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst 
b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
new file mode 100644
index 00..d518265a7fe55a
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-06-23-17-50-40.gh-issue-119614.vwPGLB.rst
@@ -0,0 +1,2 @@
+Fix truncation of strings with embedded null characters in some internal
+operations in :mod:`tkinter`.
diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c
index 8fe2c5ba295148..cd3722f54c24ce 100644
--- a/Modules/_tkinter.c
+++ b/Modules/_tkinter.c
@@ -512,7 +512,7 @@ unicodeFromTclObj(TkappObject *tkapp, Tcl_Obj *value)
 else
 Py_UNREACHABLE();
 }
-#endif
+#endif /* USE_TCL_UNICODE */
 const char *s = Tcl_GetStringFromObj(value, &len);
 return unicodeFromTclStringAndSize(s, len);
 }
@@ -1018,7 +1018,9 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;
 }
-if (PyUnicode_IS_ASCII(value)) {
+if (PyUnicode_IS_ASCII(value) &&
+strlen(PyUnicode_DATA(value)) == 
(size_t)PyUnicode_GET_LENGTH(value))
+{
 return Tcl_NewStringObj((const char *)PyUnicode_DATA(value),
 (int)size);
 }
@@ -1033,9 +1035,6 @@ AsObj(PyObject *value)
 "surrogatepass", NATIVE_BYTEORDER);
 else
 Py_UNREACHABLE();
-#else
-encoded = _PyUnicode_AsUTF8String(value, "surrogateescape");
-#endif
 if (!encoded) {
 return NULL;
 }
@@ -1045,12 +1044,39 @@ AsObj(PyObject *value)
 PyErr_SetString(PyExc_OverflowError, "string is too long");
 return NULL;

[Python-checkins] gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)

2024-06-24 Thread encukou

https://github.com/python/cpython/commit/ac61d58db0753a3b37de21dbc6e86b38f2a93f1b
commit: ac61d58db0753a3b37de21dbc6e86b38f2a93f1b
branch: main
author: Pablo Galindo Salgado 
committer: encukou 
date: 2024-06-24T14:08:12+02:00
summary:

gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from 
public API/ABI (GH-119680)


Signed-off-by: Pablo Galindo 
Co-authored-by: Petr Viktorin 

files:
M Doc/data/stable_abi.dat
M Include/internal/pycore_pyerrors.h
M Include/pyerrors.h
M Lib/codeop.py
M Lib/test/exception_hierarchy.txt
M Lib/test/test_pickle.py
M Lib/test/test_stable_abi_ctypes.py
M Misc/stable_abi.toml
M Objects/exceptions.c
M PC/python3dll.c
M Parser/pegen.c

diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat
index c18c813104cf65..1f7af436a4150b 100644
--- a/Doc/data/stable_abi.dat
+++ b/Doc/data/stable_abi.dat
@@ -226,7 +226,6 @@ var,PyExc_GeneratorExit,3.2,,
 var,PyExc_IOError,3.2,,
 var,PyExc_ImportError,3.2,,
 var,PyExc_ImportWarning,3.2,,
-var,PyExc_IncompleteInputError,3.13,,
 var,PyExc_IndentationError,3.2,,
 var,PyExc_IndexError,3.2,,
 var,PyExc_InterruptedError,3.7,,
diff --git a/Include/internal/pycore_pyerrors.h 
b/Include/internal/pycore_pyerrors.h
index 683d87a0d0b129..1187cbf7e90361 100644
--- a/Include/internal/pycore_pyerrors.h
+++ b/Include/internal/pycore_pyerrors.h
@@ -167,6 +167,10 @@ void _PyErr_FormatNote(const char *format, ...);
 
 Py_DEPRECATED(3.12) extern void _PyErr_ChainExceptions(PyObject *, PyObject *, 
PyObject *);
 
+// implementation detail for the codeop module.
+extern PyTypeObject _PyExc_IncompleteInputError;
+#define PyExc_IncompleteInputError ((PyObject *)(&_PyExc_IncompleteInputError))
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
index 68d7985dac8876..5d0028c116e2d8 100644
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -108,7 +108,6 @@ PyAPI_DATA(PyObject *) PyExc_NotImplementedError;
 PyAPI_DATA(PyObject *) PyExc_SyntaxError;
 PyAPI_DATA(PyObject *) PyExc_IndentationError;
 PyAPI_DATA(PyObject *) PyExc_TabError;
-PyAPI_DATA(PyObject *) PyExc_IncompleteInputError;
 PyAPI_DATA(PyObject *) PyExc_ReferenceError;
 PyAPI_DATA(PyObject *) PyExc_SystemError;
 PyAPI_DATA(PyObject *) PyExc_SystemExit;
diff --git a/Lib/codeop.py b/Lib/codeop.py
index 6ad60e7f85098d..a0276b52d484e3 100644
--- a/Lib/codeop.py
+++ b/Lib/codeop.py
@@ -65,7 +65,7 @@ def _maybe_compile(compiler, source, filename, symbol):
 try:
 compiler(source + "\n", filename, symbol)
 return None
-except IncompleteInputError as e:
+except _IncompleteInputError as e:
 return None
 except SyntaxError as e:
 pass
diff --git a/Lib/test/exception_hierarchy.txt b/Lib/test/exception_hierarchy.txt
index 65f54859e2a21d..5e83faab9a6158 100644
--- a/Lib/test/exception_hierarchy.txt
+++ b/Lib/test/exception_hierarchy.txt
@@ -45,7 +45,7 @@ BaseException
   ├── StopAsyncIteration
   ├── StopIteration
   ├── SyntaxError
-  │└── IncompleteInputError
+  │└── _IncompleteInputError
   │└── IndentationError
   │ └── TabError
   ├── SystemError
diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py
index 19f977971570b7..49aa4b386039ec 100644
--- a/Lib/test/test_pickle.py
+++ b/Lib/test/test_pickle.py
@@ -569,7 +569,7 @@ def test_exceptions(self):
EncodingWarning,
BaseExceptionGroup,
ExceptionGroup,
-   IncompleteInputError):
+   _IncompleteInputError):
 continue
 if exc is not OSError and issubclass(exc, OSError):
 self.assertEqual(reverse_mapping('builtins', name),
diff --git a/Lib/test/test_stable_abi_ctypes.py 
b/Lib/test/test_stable_abi_ctypes.py
index 47dff5c28f6ff8..d1d8a967dbe62f 100644
--- a/Lib/test/test_stable_abi_ctypes.py
+++ b/Lib/test/test_stable_abi_ctypes.py
@@ -267,7 +267,6 @@ def test_windows_feature_macros(self):
 "PyExc_IOError",
 "PyExc_ImportError",
 "PyExc_ImportWarning",
-"PyExc_IncompleteInputError",
 "PyExc_IndentationError",
 "PyExc_IndexError",
 "PyExc_InterruptedError",
diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml
index 305978f9f0c5c4..73012193d61485 100644
--- a/Misc/stable_abi.toml
+++ b/Misc/stable_abi.toml
@@ -2480,8 +2480,6 @@
 [function._Py_SetRefcnt]
 added = '3.13'
 abi_only = true
-[data.PyExc_IncompleteInputError]
-added = '3.13'
 [function.PyList_GetItemRef]
 added = '3.13'
 [typedef.PyCFunctionFast]
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
index 3a72cce1dff0c7..6376f2f012a7d6 100644
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -545,10 +545,10 @@ static PyTypeObject _PyExc_ ## EXCNAME = { \
 }; \
 PyObject *PyExc_ ## EXCNAME = (PyObject *)&_PyExc_ ## EXCNAME
 
-#define Midd

[Python-checkins] gh-120373: Mark test_audit.test_http as requiring the network resource (#120374)

2024-06-24 Thread carljm

https://github.com/python/cpython/commit/b0e1c51882e3a129d1e4db8291f7a0d869d6f1d6
commit: b0e1c51882e3a129d1e4db8291f7a0d869d6f1d6
branch: main
author: Itamar Oren 
committer: carljm 
date: 2024-06-24T07:18:46-06:00
summary:

gh-120373: Mark test_audit.test_http as requiring the network resource (#120374)

files:
M Lib/test/test_audit.py

diff --git a/Lib/test/test_audit.py b/Lib/test/test_audit.py
index 321d4f9abce8c7..7206307d8b0664 100644
--- a/Lib/test/test_audit.py
+++ b/Lib/test/test_audit.py
@@ -140,6 +140,7 @@ def test_gc(self):
 )
 
 
[email protected]_resource('network')
 def test_http(self):
 import_helper.import_module("http.client")
 returncode, events, stderr = self.run_python("test_http_client")

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] docs: puremagic.what() as replacement for imghdr.what() (#120871)

2024-06-24 Thread vstinner

https://github.com/python/cpython/commit/22b8a35d6e6660cf7457ed6636cb8c12fff7e8e7
commit: 22b8a35d6e6660cf7457ed6636cb8c12fff7e8e7
branch: main
author: Christian Clauss 
committer: vstinner 
date: 2024-06-24T15:32:13+02:00
summary:

docs: puremagic.what() as replacement for imghdr.what() (#120871)

files:
M Doc/whatsnew/3.13.rst

diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst
index 02c2a5ce02b730..5c195049980a4b 100644
--- a/Doc/whatsnew/3.13.rst
+++ b/Doc/whatsnew/3.13.rst
@@ -1307,6 +1307,9 @@ PEP 594: dead batteries (and other module removals)
 
   * :mod:`!imghdr`: use the projects :pypi:`filetype`,
 :pypi:`puremagic`, or :pypi:`python-magic` instead.
+The ``puremagic.what()`` function can be used to replace
+the ``imghdr.what()`` function for all file formats that
+were supported by ``imghdr``.
 (Contributed by Victor Stinner in :gh:`104773`.)
 
   * :mod:`!mailcap`.

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.12] gh-120373: Mark test_audit.test_http as requiring the network resource (GH-120374) (#120949)

2024-06-24 Thread carljm

https://github.com/python/cpython/commit/7a874b4ca6150681994a2a38b988711575f53d36
commit: 7a874b4ca6150681994a2a38b988711575f53d36
branch: 3.12
author: Miss Islington (bot) <[email protected]>
committer: carljm 
date: 2024-06-24T13:37:51Z
summary:

[3.12] gh-120373: Mark test_audit.test_http as requiring the network resource 
(GH-120374) (#120949)

gh-120373: Mark test_audit.test_http as requiring the network resource 
(GH-120374)
(cherry picked from commit b0e1c51882e3a129d1e4db8291f7a0d869d6f1d6)

Co-authored-by: Itamar Oren 

files:
M Lib/test/test_audit.py

diff --git a/Lib/test/test_audit.py b/Lib/test/test_audit.py
index 9e3e03748da14d..9076448ccfd45f 100644
--- a/Lib/test/test_audit.py
+++ b/Lib/test/test_audit.py
@@ -140,6 +140,7 @@ def test_gc(self):
 )
 
 
[email protected]_resource('network')
 def test_http(self):
 import_helper.import_module("http.client")
 returncode, events, stderr = self.run_python("test_http_client")

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] gh-120373: Mark test_audit.test_http as requiring the network resource (GH-120374) (#120948)

2024-06-24 Thread carljm

https://github.com/python/cpython/commit/502a99a7e66adc095842addfb366bba6953f34a1
commit: 502a99a7e66adc095842addfb366bba6953f34a1
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: carljm 
date: 2024-06-24T13:44:06Z
summary:

[3.13] gh-120373: Mark test_audit.test_http as requiring the network resource 
(GH-120374) (#120948)

gh-120373: Mark test_audit.test_http as requiring the network resource 
(GH-120374)
(cherry picked from commit b0e1c51882e3a129d1e4db8291f7a0d869d6f1d6)

Co-authored-by: Itamar Oren 

files:
M Lib/test/test_audit.py

diff --git a/Lib/test/test_audit.py b/Lib/test/test_audit.py
index 321d4f9abce8c7..7206307d8b0664 100644
--- a/Lib/test/test_audit.py
+++ b/Lib/test/test_audit.py
@@ -140,6 +140,7 @@ def test_gc(self):
 )
 
 
[email protected]_resource('network')
 def test_http(self):
 import_helper.import_module("http.client")
 returncode, events, stderr = self.run_python("test_http_client")

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] docs: puremagic.what() as replacement for imghdr.what() (GH-120871) (#120951)

2024-06-24 Thread vstinner

https://github.com/python/cpython/commit/a19a5895b5f8211a85366a1518e9d4f5fb3335c5
commit: a19a5895b5f8211a85366a1518e9d4f5fb3335c5
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: vstinner 
date: 2024-06-24T13:49:13Z
summary:

[3.13] docs: puremagic.what() as replacement for imghdr.what() (GH-120871) 
(#120951)

docs: puremagic.what() as replacement for imghdr.what() (GH-120871)
(cherry picked from commit 22b8a35d6e6660cf7457ed6636cb8c12fff7e8e7)

Co-authored-by: Christian Clauss 

files:
M Doc/whatsnew/3.13.rst

diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst
index 35853ff096b7d0..971f92c39298ce 100644
--- a/Doc/whatsnew/3.13.rst
+++ b/Doc/whatsnew/3.13.rst
@@ -1295,6 +1295,9 @@ PEP 594: dead batteries (and other module removals)
 
   * :mod:`!imghdr`: use the projects :pypi:`filetype`,
 :pypi:`puremagic`, or :pypi:`python-magic` instead.
+The ``puremagic.what()`` function can be used to replace
+the ``imghdr.what()`` function for all file formats that
+were supported by ``imghdr``.
 (Contributed by Victor Stinner in :gh:`104773`.)
 
   * :mod:`!mailcap`.

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] gh-70278: Fix PyUnicode_FromFormat() with precision for %s and %V (GH-120365)

2024-06-24 Thread serhiy-storchaka

https://github.com/python/cpython/commit/6eb23b1311e7eebf2459076703460ee7f8044f05
commit: 6eb23b1311e7eebf2459076703460ee7f8044f05
branch: main
author: Serhiy Storchaka 
committer: serhiy-storchaka 
date: 2024-06-24T18:07:07+03:00
summary:

gh-70278: Fix PyUnicode_FromFormat() with precision for %s and %V (GH-120365)

PyUnicode_FromFormat() no longer produces the ending \ufffd
character for truncated C string when use precision with %s and %V.
It now truncates the string before the start of truncated multibyte sequences.

files:
A Misc/NEWS.d/next/C API/2024-06-11-21-38-32.gh-issue-70278.WDE4zM.rst
M Lib/test/test_capi/test_unicode.py
M Objects/unicodeobject.c

diff --git a/Lib/test/test_capi/test_unicode.py 
b/Lib/test/test_capi/test_unicode.py
index 36106b0730dd26..48a802c3f8bcb2 100644
--- a/Lib/test/test_capi/test_unicode.py
+++ b/Lib/test/test_capi/test_unicode.py
@@ -419,8 +419,29 @@ def check_format(expected, format, *args):
 # truncated string
 check_format('abc',
  b'%.3s', b'abcdef')
+check_format('abc[',
+ b'%.6s', 'abc[\u20ac]'.encode('utf8'))
+check_format('abc[\u20ac',
+ b'%.7s', 'abc[\u20ac]'.encode('utf8'))
 check_format('abc[\ufffd',
- b'%.5s', 'abc[\u20ac]'.encode('utf8'))
+ b'%.5s', b'abc[\xff]')
+check_format('abc[',
+ b'%.6s', b'abc[\xe2\x82]')
+check_format('abc[\ufffd]',
+ b'%.7s', b'abc[\xe2\x82]')
+check_format('abc[\ufffd',
+ b'%.7s', b'abc[\xe2\x82\0')
+check_format('  abc[',
+ b'%10.6s', 'abc[\u20ac]'.encode('utf8'))
+check_format(' abc[\u20ac',
+ b'%10.7s', 'abc[\u20ac]'.encode('utf8'))
+check_format(' abc[\ufffd',
+ b'%10.5s', b'abc[\xff]')
+check_format('  abc[',
+ b'%10.6s', b'abc[\xe2\x82]')
+check_format('abc[\ufffd]',
+ b'%10.7s', b'abc[\xe2\x82]')
+
 check_format("'\\u20acABC'",
  b'%A', '\u20acABC')
 check_format("'\\u20",
@@ -433,10 +454,31 @@ def check_format(expected, format, *args):
  b'%.3S', '\u20acABCDEF')
 check_format('\u20acAB',
  b'%.3U', '\u20acABCDEF')
+
 check_format('\u20acAB',
  b'%.3V', '\u20acABCDEF', None)
+check_format('abc[',
+ b'%.6V', None, 'abc[\u20ac]'.encode('utf8'))
+check_format('abc[\u20ac',
+ b'%.7V', None, 'abc[\u20ac]'.encode('utf8'))
 check_format('abc[\ufffd',
- b'%.5V', None, 'abc[\u20ac]'.encode('utf8'))
+ b'%.5V', None, b'abc[\xff]')
+check_format('abc[',
+ b'%.6V', None, b'abc[\xe2\x82]')
+check_format('abc[\ufffd]',
+ b'%.7V', None, b'abc[\xe2\x82]')
+check_format('  abc[',
+ b'%10.6V', None, 'abc[\u20ac]'.encode('utf8'))
+check_format(' abc[\u20ac',
+ b'%10.7V', None, 'abc[\u20ac]'.encode('utf8'))
+check_format(' abc[\ufffd',
+ b'%10.5V', None, b'abc[\xff]')
+check_format('  abc[',
+ b'%10.6V', None, b'abc[\xe2\x82]')
+check_format('abc[\ufffd]',
+ b'%10.7V', None, b'abc[\xe2\x82]')
+check_format(' abc[\ufffd',
+ b'%10.7V', None, b'abc[\xe2\x82\0')
 
 # following tests comes from #7330
 # test width modifier and precision modifier with %S
diff --git a/Misc/NEWS.d/next/C 
API/2024-06-11-21-38-32.gh-issue-70278.WDE4zM.rst b/Misc/NEWS.d/next/C 
API/2024-06-11-21-38-32.gh-issue-70278.WDE4zM.rst
new file mode 100644
index 00..1eca36a86bc97e
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2024-06-11-21-38-32.gh-issue-70278.WDE4zM.rst  
@@ -0,0 +1,4 @@
+:c:func:`PyUnicode_FromFormat` no longer produces the ending ``\ufffd``
+character for truncated C string when use precision with ``%s`` and ``%V``.
+It now truncates the string before the start of truncated multibyte
+sequences.
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
index 279cdaa668e291..d11a9dca14b280 100644
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -2581,6 +2581,7 @@ unicode_fromformat_write_utf8(_PyUnicodeWriter *writer, 
const char *str,
   Py_ssize_t width, Py_ssize_t precision, int 
flags)
 {
 /* UTF-8 */
+Py_ssize_t *pconsumed = NULL;
 Py_ssize_t length;
 if (precision == -1) {
 length = strlen(str);
@@ -2590,15 +2591,23 @@ unicode_fromformat_write_utf8(_PyUnicodeWriter *writer, 
const char *str,
 while (length < precision && str[length]) {
 length++;
 }
+if (length == precision) {
+

[Python-checkins] gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)

2024-06-24 Thread encukou

https://github.com/python/cpython/commit/ce1064e4c9bcfd673323ad690e60f86e1ab907bb
commit: ce1064e4c9bcfd673323ad690e60f86e1ab907bb
branch: main
author: Petr Viktorin 
committer: encukou 
date: 2024-06-24T17:30:29+02:00
summary:

gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` 
(GH-120955)

files:
M Include/internal/pycore_pyerrors.h

diff --git a/Include/internal/pycore_pyerrors.h 
b/Include/internal/pycore_pyerrors.h
index 1187cbf7e90361..15071638203457 100644
--- a/Include/internal/pycore_pyerrors.h
+++ b/Include/internal/pycore_pyerrors.h
@@ -168,7 +168,8 @@ void _PyErr_FormatNote(const char *format, ...);
 Py_DEPRECATED(3.12) extern void _PyErr_ChainExceptions(PyObject *, PyObject *, 
PyObject *);
 
 // implementation detail for the codeop module.
-extern PyTypeObject _PyExc_IncompleteInputError;
+// Exported for test.test_peg_generator.test_c_parser
+PyAPI_DATA(PyTypeObject) _PyExc_IncompleteInputError;
 #define PyExc_IncompleteInputError ((PyObject *)(&_PyExc_IncompleteInputError))
 
 #ifdef __cplusplus

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854)

2024-06-24 Thread brandtbucher

https://github.com/python/cpython/commit/a47abdb45d4f1c3195c324812c33b6ef1d9147da
commit: a47abdb45d4f1c3195c324812c33b6ef1d9147da
branch: main
author: Brandt Bucher 
committer: brandtbucher 
date: 2024-06-24T08:35:10-07:00
summary:

GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854)

files:
M Python/bytecodes.c
M Python/executor_cases.c.h
M Python/optimizer.c
M Tools/jit/_stencils.py
M Tools/jit/template.c

diff --git a/Python/bytecodes.c b/Python/bytecodes.c
index a6fb862a6d476e..c00de88fc70a6e 100644
--- a/Python/bytecodes.c
+++ b/Python/bytecodes.c
@@ -4149,9 +4149,7 @@ dummy_func(
 }
 
 op(_JUMP_TO_TOP, (--)) {
-#ifndef _Py_JIT
-next_uop = ¤t_executor->trace[1];
-#endif
+JUMP_TO_JUMP_TARGET();
 }
 
 tier2 op(_SET_IP, (instr_ptr/4 --)) {
diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h
index cdfffcdec9726e..8de0309fa4dae0 100644
--- a/Python/executor_cases.c.h
+++ b/Python/executor_cases.c.h
@@ -4302,9 +4302,7 @@
 }
 
 case _JUMP_TO_TOP: {
-#ifndef _Py_JIT
-next_uop = ¤t_executor->trace[1];
-#endif
+JUMP_TO_JUMP_TARGET();
 break;
 }
 
diff --git a/Python/optimizer.c b/Python/optimizer.c
index c9b187d2e108dd..0d7118cd9e3363 100644
--- a/Python/optimizer.c
+++ b/Python/optimizer.c
@@ -1059,6 +1059,11 @@ prepare_for_execution(_PyUOpInstruction *buffer, int 
length)
 buffer[i].jump_target = 0;
 }
 }
+if (opcode == _JUMP_TO_TOP) {
+assert(buffer[0].opcode == _START_EXECUTOR);
+buffer[i].format = UOP_FORMAT_JUMP;
+buffer[i].jump_target = 1;
+}
 }
 return next_spare;
 }
diff --git a/Tools/jit/_stencils.py b/Tools/jit/_stencils.py
index c7c5ca1590d601..755649dea54570 100644
--- a/Tools/jit/_stencils.py
+++ b/Tools/jit/_stencils.py
@@ -42,8 +42,6 @@ class HoleValue(enum.Enum):
 ERROR_TARGET = enum.auto()
 # The index of the exit to be jumped through (exposed as _JIT_EXIT_INDEX):
 EXIT_INDEX = enum.auto()
-# The base address of the machine code for the first uop (exposed as 
_JIT_TOP):
-TOP = enum.auto()
 # A hardcoded value of zero (used for symbol lookups):
 ZERO = enum.auto()
 
@@ -110,7 +108,6 @@ class HoleValue(enum.Enum):
 HoleValue.JUMP_TARGET: "instruction_starts[instruction->jump_target]",
 HoleValue.ERROR_TARGET: "instruction_starts[instruction->error_target]",
 HoleValue.EXIT_INDEX: "instruction->exit_index",
-HoleValue.TOP: "instruction_starts[1]",
 HoleValue.ZERO: "",
 }
 
diff --git a/Tools/jit/template.c b/Tools/jit/template.c
index a81e866e9da4b3..813e586bd3c03d 100644
--- a/Tools/jit/template.c
+++ b/Tools/jit/template.c
@@ -105,9 +105,6 @@ _JIT_ENTRY(_PyInterpreterFrame *frame, PyObject 
**stack_pointer, PyThreadState *
 UOP_STAT_INC(uopcode, execution_count);
 
 // The actual instruction definitions (only one will be used):
-if (uopcode == _JUMP_TO_TOP) {
-PATCH_JUMP(_JIT_TOP);
-}
 switch (uopcode) {
 #include "executor_cases.c.h"
 default:

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] gh-119182: Add PyUnicodeWriter_WriteUCS4() function (#120849)

2024-06-24 Thread vstinner

https://github.com/python/cpython/commit/2e157851e36d83b0cb079b161d633b16ab899d16
commit: 2e157851e36d83b0cb079b161d633b16ab899d16
branch: main
author: Victor Stinner 
committer: vstinner 
date: 2024-06-24T17:40:39+02:00
summary:

gh-119182: Add PyUnicodeWriter_WriteUCS4() function (#120849)

files:
M Doc/c-api/unicode.rst
M Doc/whatsnew/3.14.rst
M Include/cpython/unicodeobject.h
M Lib/test/test_capi/test_unicode.py
M Misc/NEWS.d/next/C API/2024-06-07-22-12-30.gh-issue-119182.yt8Ar7.rst
M Modules/_testcapi/unicode.c
M Objects/unicodeobject.c

diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
index 4ea20bde38c1db..246cf47df62e78 100644
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1563,6 +1563,15 @@ object.
On success, return ``0``.
On error, set an exception, leave the writer unchanged, and return ``-1``.
 
+.. c:function:: int PyUnicodeWriter_WriteUCS4(PyUnicodeWriter *writer, Py_UCS4 
*str, Py_ssize_t size)
+
+   Writer the UCS4 string *str* into *writer*.
+
+   *size* is a number of UCS4 characters.
+
+   On success, return ``0``.
+   On error, set an exception, leave the writer unchanged, and return ``-1``.
+
 .. c:function:: int PyUnicodeWriter_WriteStr(PyUnicodeWriter *writer, PyObject 
*obj)
 
Call :c:func:`PyObject_Str` on *obj* and write the output into *writer*.
diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst
index b134ed31f6df40..9662044915b8ca 100644
--- a/Doc/whatsnew/3.14.rst
+++ b/Doc/whatsnew/3.14.rst
@@ -314,6 +314,7 @@ New Features
   * :c:func:`PyUnicodeWriter_Finish`.
   * :c:func:`PyUnicodeWriter_WriteChar`.
   * :c:func:`PyUnicodeWriter_WriteUTF8`.
+  * :c:func:`PyUnicodeWriter_WriteUCS4`.
   * :c:func:`PyUnicodeWriter_WriteWideChar`.
   * :c:func:`PyUnicodeWriter_WriteStr`.
   * :c:func:`PyUnicodeWriter_WriteRepr`.
diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h
index 059bec8618c8d9..91799137101280 100644
--- a/Include/cpython/unicodeobject.h
+++ b/Include/cpython/unicodeobject.h
@@ -463,6 +463,10 @@ PyAPI_FUNC(int) PyUnicodeWriter_WriteWideChar(
 PyUnicodeWriter *writer,
 const wchar_t *str,
 Py_ssize_t size);
+PyAPI_FUNC(int) PyUnicodeWriter_WriteUCS4(
+PyUnicodeWriter *writer,
+Py_UCS4 *str,
+Py_ssize_t size);
 
 PyAPI_FUNC(int) PyUnicodeWriter_WriteStr(
 PyUnicodeWriter *writer,
diff --git a/Lib/test/test_capi/test_unicode.py 
b/Lib/test/test_capi/test_unicode.py
index 48a802c3f8bcb2..9ef476a02de47d 100644
--- a/Lib/test/test_capi/test_unicode.py
+++ b/Lib/test/test_capi/test_unicode.py
@@ -1826,8 +1826,42 @@ def test_widechar(self):
 writer.write_widechar("latin1=\xE9")
 writer.write_widechar("-")
 writer.write_widechar("euro=\u20AC")
+writer.write_char("-")
+writer.write_widechar("max=\U0010")
 writer.write_char('.')
-self.assertEqual(writer.finish(), "latin1=\xE9-euro=\u20AC.")
+self.assertEqual(writer.finish(),
+ "latin1=\xE9-euro=\u20AC-max=\U0010.")
+
+def test_ucs4(self):
+writer = self.create_writer(0)
+writer.write_ucs4("ascii IGNORED", 5)
+writer.write_char("-")
+writer.write_ucs4("latin1=\xe9", 8)
+writer.write_char("-")
+writer.write_ucs4("euro=\u20ac", 6)
+writer.write_char("-")
+writer.write_ucs4("max=\U0010", 5)
+writer.write_char(".")
+self.assertEqual(writer.finish(),
+ "ascii-latin1=\xE9-euro=\u20AC-max=\U0010.")
+
+# Test some special characters
+writer = self.create_writer(0)
+# Lone surrogate character
+writer.write_ucs4("lone\uDC80", 5)
+writer.write_char("-")
+# Surrogate pair
+writer.write_ucs4("pair\uDBFF\uDFFF", 5)
+writer.write_char("-")
+writer.write_ucs4("null[\0]", 7)
+self.assertEqual(writer.finish(),
+ "lone\udc80-pair\udbff-null[\0]")
+
+# invalid size
+writer = self.create_writer(0)
+with self.assertRaises(ValueError):
+writer.write_ucs4("text", -1)
+
 
 
 @unittest.skipIf(ctypes is None, 'need ctypes')
diff --git a/Misc/NEWS.d/next/C 
API/2024-06-07-22-12-30.gh-issue-119182.yt8Ar7.rst b/Misc/NEWS.d/next/C 
API/2024-06-07-22-12-30.gh-issue-119182.yt8Ar7.rst
index 3d1384c9f3252f..243f290fbd47e2 100644
--- a/Misc/NEWS.d/next/C API/2024-06-07-22-12-30.gh-issue-119182.yt8Ar7.rst 
+++ b/Misc/NEWS.d/next/C API/2024-06-07-22-12-30.gh-issue-119182.yt8Ar7.rst 
@@ -5,9 +5,12 @@ Add a new :c:type:`PyUnicodeWriter` API to create a Python 
:class:`str` object:
 * :c:func:`PyUnicodeWriter_Finish`.
 * :c:func:`PyUnicodeWriter_WriteChar`.
 * :c:func:`PyUnicodeWriter_WriteUTF8`.
+* :c:func:`PyUnicodeWriter_WriteUCS4`.
+* :c:func:`PyUnicodeWriter_WriteWideChar`.
 * :c:func:`PyUnicodeWriter_WriteStr`.
 * :c:func:`PyUnicodeWriter_WriteRepr`.
 * :c:func:`PyUnicodeWriter_WriteSubstring`.
 * :c:func:`PyUnicodeWri

[Python-checkins] Fixes loop variables to be the same types as their limit (GH-120958)

2024-06-24 Thread zooba

https://github.com/python/cpython/commit/e7315543377322e4c6e0d8d2c4a4bb4626e43f4c
commit: e7315543377322e4c6e0d8d2c4a4bb4626e43f4c
branch: main
author: Steve Dower 
committer: zooba 
date: 2024-06-24T17:11:47+01:00
summary:

Fixes loop variables to be the same types as their limit (GH-120958)

files:
M Modules/_io/_iomodule.c
M Modules/_sqlite/blob.c
M Modules/_testinternalcapi/test_critical_sections.c
M Objects/codeobject.c
M Objects/unicodeobject.c
M Objects/unionobject.c
M Parser/action_helpers.c
M Python/ast_opt.c
M Python/compile.c
M Python/flowgraph.c
M Python/future.c
M Python/getargs.c
M Python/suggestions.c
M Python/symtable.c

diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
index d236098e6977e8..1238e6074246d0 100644
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -202,7 +202,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char 
*mode,
   const char *newline, int closefd, PyObject *opener)
 /*[clinic end generated code: output=aefafc4ce2b46dc0 input=cd034e7cdfbf4e78]*/
 {
-unsigned i;
+size_t i;
 
 int creating = 0, reading = 0, writing = 0, appending = 0, updating = 0;
 int text = 0, binary = 0;
diff --git a/Modules/_sqlite/blob.c b/Modules/_sqlite/blob.c
index 7deb58bf1b9b82..d1a549a971c24a 100644
--- a/Modules/_sqlite/blob.c
+++ b/Modules/_sqlite/blob.c
@@ -99,7 +99,7 @@ blob_close_impl(pysqlite_Blob *self)
 void
 pysqlite_close_all_blobs(pysqlite_Connection *self)
 {
-for (int i = 0; i < PyList_GET_SIZE(self->blobs); i++) {
+for (Py_ssize_t i = 0; i < PyList_GET_SIZE(self->blobs); i++) {
 PyObject *weakref = PyList_GET_ITEM(self->blobs, i);
 PyObject *blob;
 if (!PyWeakref_GetRef(weakref, &blob)) {
diff --git a/Modules/_testinternalcapi/test_critical_sections.c 
b/Modules/_testinternalcapi/test_critical_sections.c
index 0129bd49ca93c3..1df960f9881f70 100644
--- a/Modules/_testinternalcapi/test_critical_sections.c
+++ b/Modules/_testinternalcapi/test_critical_sections.c
@@ -185,7 +185,7 @@ test_critical_sections_threads(PyObject *self, PyObject 
*Py_UNUSED(args))
 assert(test_data.obj2 != NULL);
 assert(test_data.obj3 != NULL);
 
-for (int i = 0; i < NUM_THREADS; i++) {
+for (Py_ssize_t i = 0; i < NUM_THREADS; i++) {
 PyThread_start_new_thread(&thread_critical_sections, &test_data);
 }
 PyEvent_Wait(&test_data.done_event);
@@ -271,7 +271,7 @@ test_critical_sections_gc(PyObject *self, PyObject 
*Py_UNUSED(args))
 };
 assert(test_data.obj != NULL);
 
-for (int i = 0; i < NUM_THREADS; i++) {
+for (Py_ssize_t i = 0; i < NUM_THREADS; i++) {
 PyThread_start_new_thread(&thread_gc, &test_data);
 }
 PyEvent_Wait(&test_data.done_event);
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
index 95da63506c670a..4be17708d3aab7 100644
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -573,7 +573,7 @@ get_line_delta(const uint8_t *ptr)
 static PyObject *
 remove_column_info(PyObject *locations)
 {
-int offset = 0;
+Py_ssize_t offset = 0;
 const uint8_t *data = (const uint8_t *)PyBytes_AS_STRING(locations);
 PyObject *res = PyBytes_FromStringAndSize(NULL, 32);
 if (res == NULL) {
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
index 698e57f5ad0407..0710c6286c80da 100644
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -8373,7 +8373,7 @@ PyUnicode_BuildEncodingMap(PyObject* string)
 int count2 = 0, count3 = 0;
 int kind;
 const void *data;
-Py_ssize_t length;
+int length;
 Py_UCS4 ch;
 
 if (!PyUnicode_Check(string) || !PyUnicode_GET_LENGTH(string)) {
@@ -8382,8 +8382,7 @@ PyUnicode_BuildEncodingMap(PyObject* string)
 }
 kind = PyUnicode_KIND(string);
 data = PyUnicode_DATA(string);
-length = PyUnicode_GET_LENGTH(string);
-length = Py_MIN(length, 256);
+length = (int)Py_MIN(PyUnicode_GET_LENGTH(string), 256);
 memset(level1, 0xFF, sizeof level1);
 memset(level2, 0xFF, sizeof level2);
 
diff --git a/Objects/unionobject.c b/Objects/unionobject.c
index 49b01e0aeb5d83..7931f4345f7fdd 100644
--- a/Objects/unionobject.c
+++ b/Objects/unionobject.c
@@ -81,7 +81,7 @@ is_same(PyObject *left, PyObject *right)
 static int
 contains(PyObject **items, Py_ssize_t size, PyObject *obj)
 {
-for (int i = 0; i < size; i++) {
+for (Py_ssize_t i = 0; i < size; i++) {
 int is_duplicate = is_same(items[i], obj);
 if (is_duplicate) {  // -1 or 1
 return is_duplicate;
@@ -97,7 +97,7 @@ merge(PyObject **items1, Py_ssize_t size1,
 PyObject *tuple = NULL;
 Py_ssize_t pos = 0;
 
-for (int i = 0; i < size2; i++) {
+for (Py_ssize_t i = 0; i < size2; i++) {
 PyObject *arg = items2[i];
 int is_duplicate = contains(items1, size1, arg);
 if (is_duplicate < 0) {
diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c
index 6ebc457f6328c5..44bf87da8288a6 100644
--- a/Parser/action_hel

[Python-checkins] gh-120956: Avoid comparison of int to Py_ssize_t in parser (#120959)

2024-06-24 Thread lysnikolaou

https://github.com/python/cpython/commit/348184845a72088368021d1f42e96ceea3eee88c
commit: 348184845a72088368021d1f42e96ceea3eee88c
branch: main
author: Lysandros Nikolaou 
committer: lysnikolaou 
date: 2024-06-24T18:13:02+02:00
summary:

gh-120956: Avoid comparison of int to Py_ssize_t in parser (#120959)

files:
M Parser/parser.c
M Tools/peg_generator/pegen/c_generator.py

diff --git a/Parser/parser.c b/Parser/parser.c
index 05cd93c2c92f29..cbbe9a9be87178 100644
--- a/Parser/parser.c
+++ b/Parser/parser.c
@@ -25632,7 +25632,7 @@ _loop0_1_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -25699,7 +25699,7 @@ _loop0_2_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -25771,7 +25771,7 @@ _loop1_3_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -25847,7 +25847,7 @@ _loop0_5_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -26424,7 +26424,7 @@ _loop1_14_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -26660,7 +26660,7 @@ _loop0_19_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -26777,7 +26777,7 @@ _loop0_21_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -26988,7 +26988,7 @@ _loop0_24_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27060,7 +27060,7 @@ _loop1_25_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27136,7 +27136,7 @@ _loop0_27_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27299,7 +27299,7 @@ _loop0_30_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27458,7 +27458,7 @@ _loop1_32_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27666,7 +27666,7 @@ _loop0_36_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, 
_children[i]);
 PyMem_Free(_children);
 p->level--;
 return _seq;
@@ -27733,7 +27733,7 @@ _loop0_37_rule(Parser *p)
 p->level--;
 return NULL;
 }
-for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);
+for (Py_ssize_t i = 0; i < _n;

[Python-checkins] gh-120834: fix over-allocation in PyGenObject, PyCoroObject, PyAsyncGenObject. (#120941)

2024-06-24 Thread iritkatriel

https://github.com/python/cpython/commit/8ac08f36fe9016a8109edf9e167b06627fb13b5a
commit: 8ac08f36fe9016a8109edf9e167b06627fb13b5a
branch: main
author: Irit Katriel <[email protected]>
committer: iritkatriel <[email protected]>
date: 2024-06-24T18:41:53+01:00
summary:

gh-120834: fix over-allocation in PyGenObject, PyCoroObject, PyAsyncGenObject. 
(#120941)

files:
M Objects/genobject.c

diff --git a/Objects/genobject.c b/Objects/genobject.c
index 445622e878bc93..9a3194d3de2614 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -813,7 +813,7 @@ static PyAsyncMethods gen_as_async = {
 PyTypeObject PyGen_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "generator",/* tp_name */
-sizeof(PyGenObject),/* tp_basicsize */
+offsetof(PyGenObject, gi_iframe.localsplus), /* tp_basicsize */
 sizeof(PyObject *), /* tp_itemsize */
 /* methods */
 (destructor)gen_dealloc,/* tp_dealloc */
@@ -1164,7 +1164,7 @@ static PyAsyncMethods coro_as_async = {
 PyTypeObject PyCoro_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "coroutine",/* tp_name */
-sizeof(PyCoroObject),   /* tp_basicsize */
+offsetof(PyCoroObject, cr_iframe.localsplus),/* tp_basicsize */
 sizeof(PyObject *), /* tp_itemsize */
 /* methods */
 (destructor)gen_dealloc,/* tp_dealloc */
@@ -1579,7 +1579,7 @@ static PyAsyncMethods async_gen_as_async = {
 PyTypeObject PyAsyncGen_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "async_generator",  /* tp_name */
-sizeof(PyAsyncGenObject),   /* tp_basicsize */
+offsetof(PyAsyncGenObject, ag_iframe.localsplus), /* tp_basicsize */
 sizeof(PyObject *), /* tp_itemsize */
 /* methods */
 (destructor)gen_dealloc,/* tp_dealloc */

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] Fix typos in comments (#120821)

2024-06-24 Thread vstinner

https://github.com/python/cpython/commit/0153fd094019b84e18b8e8451019694595f67f9e
commit: 0153fd094019b84e18b8e8451019694595f67f9e
branch: main
author: Xie Yanbo 
committer: vstinner 
date: 2024-06-24T19:47:00+02:00
summary:

Fix typos in comments (#120821)

files:
M Objects/codeobject.c
M Objects/complexobject.c
M Objects/dictobject.c
M Objects/listobject.c
M Objects/mimalloc/bitmap.c
M Objects/mimalloc/heap.c
M Objects/obmalloc.c
M Objects/unicodeobject.c

diff --git a/Objects/codeobject.c b/Objects/codeobject.c
index 4be17708d3aab7..7493280c898750 100644
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -236,7 +236,7 @@ intern_constants(PyObject *tuple, int *modified)
 Py_DECREF(tmp);
 }
 
-// Intern non-string consants in the free-threaded build, but only if
+// Intern non-string constants in the free-threaded build, but only if
 // we are also immortalizing objects that use deferred reference
 // counting.
 PyThreadState *tstate = PyThreadState_GET();
diff --git a/Objects/complexobject.c b/Objects/complexobject.c
index a8be266970afd0..7b62fe30b2b007 100644
--- a/Objects/complexobject.c
+++ b/Objects/complexobject.c
@@ -912,7 +912,7 @@ complex_subtype_from_string(PyTypeObject *type, PyObject *v)
  * handles the case of no arguments and one positional argument, and calls
  * complex_new(), implemented with Argument Clinic, to handle the remaining
  * cases: 'real' and 'imag' arguments.  This separation is well suited
- * for different constructor roles: convering a string or number to a complex
+ * for different constructor roles: converting a string or number to a complex
  * number and constructing a complex number from real and imaginary parts.
  */
 static PyObject *
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index 51cfdf1c4d8c55..5529e527d8bea3 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -5383,7 +5383,7 @@ dictiter_iternextitem_lock_held(PyDictObject *d, PyObject 
*self,
 #ifdef Py_GIL_DISABLED
 
 // Grabs the key and/or value from the provided locations and if successful
-// returns them with an increased reference count.  If either one is 
unsucessful
+// returns them with an increased reference count.  If either one is 
unsuccessful
 // nothing is incref'd and returns -1.
 static int
 acquire_key_value(PyObject **key_loc, PyObject *value, PyObject **value_loc,
diff --git a/Objects/listobject.c b/Objects/listobject.c
index dc9df3c3614fb4..9eae9626f7c1f1 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -1866,7 +1866,7 @@ count_run(MergeState *ms, sortslice *slo, Py_ssize_t 
nremaining)
 /* In general, as things go on we've established that the slice starts
with a monotone run of n elements, starting at lo. */
 
-/* We're n elements into the slice, and the most recent neq+1 elments are
+/* We're n elements into the slice, and the most recent neq+1 elements are
  * all equal. This reverses them in-place, and resets neq for reuse.
  */
 #define REVERSE_LAST_NEQ\
@@ -1918,7 +1918,7 @@ count_run(MergeState *ms, sortslice *slo, Py_ssize_t 
nremaining)
 Py_ssize_t neq = 0;
 for ( ; n < nremaining; ++n) {
 IF_NEXT_SMALLER {
-/* This ends the most recent run of equal elments, but still in
+/* This ends the most recent run of equal elements, but still in
  * the "descending" direction.
  */
 REVERSE_LAST_NEQ
diff --git a/Objects/mimalloc/bitmap.c b/Objects/mimalloc/bitmap.c
index ec3c755822dac1..31830756f58c7c 100644
--- a/Objects/mimalloc/bitmap.c
+++ b/Objects/mimalloc/bitmap.c
@@ -7,7 +7,7 @@ terms of the MIT license. A copy of the license can be found in 
the file
 
 /* 
 Concurrent bitmap that can set/reset sequences of bits atomically,
-represeted as an array of fields where each field is a machine word (`size_t`)
+represented as an array of fields where each field is a machine word (`size_t`)
 
 There are two api's; the standard one cannot have sequences that cross
 between the bitmap fields (and a sequence must be <= MI_BITMAP_FIELD_BITS).
@@ -108,7 +108,7 @@ bool _mi_bitmap_try_find_from_claim(mi_bitmap_t bitmap, 
const size_t bitmap_fiel
   return false;
 }
 
-// Like _mi_bitmap_try_find_from_claim but with an extra predicate that must 
be fullfilled
+// Like _mi_bitmap_try_find_from_claim but with an extra predicate that must 
be fulfilled
 bool _mi_bitmap_try_find_from_claim_pred(mi_bitmap_t bitmap, const size_t 
bitmap_fields,
 const size_t start_field_idx, const size_t count,
 mi_bitmap_pred_fun_t pred_fun, void* pred_arg,
diff --git a/Objects/mimalloc/heap.c b/Objects/mimalloc/heap.c
index 26777f39fb6aa5..d92dc768e5ec28 100644
--- a/Objects/mimalloc/heap.c
+++ b/Objects/mimalloc/heap.c
@@ -446,7 +446,7 @@ void mi_heap_delete(mi_heap_t* heap)
   if (heap==

[Python-checkins] gh-120860: Fix a few bugs in `type_setattro` error paths. (#120861)

2024-06-24 Thread colesbury

https://github.com/python/cpython/commit/dee63cb35971b87a09ddda5d6f29cd941f570720
commit: dee63cb35971b87a09ddda5d6f29cd941f570720
branch: main
author: Sam Gross 
committer: colesbury 
date: 2024-06-24T14:08:23-04:00
summary:

gh-120860: Fix a few bugs in `type_setattro` error paths. (#120861)

Moves the logic to update the type's dictionary to its own function in order
to make the lock scoping more clear.

Also, ensure that `name` is decref'd on the error path.

files:
M Objects/typeobject.c

diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 53c40885fd621a..71fcb6559fff73 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -5656,6 +5656,42 @@ _Py_type_getattro(PyObject *type, PyObject *name)
 return _Py_type_getattro_impl((PyTypeObject *)type, name, NULL);
 }
 
+static int
+type_update_dict(PyTypeObject *type, PyDictObject *dict, PyObject *name,
+ PyObject *value, PyObject **old_value)
+{
+// We don't want any re-entrancy between when we update the dict
+// and call type_modified_unlocked, including running the destructor
+// of the current value as it can observe the cache in an inconsistent
+// state.  Because we have an exact unicode and our dict has exact
+// unicodes we know that this will all complete without releasing
+// the locks.
+if (_PyDict_GetItemRef_Unicode_LockHeld(dict, name, old_value) < 0) {
+return -1;
+}
+
+/* Clear the VALID_VERSION flag of 'type' and all its
+subclasses.  This could possibly be unified with the
+update_subclasses() recursion in update_slot(), but carefully:
+they each have their own conditions on which to stop
+recursing into subclasses. */
+type_modified_unlocked(type);
+
+if (_PyDict_SetItem_LockHeld(dict, name, value) < 0) {
+PyErr_Format(PyExc_AttributeError,
+ "type object '%.50s' has no attribute '%U'",
+ ((PyTypeObject*)type)->tp_name, name);
+_PyObject_SetAttributeErrorContext((PyObject *)type, name);
+return -1;
+}
+
+if (is_dunder_name(name)) {
+return update_slot(type, name);
+}
+
+return 0;
+}
+
 static int
 type_setattro(PyObject *self, PyObject *name, PyObject *value)
 {
@@ -5698,12 +5734,11 @@ type_setattro(PyObject *self, PyObject *name, PyObject 
*value)
 assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_INLINE_VALUES));
 assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_MANAGED_DICT));
 
-PyObject *old_value;
+PyObject *old_value = NULL;
 PyObject *descr = _PyType_LookupRef(metatype, name);
 if (descr != NULL) {
 descrsetfunc f = Py_TYPE(descr)->tp_descr_set;
 if (f != NULL) {
-old_value = NULL;
 res = f(descr, (PyObject *)type, value);
 goto done;
 }
@@ -5719,47 +5754,16 @@ type_setattro(PyObject *self, PyObject *name, PyObject 
*value)
 }
 END_TYPE_LOCK();
 if (dict == NULL) {
-return -1;
+res = -1;
+goto done;
 }
 }
 
-// We don't want any re-entrancy between when we update the dict
-// and call type_modified_unlocked, including running the destructor
-// of the current value as it can observe the cache in an inconsistent
-// state.  Because we have an exact unicode and our dict has exact
-// unicodes we know that this will all complete without releasing
-// the locks.
 BEGIN_TYPE_DICT_LOCK(dict);
-
-if (_PyDict_GetItemRef_Unicode_LockHeld((PyDictObject *)dict, name, 
&old_value) < 0) {
-return -1;
-}
-
-/* Clear the VALID_VERSION flag of 'type' and all its
-subclasses.  This could possibly be unified with the
-update_subclasses() recursion in update_slot(), but carefully:
-they each have their own conditions on which to stop
-recursing into subclasses. */
-type_modified_unlocked(type);
-
-res = _PyDict_SetItem_LockHeld((PyDictObject *)dict, name, value);
-
-if (res == 0) {
-if (is_dunder_name(name)) {
-res = update_slot(type, name);
-}
-}
-else if (PyErr_ExceptionMatches(PyExc_KeyError)) {
-PyErr_Format(PyExc_AttributeError,
-"type object '%.50s' has no attribute '%U'",
-((PyTypeObject*)type)->tp_name, name);
-
-_PyObject_SetAttributeErrorContext((PyObject *)type, name);
-}
-
+res = type_update_dict(type, (PyDictObject *)dict, name, value, 
&old_value);
 assert(_PyType_CheckConsistency(type));
-
 END_TYPE_DICT_LOCK();
+
 done:
 Py_DECREF(name);
 Py_XDECREF(descr);

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] gh-120858: PyDict_Next should not lock the dict (#120859)

2024-06-24 Thread colesbury

https://github.com/python/cpython/commit/375b723d5873f948696c7e85a97f4778d9e00ff0
commit: 375b723d5873f948696c7e85a97f4778d9e00ff0
branch: main
author: Sam Gross 
committer: colesbury 
date: 2024-06-24T14:15:15-04:00
summary:

gh-120858: PyDict_Next should not lock the dict (#120859)

PyDict_Next no longer locks the dictionary in the free-threaded build. Locking
around individual PyDict_Next calls is not sufficient because the function
returns borrowed references and because it allows concurrent modifications
during the iteraiton loop.

The internal locking also interferes with correct external synchronization
because it may suspend outer critical sections created by the caller.

files:
A Misc/NEWS.d/next/C API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst
M Doc/c-api/dict.rst
M Doc/howto/free-threading-extensions.rst
M Objects/dictobject.c

diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst
index 49a78583a6fe26..b4fdf47bc915bd 100644
--- a/Doc/c-api/dict.rst
+++ b/Doc/c-api/dict.rst
@@ -290,6 +290,17 @@ Dictionary Objects
   Py_DECREF(o);
   }
 
+   The function is not thread-safe in the :term:`free-threaded `
+   build without external synchronization.  You can use
+   :c:macro:`Py_BEGIN_CRITICAL_SECTION` to lock the dictionary while iterating
+   over it::
+
+  Py_BEGIN_CRITICAL_SECTION(self->dict);
+  while (PyDict_Next(self->dict, &pos, &key, &value)) {
+  ...
+  }
+  Py_END_CRITICAL_SECTION();
+
 
 .. c:function:: int PyDict_Merge(PyObject *a, PyObject *b, int override)
 
diff --git a/Doc/howto/free-threading-extensions.rst 
b/Doc/howto/free-threading-extensions.rst
index 080170de1e5d16..1ba91b09516f9c 100644
--- a/Doc/howto/free-threading-extensions.rst
+++ b/Doc/howto/free-threading-extensions.rst
@@ -114,6 +114,24 @@ Containers like :c:struct:`PyListObject`,
 in the free-threaded build.  For example, the :c:func:`PyList_Append` will
 lock the list before appending an item.
 
+.. _PyDict_Next:
+
+``PyDict_Next``
+'''
+
+A notable exception is :c:func:`PyDict_Next`, which does not lock the
+dictionary.  You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect
+the dictionary while iterating over it if the dictionary may be concurrently
+modified::
+
+Py_BEGIN_CRITICAL_SECTION(dict);
+PyObject *key, *value;
+Py_ssize_t pos = 0;
+while (PyDict_Next(dict, &pos, &key, &value)) {
+...
+}
+Py_END_CRITICAL_SECTION();
+
 
 Borrowed References
 ===
@@ -141,7 +159,7 @@ that return :term:`strong references `.
 +---+---+
 | :c:func:`PyDict_SetDefault`   | :c:func:`PyDict_SetDefaultRef`|
 +---+---+
-| :c:func:`PyDict_Next` | no direct replacement |
+| :c:func:`PyDict_Next` | none (see :ref:`PyDict_Next`) |
 +---+---+
 | :c:func:`PyWeakref_GetObject` | :c:func:`PyWeakref_GetRef`|
 +---+---+
diff --git a/Misc/NEWS.d/next/C 
API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst b/Misc/NEWS.d/next/C 
API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst
new file mode 100644
index 00..b5df2a567b9da8
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst 
@@ -0,0 +1,3 @@
+:c:func:`PyDict_Next` no longer locks the dictionary in the free-threaded
+build.  The locking needs to be done by the caller around the entire iteration
+loop.
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index 5529e527d8bea3..5d325465608f99 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2808,8 +2808,6 @@ _PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject 
**pkey,
 if (!PyDict_Check(op))
 return 0;
 
-ASSERT_DICT_LOCKED(op);
-
 mp = (PyDictObject *)op;
 i = *ppos;
 if (_PyDict_HasSplitTable(mp)) {
@@ -2882,11 +2880,7 @@ _PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject 
**pkey,
 int
 PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject **pkey, PyObject **pvalue)
 {
-int res;
-Py_BEGIN_CRITICAL_SECTION(op);
-res = _PyDict_Next(op, ppos, pkey, pvalue, NULL);
-Py_END_CRITICAL_SECTION();
-return res;
+return _PyDict_Next(op, ppos, pkey, pvalue, NULL);
 }
 
 

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680, GH-120955) (GH-120944)

2024-06-24 Thread encukou

https://github.com/python/cpython/commit/447e07ab3d569bb4b2209ccfe3889fafa3ad6693
commit: 447e07ab3d569bb4b2209ccfe3889fafa3ad6693
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: encukou 
date: 2024-06-24T20:23:30+02:00
summary:

[3.13] gh-119521: Rename IncompleteInputError to _IncompleteInputError and 
remove from public API/ABI (GH-119680, GH-120955) (GH-120944)

- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove 
from public API/ABI (GH-119680)
  (cherry picked from commit ce1064e4c9bcfd673323ad690e60f86e1ab907bb)

- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` 
(GH-120955)
  (cherry picked from commit ac61d58db0753a3b37de21dbc6e86b38f2a93f1b)

Co-authored-by: Pablo Galindo Salgado 
Co-authored-by: Petr Viktorin 

files:
M Doc/data/stable_abi.dat
M Include/internal/pycore_pyerrors.h
M Include/pyerrors.h
M Lib/codeop.py
M Lib/test/exception_hierarchy.txt
M Lib/test/test_pickle.py
M Lib/test/test_stable_abi_ctypes.py
M Misc/stable_abi.toml
M Objects/exceptions.c
M PC/python3dll.c
M Parser/pegen.c

diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat
index 76a035f194d911..9c17223a49a498 100644
--- a/Doc/data/stable_abi.dat
+++ b/Doc/data/stable_abi.dat
@@ -226,7 +226,6 @@ var,PyExc_GeneratorExit,3.2,,
 var,PyExc_IOError,3.2,,
 var,PyExc_ImportError,3.2,,
 var,PyExc_ImportWarning,3.2,,
-var,PyExc_IncompleteInputError,3.13,,
 var,PyExc_IndentationError,3.2,,
 var,PyExc_IndexError,3.2,,
 var,PyExc_InterruptedError,3.7,,
diff --git a/Include/internal/pycore_pyerrors.h 
b/Include/internal/pycore_pyerrors.h
index 683d87a0d0b129..15071638203457 100644
--- a/Include/internal/pycore_pyerrors.h
+++ b/Include/internal/pycore_pyerrors.h
@@ -167,6 +167,11 @@ void _PyErr_FormatNote(const char *format, ...);
 
 Py_DEPRECATED(3.12) extern void _PyErr_ChainExceptions(PyObject *, PyObject *, 
PyObject *);
 
+// implementation detail for the codeop module.
+// Exported for test.test_peg_generator.test_c_parser
+PyAPI_DATA(PyTypeObject) _PyExc_IncompleteInputError;
+#define PyExc_IncompleteInputError ((PyObject *)(&_PyExc_IncompleteInputError))
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
index 68d7985dac8876..5d0028c116e2d8 100644
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -108,7 +108,6 @@ PyAPI_DATA(PyObject *) PyExc_NotImplementedError;
 PyAPI_DATA(PyObject *) PyExc_SyntaxError;
 PyAPI_DATA(PyObject *) PyExc_IndentationError;
 PyAPI_DATA(PyObject *) PyExc_TabError;
-PyAPI_DATA(PyObject *) PyExc_IncompleteInputError;
 PyAPI_DATA(PyObject *) PyExc_ReferenceError;
 PyAPI_DATA(PyObject *) PyExc_SystemError;
 PyAPI_DATA(PyObject *) PyExc_SystemExit;
diff --git a/Lib/codeop.py b/Lib/codeop.py
index 6ad60e7f85098d..a0276b52d484e3 100644
--- a/Lib/codeop.py
+++ b/Lib/codeop.py
@@ -65,7 +65,7 @@ def _maybe_compile(compiler, source, filename, symbol):
 try:
 compiler(source + "\n", filename, symbol)
 return None
-except IncompleteInputError as e:
+except _IncompleteInputError as e:
 return None
 except SyntaxError as e:
 pass
diff --git a/Lib/test/exception_hierarchy.txt b/Lib/test/exception_hierarchy.txt
index 65f54859e2a21d..5e83faab9a6158 100644
--- a/Lib/test/exception_hierarchy.txt
+++ b/Lib/test/exception_hierarchy.txt
@@ -45,7 +45,7 @@ BaseException
   ├── StopAsyncIteration
   ├── StopIteration
   ├── SyntaxError
-  │└── IncompleteInputError
+  │└── _IncompleteInputError
   │└── IndentationError
   │ └── TabError
   ├── SystemError
diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py
index 19f977971570b7..49aa4b386039ec 100644
--- a/Lib/test/test_pickle.py
+++ b/Lib/test/test_pickle.py
@@ -569,7 +569,7 @@ def test_exceptions(self):
EncodingWarning,
BaseExceptionGroup,
ExceptionGroup,
-   IncompleteInputError):
+   _IncompleteInputError):
 continue
 if exc is not OSError and issubclass(exc, OSError):
 self.assertEqual(reverse_mapping('builtins', name),
diff --git a/Lib/test/test_stable_abi_ctypes.py 
b/Lib/test/test_stable_abi_ctypes.py
index c06c285c5013a6..21e2c6f1d2c30e 100644
--- a/Lib/test/test_stable_abi_ctypes.py
+++ b/Lib/test/test_stable_abi_ctypes.py
@@ -267,7 +267,6 @@ def test_windows_feature_macros(self):
 "PyExc_IOError",
 "PyExc_ImportError",
 "PyExc_ImportWarning",
-"PyExc_IncompleteInputError",
 "PyExc_IndentationError",
 "PyExc_IndexError",
 "PyExc_InterruptedError",
diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml
index 77473662aaa76c..34c3a539eb9b3e 100644
--- a/Misc/stable_abi.toml
+++ b/Misc/stable_abi.toml
@@ -2480,8 +2480,6 @@
 [function._

[Python-checkins] [3.13] gh-120860: Fix a few bugs in `type_setattro` error paths. (GH-120861) (#120963)

2024-06-24 Thread colesbury

https://github.com/python/cpython/commit/6aee5edb84cfa23f430091270a4118e51894c767
commit: 6aee5edb84cfa23f430091270a4118e51894c767
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: colesbury 
date: 2024-06-24T18:33:39Z
summary:

[3.13] gh-120860: Fix a few bugs in `type_setattro` error paths. (GH-120861) 
(#120963)

Moves the logic to update the type's dictionary to its own function in order
to make the lock scoping more clear.

Also, ensure that `name` is decref'd on the error path.
(cherry picked from commit dee63cb35971b87a09ddda5d6f29cd941f570720)

Co-authored-by: Sam Gross 

files:
M Objects/typeobject.c

diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 964e0cd9340f0c..63902ebc1f9869 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -5465,6 +5465,42 @@ _Py_type_getattro(PyObject *type, PyObject *name)
 return _Py_type_getattro_impl((PyTypeObject *)type, name, NULL);
 }
 
+static int
+type_update_dict(PyTypeObject *type, PyDictObject *dict, PyObject *name,
+ PyObject *value, PyObject **old_value)
+{
+// We don't want any re-entrancy between when we update the dict
+// and call type_modified_unlocked, including running the destructor
+// of the current value as it can observe the cache in an inconsistent
+// state.  Because we have an exact unicode and our dict has exact
+// unicodes we know that this will all complete without releasing
+// the locks.
+if (_PyDict_GetItemRef_Unicode_LockHeld(dict, name, old_value) < 0) {
+return -1;
+}
+
+/* Clear the VALID_VERSION flag of 'type' and all its
+subclasses.  This could possibly be unified with the
+update_subclasses() recursion in update_slot(), but carefully:
+they each have their own conditions on which to stop
+recursing into subclasses. */
+type_modified_unlocked(type);
+
+if (_PyDict_SetItem_LockHeld(dict, name, value) < 0) {
+PyErr_Format(PyExc_AttributeError,
+ "type object '%.50s' has no attribute '%U'",
+ ((PyTypeObject*)type)->tp_name, name);
+_PyObject_SetAttributeErrorContext((PyObject *)type, name);
+return -1;
+}
+
+if (is_dunder_name(name)) {
+return update_slot(type, name);
+}
+
+return 0;
+}
+
 static int
 type_setattro(PyObject *self, PyObject *name, PyObject *value)
 {
@@ -5507,12 +5543,11 @@ type_setattro(PyObject *self, PyObject *name, PyObject 
*value)
 assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_INLINE_VALUES));
 assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_MANAGED_DICT));
 
-PyObject *old_value;
+PyObject *old_value = NULL;
 PyObject *descr = _PyType_LookupRef(metatype, name);
 if (descr != NULL) {
 descrsetfunc f = Py_TYPE(descr)->tp_descr_set;
 if (f != NULL) {
-old_value = NULL;
 res = f(descr, (PyObject *)type, value);
 goto done;
 }
@@ -5528,47 +5563,16 @@ type_setattro(PyObject *self, PyObject *name, PyObject 
*value)
 }
 END_TYPE_LOCK();
 if (dict == NULL) {
-return -1;
+res = -1;
+goto done;
 }
 }
 
-// We don't want any re-entrancy between when we update the dict
-// and call type_modified_unlocked, including running the destructor
-// of the current value as it can observe the cache in an inconsistent
-// state.  Because we have an exact unicode and our dict has exact
-// unicodes we know that this will all complete without releasing
-// the locks.
 BEGIN_TYPE_DICT_LOCK(dict);
-
-if (_PyDict_GetItemRef_Unicode_LockHeld((PyDictObject *)dict, name, 
&old_value) < 0) {
-return -1;
-}
-
-/* Clear the VALID_VERSION flag of 'type' and all its
-subclasses.  This could possibly be unified with the
-update_subclasses() recursion in update_slot(), but carefully:
-they each have their own conditions on which to stop
-recursing into subclasses. */
-type_modified_unlocked(type);
-
-res = _PyDict_SetItem_LockHeld((PyDictObject *)dict, name, value);
-
-if (res == 0) {
-if (is_dunder_name(name)) {
-res = update_slot(type, name);
-}
-}
-else if (PyErr_ExceptionMatches(PyExc_KeyError)) {
-PyErr_Format(PyExc_AttributeError,
-"type object '%.50s' has no attribute '%U'",
-((PyTypeObject*)type)->tp_name, name);
-
-_PyObject_SetAttributeErrorContext((PyObject *)type, name);
-}
-
+res = type_update_dict(type, (PyDictObject *)dict, name, value, 
&old_value);
 assert(_PyType_CheckConsistency(type));
-
 END_TYPE_DICT_LOCK();
+
 done:
 Py_DECREF(name);
 Py_XDECREF(descr);

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to

[Python-checkins] [3.13] gh-120858: PyDict_Next should not lock the dict (GH-120859) (#120964)

2024-06-24 Thread colesbury

https://github.com/python/cpython/commit/0a77058b7971ba140c1f9db443e536ea5f1d71aa
commit: 0a77058b7971ba140c1f9db443e536ea5f1d71aa
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: colesbury 
date: 2024-06-24T18:41:19Z
summary:

[3.13] gh-120858: PyDict_Next should not lock the dict (GH-120859) (#120964)

PyDict_Next no longer locks the dictionary in the free-threaded build. Locking
around individual PyDict_Next calls is not sufficient because the function
returns borrowed references and because it allows concurrent modifications
during the iteraiton loop.

The internal locking also interferes with correct external synchronization
because it may suspend outer critical sections created by the caller.
(cherry picked from commit 375b723d5873f948696c7e85a97f4778d9e00ff0)

Co-authored-by: Sam Gross 

files:
A Misc/NEWS.d/next/C API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst
M Doc/c-api/dict.rst
M Doc/howto/free-threading-extensions.rst
M Objects/dictobject.c

diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst
index 49a78583a6fe26..b4fdf47bc915bd 100644
--- a/Doc/c-api/dict.rst
+++ b/Doc/c-api/dict.rst
@@ -290,6 +290,17 @@ Dictionary Objects
   Py_DECREF(o);
   }
 
+   The function is not thread-safe in the :term:`free-threaded `
+   build without external synchronization.  You can use
+   :c:macro:`Py_BEGIN_CRITICAL_SECTION` to lock the dictionary while iterating
+   over it::
+
+  Py_BEGIN_CRITICAL_SECTION(self->dict);
+  while (PyDict_Next(self->dict, &pos, &key, &value)) {
+  ...
+  }
+  Py_END_CRITICAL_SECTION();
+
 
 .. c:function:: int PyDict_Merge(PyObject *a, PyObject *b, int override)
 
diff --git a/Doc/howto/free-threading-extensions.rst 
b/Doc/howto/free-threading-extensions.rst
index 080170de1e5d16..1ba91b09516f9c 100644
--- a/Doc/howto/free-threading-extensions.rst
+++ b/Doc/howto/free-threading-extensions.rst
@@ -114,6 +114,24 @@ Containers like :c:struct:`PyListObject`,
 in the free-threaded build.  For example, the :c:func:`PyList_Append` will
 lock the list before appending an item.
 
+.. _PyDict_Next:
+
+``PyDict_Next``
+'''
+
+A notable exception is :c:func:`PyDict_Next`, which does not lock the
+dictionary.  You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect
+the dictionary while iterating over it if the dictionary may be concurrently
+modified::
+
+Py_BEGIN_CRITICAL_SECTION(dict);
+PyObject *key, *value;
+Py_ssize_t pos = 0;
+while (PyDict_Next(dict, &pos, &key, &value)) {
+...
+}
+Py_END_CRITICAL_SECTION();
+
 
 Borrowed References
 ===
@@ -141,7 +159,7 @@ that return :term:`strong references `.
 +---+---+
 | :c:func:`PyDict_SetDefault`   | :c:func:`PyDict_SetDefaultRef`|
 +---+---+
-| :c:func:`PyDict_Next` | no direct replacement |
+| :c:func:`PyDict_Next` | none (see :ref:`PyDict_Next`) |
 +---+---+
 | :c:func:`PyWeakref_GetObject` | :c:func:`PyWeakref_GetRef`|
 +---+---+
diff --git a/Misc/NEWS.d/next/C 
API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst b/Misc/NEWS.d/next/C 
API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst
new file mode 100644
index 00..b5df2a567b9da8
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2024-06-21-16-41-21.gh-issue-120858.Z5_-Mn.rst 
@@ -0,0 +1,3 @@
+:c:func:`PyDict_Next` no longer locks the dictionary in the free-threaded
+build.  The locking needs to be done by the caller around the entire iteration
+loop.
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index fa9d8ab44efdb2..27da631a574c4b 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2801,8 +2801,6 @@ _PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject 
**pkey,
 if (!PyDict_Check(op))
 return 0;
 
-ASSERT_DICT_LOCKED(op);
-
 mp = (PyDictObject *)op;
 i = *ppos;
 if (_PyDict_HasSplitTable(mp)) {
@@ -2875,11 +2873,7 @@ _PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject 
**pkey,
 int
 PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject **pkey, PyObject **pvalue)
 {
-int res;
-Py_BEGIN_CRITICAL_SECTION(op);
-res = _PyDict_Next(op, ppos, pkey, pvalue, NULL);
-Py_END_CRITICAL_SECTION();
-return res;
+return _PyDict_Next(op, ppos, pkey, pvalue, NULL);
 }
 
 

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] GH-119054: Add "Permissions and ownership" section to pathlib docs. (#120505)

2024-06-24 Thread barneygale

https://github.com/python/cpython/commit/e4a97a7fb1c03d3b6ec6efbeff553a0230e003c7
commit: e4a97a7fb1c03d3b6ec6efbeff553a0230e003c7
branch: main
author: Barney Gale 
committer: barneygale 
date: 2024-06-24T19:05:24Z
summary:

GH-119054: Add "Permissions and ownership" section to pathlib docs. (#120505)

Add dedicated subsection for `pathlib.owner()`, `group()`, `chmod()` and
`lchmod()`.

Co-authored-by: Hugo van Kemenade <[email protected]>

files:
M Doc/library/pathlib.rst

diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
index e585bcef915fbf..28a006d9619bbc 100644
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -1543,30 +1543,39 @@ Copying, renaming and deleting
Remove this directory.  The directory must be empty.
 
 
-Other methods
-^
+Permissions and ownership
+^
 
-.. classmethod:: Path.cwd()
+.. method:: Path.owner(*, follow_symlinks=True)
 
-   Return a new path object representing the current directory (as returned
-   by :func:`os.getcwd`)::
+   Return the name of the user owning the file. :exc:`KeyError` is raised
+   if the file's user identifier (UID) isn't found in the system database.
 
-  >>> Path.cwd()
-  PosixPath('/home/antoine/pathlib')
+   This method normally follows symlinks; to get the owner of the symlink, add
+   the argument ``follow_symlinks=False``.
 
+   .. versionchanged:: 3.13
+  Raises :exc:`UnsupportedOperation` if the :mod:`pwd` module is not
+  available. In earlier versions, :exc:`NotImplementedError` was raised.
 
-.. classmethod:: Path.home()
+   .. versionchanged:: 3.13
+  The *follow_symlinks* parameter was added.
 
-   Return a new path object representing the user's home directory (as
-   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
-   directory can't be resolved, :exc:`RuntimeError` is raised.
 
-   ::
+.. method:: Path.group(*, follow_symlinks=True)
 
-  >>> Path.home()
-  PosixPath('/home/antoine')
+   Return the name of the group owning the file. :exc:`KeyError` is raised
+   if the file's group identifier (GID) isn't found in the system database.
 
-   .. versionadded:: 3.5
+   This method normally follows symlinks; to get the group of the symlink, add
+   the argument ``follow_symlinks=False``.
+
+   .. versionchanged:: 3.13
+  Raises :exc:`UnsupportedOperation` if the :mod:`grp` module is not
+  available. In earlier versions, :exc:`NotImplementedError` was raised.
+
+   .. versionchanged:: 3.13
+  The *follow_symlinks* parameter was added.
 
 
 .. method:: Path.chmod(mode, *, follow_symlinks=True)
@@ -1589,57 +1598,52 @@ Other methods
.. versionchanged:: 3.10
   The *follow_symlinks* parameter was added.
 
-.. method:: Path.expanduser()
 
-   Return a new path with expanded ``~`` and ``~user`` constructs,
-   as returned by :meth:`os.path.expanduser`. If a home directory can't be
-   resolved, :exc:`RuntimeError` is raised.
+.. method:: Path.lchmod(mode)
 
-   ::
+   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
+   symbolic link's mode is changed rather than its target's.
 
-  >>> p = PosixPath('~/films/Monty Python')
-  >>> p.expanduser()
-  PosixPath('/home/eric/films/Monty Python')
 
-   .. versionadded:: 3.5
+Other methods
+^
 
+.. classmethod:: Path.cwd()
 
-.. method:: Path.group(*, follow_symlinks=True)
+   Return a new path object representing the current directory (as returned
+   by :func:`os.getcwd`)::
 
-   Return the name of the group owning the file. :exc:`KeyError` is raised
-   if the file's gid isn't found in the system database.
+  >>> Path.cwd()
+  PosixPath('/home/antoine/pathlib')
 
-   This method normally follows symlinks; to get the group of the symlink, add
-   the argument ``follow_symlinks=False``.
 
-   .. versionchanged:: 3.13
-  Raises :exc:`UnsupportedOperation` if the :mod:`grp` module is not
-  available. In previous versions, :exc:`NotImplementedError` was raised.
+.. classmethod:: Path.home()
 
-   .. versionchanged:: 3.13
-  The *follow_symlinks* parameter was added.
+   Return a new path object representing the user's home directory (as
+   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
+   directory can't be resolved, :exc:`RuntimeError` is raised.
 
+   ::
 
-.. method:: Path.lchmod(mode)
+  >>> Path.home()
+  PosixPath('/home/antoine')
 
-   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
-   symbolic link's mode is changed rather than its target's.
+   .. versionadded:: 3.5
 
 
-.. method:: Path.owner(*, follow_symlinks=True)
+.. method:: Path.expanduser()
 
-   Return the name of the user owning the file. :exc:`KeyError` is raised
-   if the file's uid isn't found in the system database.
+   Return a new path with expanded ``~`` and ``~user`` constructs,
+   as returned by :meth:`os.path.expanduser`. If a home directory can

[Python-checkins] [3.13] GH-119054: Add "Permissions and ownership" section to pathlib docs. (GH-120505) (#120967)

2024-06-24 Thread barneygale

https://github.com/python/cpython/commit/4619f892966cd393395db221f936340eadce17d2
commit: 4619f892966cd393395db221f936340eadce17d2
branch: 3.13
author: Barney Gale 
committer: barneygale 
date: 2024-06-24T20:32:08+01:00
summary:

[3.13] GH-119054: Add "Permissions and ownership" section to pathlib docs. 
(GH-120505) (#120967)

Add dedicated subsection for `pathlib.owner()`, `group()`, `chmod()` and
`lchmod()`.

(cherry picked from commit e4a97a7fb1c03d3b6ec6efbeff553a0230e003c7)

files:
M Doc/library/pathlib.rst

diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
index ebed9d50931061..3b48d25f96ddbc 100644
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -1475,30 +1475,39 @@ Renaming and deleting
Remove this directory.  The directory must be empty.
 
 
-Other methods
-^
+Permissions and ownership
+^
 
-.. classmethod:: Path.cwd()
+.. method:: Path.owner(*, follow_symlinks=True)
 
-   Return a new path object representing the current directory (as returned
-   by :func:`os.getcwd`)::
+   Return the name of the user owning the file. :exc:`KeyError` is raised
+   if the file's user identifier (UID) isn't found in the system database.
 
-  >>> Path.cwd()
-  PosixPath('/home/antoine/pathlib')
+   This method normally follows symlinks; to get the owner of the symlink, add
+   the argument ``follow_symlinks=False``.
 
+   .. versionchanged:: 3.13
+  Raises :exc:`UnsupportedOperation` if the :mod:`pwd` module is not
+  available. In earlier versions, :exc:`NotImplementedError` was raised.
 
-.. classmethod:: Path.home()
+   .. versionchanged:: 3.13
+  The *follow_symlinks* parameter was added.
 
-   Return a new path object representing the user's home directory (as
-   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
-   directory can't be resolved, :exc:`RuntimeError` is raised.
 
-   ::
+.. method:: Path.group(*, follow_symlinks=True)
 
-  >>> Path.home()
-  PosixPath('/home/antoine')
+   Return the name of the group owning the file. :exc:`KeyError` is raised
+   if the file's group identifier (GID) isn't found in the system database.
 
-   .. versionadded:: 3.5
+   This method normally follows symlinks; to get the group of the symlink, add
+   the argument ``follow_symlinks=False``.
+
+   .. versionchanged:: 3.13
+  Raises :exc:`UnsupportedOperation` if the :mod:`grp` module is not
+  available. In earlier versions, :exc:`NotImplementedError` was raised.
+
+   .. versionchanged:: 3.13
+  The *follow_symlinks* parameter was added.
 
 
 .. method:: Path.chmod(mode, *, follow_symlinks=True)
@@ -1522,57 +1531,51 @@ Other methods
   The *follow_symlinks* parameter was added.
 
 
-.. method:: Path.expanduser()
-
-   Return a new path with expanded ``~`` and ``~user`` constructs,
-   as returned by :meth:`os.path.expanduser`. If a home directory can't be
-   resolved, :exc:`RuntimeError` is raised.
+.. method:: Path.lchmod(mode)
 
-   ::
+   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
+   symbolic link's mode is changed rather than its target's.
 
-  >>> p = PosixPath('~/films/Monty Python')
-  >>> p.expanduser()
-  PosixPath('/home/eric/films/Monty Python')
 
-   .. versionadded:: 3.5
+Other methods
+^
 
+.. classmethod:: Path.cwd()
 
-.. method:: Path.group(*, follow_symlinks=True)
+   Return a new path object representing the current directory (as returned
+   by :func:`os.getcwd`)::
 
-   Return the name of the group owning the file. :exc:`KeyError` is raised
-   if the file's gid isn't found in the system database.
+  >>> Path.cwd()
+  PosixPath('/home/antoine/pathlib')
 
-   This method normally follows symlinks; to get the group of the symlink, add
-   the argument ``follow_symlinks=False``.
 
-   .. versionchanged:: 3.13
-  Raises :exc:`UnsupportedOperation` if the :mod:`grp` module is not
-  available. In previous versions, :exc:`NotImplementedError` was raised.
+.. classmethod:: Path.home()
 
-   .. versionchanged:: 3.13
-  The *follow_symlinks* parameter was added.
+   Return a new path object representing the user's home directory (as
+   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
+   directory can't be resolved, :exc:`RuntimeError` is raised.
 
+   ::
 
-.. method:: Path.lchmod(mode)
+  >>> Path.home()
+  PosixPath('/home/antoine')
 
-   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
-   symbolic link's mode is changed rather than its target's.
+   .. versionadded:: 3.5
 
 
-.. method:: Path.owner(*, follow_symlinks=True)
+.. method:: Path.expanduser()
 
-   Return the name of the user owning the file. :exc:`KeyError` is raised
-   if the file's uid isn't found in the system database.
+   Return a new path with expanded ``~`` and ``~user`` constructs,
+   as returned by :meth:`os.path.expanduser`. If a home directory can't be
+   resolved

[Python-checkins] [3.12] GH-119054: Add "Permissions and ownership" section to pathlib docs. (GH-120505) (#120968)

2024-06-24 Thread barneygale

https://github.com/python/cpython/commit/52bc99743c0c915c023e39fd714eb56c20c22c18
commit: 52bc99743c0c915c023e39fd714eb56c20c22c18
branch: 3.12
author: Barney Gale 
committer: barneygale 
date: 2024-06-24T20:32:24+01:00
summary:

[3.12] GH-119054: Add "Permissions and ownership" section to pathlib docs. 
(GH-120505) (#120968)

Add dedicated subsection for `pathlib.owner()`, `group()`, `chmod()` and
`lchmod()`.

(cherry picked from commit e4a97a7fb1c03d3b6ec6efbeff553a0230e003c7)

files:
M Doc/library/pathlib.rst

diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
index 4a62f6e77103e2..29d289aa533f6d 100644
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -1380,30 +1380,19 @@ Renaming and deleting
Remove this directory.  The directory must be empty.
 
 
-Other methods
-^
-
-.. classmethod:: Path.cwd()
-
-   Return a new path object representing the current directory (as returned
-   by :func:`os.getcwd`)::
-
-  >>> Path.cwd()
-  PosixPath('/home/antoine/pathlib')
-
+Ownership and permissions
+^
 
-.. classmethod:: Path.home()
+.. method:: Path.owner()
 
-   Return a new path object representing the user's home directory (as
-   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
-   directory can't be resolved, :exc:`RuntimeError` is raised.
+   Return the name of the user owning the file.  :exc:`KeyError` is raised
+   if the file's user identifier (UID) isn't found in the system database.
 
-   ::
 
-  >>> Path.home()
-  PosixPath('/home/antoine')
+.. method:: Path.group()
 
-   .. versionadded:: 3.5
+   Return the name of the group owning the file.  :exc:`KeyError` is raised
+   if the file's group identifier (GID) isn't found in the system database.
 
 
 .. method:: Path.chmod(mode, *, follow_symlinks=True)
@@ -1427,37 +1416,51 @@ Other methods
   The *follow_symlinks* parameter was added.
 
 
-.. method:: Path.expanduser()
+.. method:: Path.lchmod(mode)
 
-   Return a new path with expanded ``~`` and ``~user`` constructs,
-   as returned by :meth:`os.path.expanduser`. If a home directory can't be
-   resolved, :exc:`RuntimeError` is raised.
+   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
+   symbolic link's mode is changed rather than its target's.
 
-   ::
 
-  >>> p = PosixPath('~/films/Monty Python')
-  >>> p.expanduser()
-  PosixPath('/home/eric/films/Monty Python')
+Other methods
+^
 
-   .. versionadded:: 3.5
+.. classmethod:: Path.cwd()
 
+   Return a new path object representing the current directory (as returned
+   by :func:`os.getcwd`)::
 
-.. method:: Path.group()
+  >>> Path.cwd()
+  PosixPath('/home/antoine/pathlib')
 
-   Return the name of the group owning the file.  :exc:`KeyError` is raised
-   if the file's gid isn't found in the system database.
 
+.. classmethod:: Path.home()
 
-.. method:: Path.lchmod(mode)
+   Return a new path object representing the user's home directory (as
+   returned by :func:`os.path.expanduser` with ``~`` construct). If the home
+   directory can't be resolved, :exc:`RuntimeError` is raised.
 
-   Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
-   symbolic link's mode is changed rather than its target's.
+   ::
+
+  >>> Path.home()
+  PosixPath('/home/antoine')
 
+   .. versionadded:: 3.5
 
-.. method:: Path.owner()
 
-   Return the name of the user owning the file.  :exc:`KeyError` is raised
-   if the file's uid isn't found in the system database.
+.. method:: Path.expanduser()
+
+   Return a new path with expanded ``~`` and ``~user`` constructs,
+   as returned by :meth:`os.path.expanduser`. If a home directory can't be
+   resolved, :exc:`RuntimeError` is raised.
+
+   ::
+
+  >>> p = PosixPath('~/films/Monty Python')
+  >>> p.expanduser()
+  PosixPath('/home/eric/films/Monty Python')
+
+   .. versionadded:: 3.5
 
 
 .. method:: Path.readlink()

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] Add --with-lto back to Linux JIT CI (GH-120921)

2024-06-24 Thread brandtbucher

https://github.com/python/cpython/commit/fd0f814ade43fa479bfbe76dc226424db14a9354
commit: fd0f814ade43fa479bfbe76dc226424db14a9354
branch: main
author: Savannah Ostrowski 
committer: brandtbucher 
date: 2024-06-24T13:16:22-07:00
summary:

Add --with-lto back to Linux JIT CI (GH-120921)

files:
M .github/workflows/jit.yml

diff --git a/.github/workflows/jit.yml b/.github/workflows/jit.yml
index 8c760a81d52662..5e3ac9e9e0fada 100644
--- a/.github/workflows/jit.yml
+++ b/.github/workflows/jit.yml
@@ -133,17 +133,15 @@ jobs:
   make all --jobs 4
   ./python.exe -m test --multiprocess 0 --timeout 4500 --verbose2 
--verbose3
 
-  # --with-lto has been removed temporarily as a result of an open issue 
in LLVM 18 (see https://github.com/llvm/llvm-project/issues/87553)
   - name: Native Linux
 if: runner.os == 'Linux' && matrix.architecture == 'x86_64'
 run: |
   sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh 
${{ matrix.llvm }}
   export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
-  ./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations' }}
+  ./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations --with-lto' }}
   make all --jobs 4
   ./python -m test --multiprocess 0 --timeout 4500 --verbose2 
--verbose3
 
-  # --with-lto has been removed temporarily as a result of an open issue 
in LLVM 18 (see https://github.com/llvm/llvm-project/issues/87553)
   - name: Emulated Linux
 if: runner.os == 'Linux' && matrix.architecture != 'x86_64'
 # The --ignorefile on ./python -m test is used to exclude tests known 
to fail when running on an emulated Linux.
@@ -161,7 +159,7 @@ jobs:
   CC="${{ matrix.compiler == 'clang' && 'clang --target=$HOST' || 
'$HOST-gcc' }}" \
 CPP="$CC --preprocess" \
 HOSTRUNNER=qemu-${{ matrix.architecture }} \
-./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations ' }} --build=x86_64-linux-gnu 
--host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no 
ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
+./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations --with-lto' }} 
--build=x86_64-linux-gnu --host="$HOST" 
--with-build-python=../build/bin/python3 --with-pkg-config=no 
ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
   make all --jobs 4
   ./python -m test 
--ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 
--timeout 4500 --verbose2 --verbose3
 

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] Add --with-lto back to Linux JIT CI (GH-120972)

2024-06-24 Thread brandtbucher

https://github.com/python/cpython/commit/6bfcf98ff6f6bb4f04f84c330794640c6e38d8bf
commit: 6bfcf98ff6f6bb4f04f84c330794640c6e38d8bf
branch: 3.13
author: Miss Islington (bot) <[email protected]>
committer: brandtbucher 
date: 2024-06-24T20:44:46Z
summary:

[3.13] Add --with-lto back to Linux JIT CI (GH-120972)

(cherry picked from commit fd0f814ade43fa479bfbe76dc226424db14a9354)
Co-authored-by: Savannah Ostrowski 

files:
M .github/workflows/jit.yml

diff --git a/.github/workflows/jit.yml b/.github/workflows/jit.yml
index 8c760a81d52662..5e3ac9e9e0fada 100644
--- a/.github/workflows/jit.yml
+++ b/.github/workflows/jit.yml
@@ -133,17 +133,15 @@ jobs:
   make all --jobs 4
   ./python.exe -m test --multiprocess 0 --timeout 4500 --verbose2 
--verbose3
 
-  # --with-lto has been removed temporarily as a result of an open issue 
in LLVM 18 (see https://github.com/llvm/llvm-project/issues/87553)
   - name: Native Linux
 if: runner.os == 'Linux' && matrix.architecture == 'x86_64'
 run: |
   sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh 
${{ matrix.llvm }}
   export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
-  ./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations' }}
+  ./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations --with-lto' }}
   make all --jobs 4
   ./python -m test --multiprocess 0 --timeout 4500 --verbose2 
--verbose3
 
-  # --with-lto has been removed temporarily as a result of an open issue 
in LLVM 18 (see https://github.com/llvm/llvm-project/issues/87553)
   - name: Emulated Linux
 if: runner.os == 'Linux' && matrix.architecture != 'x86_64'
 # The --ignorefile on ./python -m test is used to exclude tests known 
to fail when running on an emulated Linux.
@@ -161,7 +159,7 @@ jobs:
   CC="${{ matrix.compiler == 'clang' && 'clang --target=$HOST' || 
'$HOST-gcc' }}" \
 CPP="$CC --preprocess" \
 HOSTRUNNER=qemu-${{ matrix.architecture }} \
-./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations ' }} --build=x86_64-linux-gnu 
--host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no 
ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
+./configure --enable-experimental-jit ${{ matrix.debug && 
'--with-pydebug' || '--enable-optimizations --with-lto' }} 
--build=x86_64-linux-gnu --host="$HOST" 
--with-build-python=../build/bin/python3 --with-pkg-config=no 
ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
   make all --jobs 4
   ./python -m test 
--ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 
--timeout 4500 --verbose2 --verbose3
 

___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]

[Python-checkins] [3.13] gh-120683: Fix an error in logging.LogRecord timestamp (GH-120709) (GH-120933)

[Python-checkins] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (#112191)

[Python-checkins] [3.12] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120936)

[Python-checkins] [3.13] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120935)

[Python-checkins] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909)

[Python-checkins] gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835)

[Python-checkins] [3.12] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909) (GH-120939)

[Python-checkins] [3.13] gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909) (GH-120938)

[Python-checkins] gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)

[Python-checkins] gh-120373: Mark test_audit.test_http as requiring the network resource (#120374)

[Python-checkins] docs: puremagic.what() as replacement for imghdr.what() (#120871)

[Python-checkins] [3.12] gh-120373: Mark test_audit.test_http as requiring the network resource (GH-120374) (#120949)

[Python-checkins] [3.13] gh-120373: Mark test_audit.test_http as requiring the network resource (GH-120374) (#120948)

[Python-checkins] [3.13] docs: puremagic.what() as replacement for imghdr.what() (GH-120871) (#120951)

[Python-checkins] gh-70278: Fix PyUnicode_FromFormat() with precision for %s and %V (GH-120365)

[Python-checkins] gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)

[Python-checkins] GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854)

[Python-checkins] gh-119182: Add PyUnicodeWriter_WriteUCS4() function (#120849)

[Python-checkins] Fixes loop variables to be the same types as their limit (GH-120958)

[Python-checkins] gh-120956: Avoid comparison of int to Py_ssize_t in parser (#120959)

[Python-checkins] gh-120834: fix over-allocation in PyGenObject, PyCoroObject, PyAsyncGenObject. (#120941)

[Python-checkins] Fix typos in comments (#120821)

[Python-checkins] gh-120860: Fix a few bugs in `type_setattro` error paths. (#120861)

[Python-checkins] gh-120858: PyDict_Next should not lock the dict (#120859)

[Python-checkins] [3.13] gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680, GH-120955) (GH-120944)

[Python-checkins] [3.13] gh-120860: Fix a few bugs in `type_setattro` error paths. (GH-120861) (#120963)

[Python-checkins] [3.13] gh-120858: PyDict_Next should not lock the dict (GH-120859) (#120964)

[Python-checkins] GH-119054: Add "Permissions and ownership" section to pathlib docs. (#120505)

[Python-checkins] [3.13] GH-119054: Add "Permissions and ownership" section to pathlib docs. (GH-120505) (#120967)

[Python-checkins] [3.12] GH-119054: Add "Permissions and ownership" section to pathlib docs. (GH-120505) (#120968)

[Python-checkins] Add --with-lto back to Linux JIT CI (GH-120921)

[Python-checkins] [3.13] Add --with-lto back to Linux JIT CI (GH-120972)

32 matches

Site Navigation

Mail list logo

Footer information