Re: [Python-Dev] Best Python API for exposing posix_spawn

2018-01-10 Thread Pablo Galindo Salgado
I think I really like Antoine's suggestion so I'm going to finish implementing it that way. I think this keeps the API simple, does not bring in the os module new dependencies, keeps the C implementation clean and is consistent with the rest of the posix module. I will post an update when is

[Python-Dev] Best Python API for exposing posix_spawn

2018-01-08 Thread Pablo Galindo Salgado
Hi, I'm currently working on exposing posix_spawn in the posix module (and by extension in the os module). You can find the initial implementation in this PR: https://github.com/python/cpython/pull/5109 As pointed out by Gregory P. Smith, some changes are needed in the way the file_actions

Re: [Python-Dev] Best Python API for exposing posix_spawn

2018-01-16 Thread Pablo Galindo Salgado
if possible. Thanks you very much everyone for your time and suggestions! On Wed, 10 Jan 2018, 09:17 Pablo Galindo Salgado, <pablog...@gmail.com> wrote: > I think I really like Antoine's suggestion so I'm going to finish > implementing it that way. I think this keeps the API

[Python-Dev] Exposing different versions of a system call in Python

2018-01-19 Thread Pablo Galindo Salgado
Hello everyone, In today's episode of exposing useful Linux system calls I am exposing preadv2 in this PR: https://github.com/python/cpython/pull/5239 as requested in this issue: https://bugs.python.org/issue31368 As njsmith has commented in the PR, preadv2 only exists because regular preadv

[Python-Dev] sys.settrace does not produce events for C functions

2018-01-21 Thread Pablo Galindo Salgado
The docs for sys.settrace mention that: >> event is a string: 'call', 'line', 'return', 'exception', >> 'c_call', 'c_return', or 'c_exception' But in the code for ceval.c the only point where call_trace is invoked with PyTrace_C_CALL or PyTrace_C_RETURN is under the C_TRACE macro. In this macro

[Python-Dev] building extensions as builtins is broken in 3.7

2018-04-02 Thread Pablo Galindo Salgado
Hi, I am working on the release blocker https://bugs.python.org/issue32232. I tried to apply the patch proposed by Matthias Klose and I found that it works on Unix but it fails to build on Windows (probably because circular imports). I tried to add some tests but after some work on the problem

Re: [Python-Dev] Usage of the multiprocessing API and object lifetime

2018-12-11 Thread Pablo Galindo Salgado
> > Pablo's issue35378 evolved to add a weak reference in iterators to try > > to detect when the Pool is destroyed: raise an exception from the > > iterator, if possible. > That's an ok fix for me. I am playing with weakreferences inside the iterator and result objects, but this may not be

Re: [Python-Dev] Usage of the multiprocessing API and object lifetime

2018-12-11 Thread Pablo Galindo Salgado
reason I was experimenting with the weakreference. On Tue, 11 Dec 2018 at 18:37, Paul Moore wrote: > On Tue, 11 Dec 2018 at 17:50, Pablo Galindo Salgado > wrote: > > I agree that misusage of the pool should not be encouraged but in this > situation the fact that > > t

[Python-Dev] PEP 570

2019-03-28 Thread Pablo Galindo Salgado
into two forums (discourse and python-dev), we kindly ask you to go to discourse if you want to participate in the debate :) Here is the full document of the PEP: https://www.python.org/dev/peps/pep-0570/ Pablo Galindo Salgado ___ Python-Dev mailing list

Re: [Python-Dev] Parser module in the stdlib

2019-05-16 Thread Pablo Galindo Salgado
cal new package (with some new API over it may be) is already undocumented as an implementation detail of lib2to3 (and some people are already using it directly). On Thu, 16 May 2019 at 23:41, Nathaniel Smith wrote: > On Thu, May 16, 2019 at 2:13 PM Pablo Galindo Salgado > wrote: > >

[Python-Dev] Parser module in the stdlib

2019-05-16 Thread Pablo Galindo Salgado
Hi everyone, TLDR = I propose to remove the current parser module and expose pgen2 as a standard library module. Some context === The parser module has been "deprecated" (technically we recommend to prefer the ast module instead) since Python2.5 but is still in the standard

[Python-Dev] Recent buildbot reports and asyncio test failures

2019-06-03 Thread Pablo Galindo Salgado
several timezones is very difficult to get them all. Thanks to everyone that is helping solving these bugs :) Regards from sunny London, Pablo Galindo Salgado ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

2019-06-01 Thread Pablo Galindo Salgado
> > I propose to make co_argcount meaning the number of positional > parameters (i.e. positional-only + positional-or-keyword). This would > remove the need of changing the code that uses co_argcount. > I like the proposal, it will certainly make handling normal cases downstream much easier

Re: [Python-Dev] Parser module in the stdlib

2019-05-20 Thread Pablo Galindo Salgado
mail. Perhaps we can be more productive if we focus on just deprecating the "parser" module, but I thought it was an opportunity to solve two (related) problems at once. On Mon, 20 May 2019 at 17:28, Guido van Rossum wrote: > On Thu, May 16, 2019 at 3:51 PM Pablo Galindo Salgado > wrote

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
I think this summarizes the situation very well :) https://xkcd.com/1172/ On Fri, 5 Jul 2019 at 22:28, Pablo Galindo Salgado wrote: > Hi, > > Recently, we moved the optimization for the removal of dead code of the > form > > if 0: > > > to the ast so we use JU

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
an "if 1:" block. Ok, these were very good points. I am convinced. I agree that the better thing to do at this pointis to revert this until we can think this a bit more :) I have made a PR to revert the change. On Fri, 5 Jul 2019 at 23:27, Tim Peters wrote: > [Pablo Galindo Salgado ]

[Python-Dev] Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
Hi, Recently, we moved the optimization for the removal of dead code of the form if 0: to the ast so we use JUMP bytecodes instead (being completed in PR14116). The reason is that currently, any syntax error in the block will never be reported. For example: if 0: return if 1:

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
> Until a solution which makes everyone happy can be found, I suggest to move back to the status quo: revert the change. >More people seems to expect "if 0: ..." to be removed, than people who care of syntax errors on "if 0". I don't think this is that clear. As Paul wrote on the issue this is

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
fundamentally wrong (which is possible as is late here) is very simple and covers all the cases. It would be great if someone could review the PR to check if I missed anything with this argument. On Fri, 5 Jul 2019 at 22:28, Pablo Galindo Salgado wrote: > Hi, > > Recently, we moved the opt

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Pablo Galindo Salgado
Thanks Terry for your comments! > I presume that most of us agree that the last option, doing both, would be > best, or at least agreeable. I created a PR reverting the change but I (may) have found a way of doing both things :) ___ Python-Dev

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-08 Thread Pablo Galindo Salgado
>. If it doesn't, could this one optimization be left in the peephole optimizer at bytecode level? Sadly no, because at that time there is not enough information left to do things correctly. The problem manifest with constructs like if something or __debug__: ... You cannot just look at the

[Python-Dev] Multiple reference leaks in master

2019-12-03 Thread Pablo Galindo Salgado
London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives

[Python-Dev] Re: Multiple reference leaks in master

2019-12-03 Thread Pablo Galindo Salgado
, Guido van Rossum wrote: > How do I find the refleak buildbots? I went to the devguide and searched > for "buildbot", which pointed to https://www.python.org/dev/buildbot/ -- > but searching there for "refleak" finds nothing. > > On Tue, Dec 3, 2019 at 1:16 PM

[Python-Dev] Re: Multiple reference leaks in master

2019-12-03 Thread Pablo Galindo Salgado
frustrating that they are so hard to find. > > On Tue, Dec 3, 2019 at 2:17 PM Pablo Galindo Salgado > wrote: > >> > How do I find the refleak buildbots? >> >> In this page: >> >> https://buildbot.python.org/all/#/builders >> >> all the bui

[Python-Dev] Exposing Tools/parser/unparse.py in the stdlib?

2019-11-18 Thread Pablo Galindo Salgado
that I am not contemplating? Regards from rainy London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org

[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

2019-11-20 Thread Pablo Galindo Salgado
Opened https://bugs.python.org/issue38870 to track this. On Tue, 19 Nov 2019 at 00:40, Pablo Galindo Salgado wrote: > Hi, > > What do people feel about exposing Tools/parser/unparse.py in the standard > library? Here is my initial rationale: > > * The tool already need

[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

2019-11-19 Thread Pablo Galindo Salgado
uarantee anything other than roundtrip over the generated source initially. > I would prefer to keep a separated module, like "import ast.unparse" or "import unparse". Why? I think ast.unparse is a natural fit. It will likely be only one function exposed. On Tue, 19 Nov 201

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-16 Thread Pablo Galindo Salgado
After the feedback received in the language summit, we have made a modification to the proposed migration plan in PEP 617 so the new parser will be the default in 3.9alpha6: https://github.com/python/peps/pull/1369 ___ Python-Dev mailing list --

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-05 Thread Pablo Galindo Salgado
> The only thing I'm missing from the PEP is more detail about how the cross-language nature of the parser actions are handled. Expanded the "actions" section in the PEP here: https://github.com/python/peps/pull/1357 ___ Python-Dev mailing list --

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-02 Thread Pablo Galindo Salgado
> Just to clarify, this means that 3.9 will ship with the PEG parser as default, > right? If so, this would be a new feature, post beta. Since that is counter > to our > general policy, we would need to get explicit RM approval for such a change. The idea is merging it *before beta* and make

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-02 Thread Pablo Galindo Salgado
> About the migration, can I ask who is going to (help to) fix projects which rely on the AST? I think you misunderstood: The AST is exactly the same as the old and the new parser. The only the thing that the new parser does is not generate an immediate CST (Concrete Syntax Tree) and that is

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-02 Thread Pablo Galindo Salgado
> About the migration, can I ask who is going to (help to) fix projects which rely on the AST? Whoops, I send the latest email before finishing it by mistake. Here is the extended version of the answer: I think there is a misunderstanding here: The new parser generates the same AST as the old

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-03 Thread Pablo Galindo Salgado
> That paragraph seems rather confused. I think what it might be > trying to say is that a PEG parser allows you to write productions > with overlapping first sets (which would be "ambiguous" for an > LL parser), but still somehow guarantees that a unique parse tree > is produced. The latter

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-03 Thread Pablo Galindo Salgado
>The only thing I'm missing from the PEP is more detail about how the > cross-language nature of the parser actions are handled. The example covers > just C, and the description of the actions says they're C expressions. The > only mention of Python code generation is for alternatives without

[Python-Dev] Re: Detect memory leaks in unit tests

2020-05-13 Thread Pablo Galindo Salgado
> But again this is for PyObjects only. Not really, we check also memory blocks: https://github.com/python/cpython/blob/master/Lib/test/libregrtest/refleak.py#L72 as long as you don't directly call malloc and use one of the Python specific APIs like PyMem_Malloc then the reflect code should

[Python-Dev] Re: Hygenic macros PEP.

2020-09-15 Thread Pablo Galindo Salgado
Thanks for the proposal Mark! I wanted to make some comments regarding converting AST nodes to PyObjects internally. I see some challenges here: * Not using an arena allocator for the nodes can introduce more challenges than simplifications. The first is that deleting a deep tree currently is

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Pablo Galindo Salgado
> > > * Not using an arena allocator for the nodes can introduce more challenges >> than simplifications. The first is that deleting a deep tree currently is >> just freeing the arena block, while if the nodes were PyObjects it will >> involve recursive destruction. That could potentially segfault

[Python-Dev] Re: PEP 617 -- New PEG parser for CPython

2020-10-08 Thread Pablo Galindo Salgado
Our experience with automatic testing is that unfortunately is very difficult to extract real problems with it. We tried some of the new experimental source generators on top of hypothesis ( https://pypi.org/project/hypothesmith/) and sadly we could not catch many important things that parsing

[Python-Dev] Re: [python-committers] Thank you Larry Hastings!

2020-10-05 Thread Pablo Galindo Salgado
As someone that went through doing a release just now and now what it entailsthanks a lot for all the work, Larry! :) On Mon, 5 Oct 2020 at 19:39, Barry Warsaw wrote: > They say being a Python Release Manager is a thankless job, so the Python > Secret Underground (PSU), which emphatically

[Python-Dev] Re: Modified parser does not seem to work. What am I doing wrong?

2020-10-16 Thread Pablo Galindo Salgado
Hi Stefano, One of the problems you have is that the rule for slices has a negative lookahead for the comma: slices[expr_ty]: | a=slice !',' { a } IIRC the reason that is there is to allow "x[3,]" to be parsed. Also, to allow "a[k=3]" you need to to create a rule that allows skipping the

[Python-Dev] Re: [RELEASE] Python 3.9.0 is now available, and you can already test 3.10.0a1!

2020-10-09 Thread Pablo Galindo Salgado
IRC. Do you know if the link to the file you mentioned used to be there? Thanks, Pablo On Thu, 8 Oct 2020, 09:25 Miro Hrončok, wrote: > On 05. 10. 20 22:22, Łukasz Langa wrote: > > In fact, our newest Release Manager, Pablo Galindo Salgado, prepared the > first > > alpha releas

[Python-Dev] Re: [python-committers] Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
d > the figures, and moved on, but maybe I was wrong to do so... > > Paul > > On Wed, 14 Oct 2020 at 14:17, Pablo Galindo Salgado > wrote: > > > > Hi! > > > > I have updated the branch benchmarks in the pyperformance server and now > they

[Python-Dev] Re: [python-committers] Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
tps://github.com/python/pyperformance/blob/master/pyperformance/benchmarks/bm_regex_dna.py > > Thanks. > > On 14.10.2020 15:16, Pablo Galindo Salgado wrote: > > Hi! > > > > I have updated the branch benchmarks in the pyperformance server and now > they >

[Python-Dev] Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
meline" tabs ( https://speed.python.org/timeline/). * Once the daily builds are working as expected, I plan to work on trying to automatically comment or PRs or on bpo if we detect that a commit has introduced some notable performance regression. Regards from sunny London, Pablo Galindo

[Python-Dev] Re: [python-committers] Re: Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
use the micro-benchmarks published in the What's new of 3.9 were confusing a lot of users that were thinking if 3.9 was slower. On Wed, 14 Oct 2020 at 15:14, Antoine Pitrou wrote: > > Le 14/10/2020 à 15:16, Pablo Galindo Salgado a écrit : > > Hi! > > > > I have up

[Python-Dev] Re: [python-committers] Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
ing :) That's why from now on I am trying to invest in daily builds for master, so we can answer that exact question if we detect regressions in the future. On Wed, 14 Oct 2020 at 15:04, M.-A. Lemburg wrote: > On 14.10.2020 16:00, Pablo Galindo Salgado wrote: > >> Would it be possible

[Python-Dev] Re: [python-committers] Re: Performance benchmarks for 3.9

2020-10-14 Thread Pablo Galindo Salgado
18:58, Chris Jerdonek wrote: > MOn Wed, Oct 14, 2020 at 8:03 AM Pablo Galindo Salgado < > pablog...@gmail.com> wrote: > >> > Would it be possible rerun the tests with the current >> setup for say the last 1000 revisions or perhaps a subset of these >> (e.g. every

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Pablo Galindo Salgado
> > Don't we need to do all of this in the _ast module, already? > > We already have an AST composed of Python objects We have AST of Python objects, but the python versions are not used internally, especially in the parser, where they are created. The parser and the compiler currently use

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Pablo Galindo Salgado
Sep 2020 at 12:48, Mark Shannon wrote: > > > On 16/09/2020 12:22 pm, Pablo Galindo Salgado wrote: > > > Don't we need to do all of this in the _ast module, already? > > > We already have an AST composed of Python objects > > > > > > We have AST o

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

2020-06-25 Thread Pablo Galindo Salgado
I was talking with a colleague today about the PEP and he raised a couple of question regarding the match protocol and the proxy result. One question is that taking into account that 'case object(x)' is valid for every object, but it does (could do) something different for objects that have a

[Python-Dev] Re: Can we stop adding to the C API, please?

2020-06-03 Thread Pablo Galindo Salgado
Just some comments on the GC stuff as I added them myself. > Shouldn't GC track *all* objects? No, extension types need to opt-in to the garbage collector and if so, implement the interface. > Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing internal implementation

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-17 Thread Pablo Galindo Salgado
I like the proposal in general but I am against removing lnotab. The reason is that many tools rely on reading this attribute to figure out the Python call stack information. For instance, many sampler profilers read this memory by using ptrace or process_vm_readv and they cannot execute any code

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-23 Thread Pablo Galindo Salgado
> In theory, this table could be stored somewhere other than the code object, so that it doesn't actually get paged in or occupy cache unless tracing is on. As some of us mentioned before, that will hurt the ecosystem of profilers and debugger tools considerably On Thu, 23 Jul 2020 at 18:08,

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-11 Thread Pablo Galindo Salgado
This may be related to the changes in https://bugs.python.org/issue42246. Could you open a new issue and add Mark Shannon to it if that turns to be the case? Pablo On Mon, 11 Jan 2021 at 19:36, Chris Angelico wrote: > On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann wrote: > > > > > > On 1/8/21

[Python-Dev] [RELEASE] Python 3.9.1 is now available, together with 3.10.0a3 and 3.8.7rc1

2020-12-07 Thread Pablo Galindo Salgado
It's starting to get very cold (at least on the Northern hemisphere) so we have been carefully packaging a total of three new Python releases to keep you warm these days! Python 3.9.1 is the first maintenance release of Python 3.9, and also the first version of Python to support macOS 11 Big Sur

[Python-Dev] CPython codebase plots

2020-12-21 Thread Pablo Galindo Salgado
://discuss.python.org/t/cpython-codebase-plots/6267 Regards from cloudy London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python

[Python-Dev] [RELEASE] Python 3.10.0a2 available for testing

2020-11-03 Thread Pablo Galindo Salgado
of these infinities are bigger than others. These infinite cardinalities normally are represented using aleph numbers. Infinite sets are strange beasts indeed. Regards from cold London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org

[Python-Dev] Re: Accepting PEP 626

2020-10-29 Thread Pablo Galindo Salgado
ty sure it will be negligible impact and the performance check should be just a routine confirmation. Cheers, Pablo On Thu, 29 Oct 2020, 09:47 Mark Shannon, wrote: > Hi, > > That's great. Thanks Pablo. > > On 29/10/2020 1:32 am, Pablo Galindo Salgado wrote: > > On beha

[Python-Dev] Re: Accepting PEP 626

2020-10-29 Thread Pablo Galindo Salgado
. Cheers, Pablo On Thu, 29 Oct 2020 at 10:55, Mark Shannon wrote: > Hi Pablo, > > On 29/10/2020 9:56 am, Pablo Galindo Salgado wrote: > > > Performance compared to what? > > > > Compared before the patch. The comparison that I mentioned is before and > > a

[Python-Dev] Accepting PEP 626

2020-10-28 Thread Pablo Galindo Salgado
lations Mark Shannon! Thanks also to everyone else who provided feedback on this PEP! Regards from rainy London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org htt

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Pablo Galindo Salgado
Regarding having a Solaris buildbot: if someone provides a Solaris buildbot then the deal is that that someone or some other party must look after that buildbot and fix problems that appear in it in a timely manner. Broken buildbots stop releases and I don't want to be in a situation in which I

[Python-Dev] Re: Accepting PEP 626

2020-10-29 Thread Pablo Galindo Salgado
tion :) On Thu, 29 Oct 2020 at 13:29, Mark Shannon wrote: > Hi Pablo, > > On 29/10/2020 11:08 am, Pablo Galindo Salgado wrote: > > > The new semantics may well result in some slowdowns. That's stated in > > the PEP.I don't think I can reliably isolate the effe

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Pablo Galindo Salgado
>Two volunteer core developers and at least one buildbot would help a > lot to ensure that Python is working on Solaris for real, and reduce > the number of open Solaris issues. If it happens, I'm perfectly fine > with keeping Solaris support. > I also hope that more people will contribute to

[Python-Dev] [RELEASE] Python 3.10.0a4 is now available

2021-01-04 Thread Pablo Galindo Salgado
surprisingly, the movement of a test particle in such spacetime is not only a very chaotic system but also has some fractals <https://arxiv.org/abs/gr-qc/9502014> hiding the complexity of its movement. Regards from cold London, Pablo Galindo Salgado ___ Pytho

[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-20 Thread Pablo Galindo Salgado
pros and cons. Regards from cloudy London, Pablo Galindo Salgado On Wed, 20 Jan 2021 at 15:12, Mark Shannon wrote: > Hi Pablo, > > On 19/01/2021 6:46 pm, Pablo Galindo Salgado wrote: > > Hi Mark, > > > > Thanks for gathering this proposal! Looks very interesting.

[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-20 Thread Pablo Galindo Salgado
, Pablo Galindo Salgado On Wed, 20 Jan 2021 at 16:06, Mark Shannon wrote: > Hi Pablo, > > On 20/01/2021 3:18 pm, Pablo Galindo Salgado wrote: > > >It depends on what you mean by "similar tools". > > > > Any 3rd party tool or debugger that is printing merged

[Python-Dev] Re: PyGC and PyObject_Malloc introspection

2021-01-14 Thread Pablo Galindo Salgado
t's underneath (pymalloc or libc malloc or whatever else) may actually allocate even more than what you asked. Regards from cloudy London, Pablo Galindo Salgado On Thu, 14 Jan 2021 at 16:10, Julien Danjou wrote: > Hi there, > > I've spent quite some time on memory profiling for Pyt

[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-19 Thread Pablo Galindo Salgado
object in the stack (like gdb does). Is this change going to affect these tools or they will continue working as before? In case this change will affect this tools, is there any workaround to produce the unified C/Python call stack given the Python stack and the C stack? Kind regards, P

[Python-Dev] Re: PyGC and PyObject_Malloc introspection

2021-01-15 Thread Pablo Galindo Salgado
> Exactly, which is a bit a bummer. Considering Python provides 3 > different memory allocator, it'd be great if there was some ability to > be sure that PyObject_Malloc pointer are actually PyObject, not > Py_GC_HEAD. The allocators are specialized based on the allocation strategy and

[Python-Dev] Re: PyGC and PyObject_Malloc introspection

2021-01-15 Thread Pablo Galindo Salgado
forced PyObject_Init and _PyObject_Init to not be inlined and that made a 7-13% speed impact overall in the performance test suite. If you want to track all objects creation, your best bet IMHO is a debug build and to use sys.getobjects(). Regards from sunny London, Pablo Galindo Salgado On Fri,

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Pablo Galindo Salgado
One worry that I have in general with this move is the usage of _PyType_GetModuleByDef to get the type object from the module definition. This normally involves getting a TLS in every instance creation, which can impact notably performance for some perf-sensitive types or types that are created a

[Python-Dev] [ Release ] Python 3.10a5 and release blockers

2021-02-01 Thread Pablo Galindo Salgado
London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org

[Python-Dev] Python 3.10.0a5 is now available

2021-02-03 Thread Pablo Galindo Salgado
neutron star or black hole. Those with masses up to the limit remain stable as white dwarfs. The currently accepted value of the Chandrasekhar limit is about 1.4 M☉ (2.765×1030 kg). So we can be safe knowing that our sun is not going to become a black hole! Re

[Python-Dev] [RELEASE] Python 3.10.0b3 is available

2021-06-17 Thread Pablo Galindo Salgado
Summer is almost here (at least in half of the planet) and Python 3.10 is finishing baking in the oven. For those of you that want to taste it before is finally ready (and if you are a library developer, you certainly do!) you can have the second-to-last beta now, but be careful as is still very

[Python-Dev] April Steering Council update

2021-06-21 Thread Pablo Galindo Salgado
and could help to avoid misunderstandings. It was decided that some devguide changes could be proposed. Regards from rainy London. Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
in the Python API (to preserve all possible information about the original code), so this complicates quite a lot the API to get this, as `ast.parse` is not enough to get the original tree. On Tue, 18 May 2021 at 08:53, Pablo Galindo Salgado wrote: > Hi Nathaniel, > > Thanks a lot for your s

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
sts to check this). On Tue, 18 May 2021 at 13:25, Pablo Galindo Salgado wrote: > Yet another problem that I found: > > One integer is actually not enough to assign IDs. One unsigned integer can > cover 4,294,967,295 AST nodes, but is technically possible > to have more than that in a

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
get assigned (and the parser doesn't have an easy way to know how many IDs has thrown away). This forces us to either use something bigger than an integer (probably a size_t) or to deal with overflow. On Tue, 18 May 2021 at 10:23, Pablo Galindo Salgado wrote: > Hu, actually another p

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
ore functionality. On Tue, 18 May 2021 at 13:47, Pablo Galindo Salgado wrote: > > One integer is actually not enough to assign IDs. > > Actually, disregard this particular problem. I think that we could > perfectly stop assigning IDs if we reach the overflow limit and call it a >

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
this information is lost, although one could argue that then is not extremely useful... Regards from sunny London, Pablo Galindo Salgado On Tue, 18 May 2021, 01:43 Nathaniel Smith, wrote: > On Mon, May 17, 2021 at 6:18 AM Mark Shannon wrote: > > 2. Repeated binary operations on the

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
ven that the reasons to deactivate this option exist, but I expect them to be very rare, I would prefer to maximize simplicity and maintainability. On Sat, 8 May 2021 at 21:50, Pablo Galindo Salgado wrote: > > I don't think the optional existence of column number information needs > a di

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
piled without the flag. On Sat, 8 May 2021 at 21:13, Gregory P. Smith wrote: > > > On Sat, May 8, 2021 at 11:58 AM Pablo Galindo Salgado > wrote: > >> Hi Brett, >> >> Just to be clear, .pyo files have not existed for a while: >>> https://www.python.or

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
Pablo Galindo Salgado wrote: > >> We can't piggy back on -OO as the only way to disable this, it needs to > >> have an option of its own. -OO is unusable as code that relies on > "doc" > >> strings as application data such as http://www.dabeaz.com/ply/pl

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
: > > > On Sat, May 8, 2021 at 2:09 PM Pablo Galindo Salgado > wrote: > >> > Why not put in it -O instead? Then -O means lose asserts and lose >> fine-grained tracebacks, while -OO continues to also >> strip out doc strings. >> >> What if someone

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
opping this extra data. This is indeed the plan, sorry for the confusion. The opt-out mechanism is using -OO, precisely as we are already dropping other data. Thanks for the clarifications! On Sat, 8 May 2021 at 19:41, Brett Cannon wrote: > > > On Fri, May 7, 2021 at 7:31 PM Pablo Galindo Sal

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
ink is overkill. On Sat, 8 May 2021 at 21:45, Gregory P. Smith wrote: > > On Sat, May 8, 2021 at 1:32 PM Pablo Galindo Salgado > wrote: > >> > We can't piggy back on -OO as the only way to disable this, it needs >> to have an option of its own. -OO is unusable as code

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
one), but that's much much better than something that scales with the number of instructions :) What's your opinion on this? On Sat, 8 May 2021 at 21:45, Gregory P. Smith wrote: > > On Sat, May 8, 2021 at 1:32 PM Pablo Galindo Salgado > wrote: > >> > We can't piggy back on -OO as

[Python-Dev] PEP 657 – Include Fine Grained Error Locations in Tracebacks

2021-05-09 Thread Pablo Galindo Salgado
-in-tracebacks/8629 To avoid splitting the discussion, *please redirect your comments there* instead of replying to this thread. Thanks! Regards from sunny London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-10 Thread Pablo Galindo Salgado
That is going to be very hard to read, unfortunately. Especially when the line is not simple. Highlighting the range is quite a fundamental part of the proposal and is driven by the great welcoming of highlighting ranges for syntax errors, which many users have reached to say that they find it

[Python-Dev] Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
reporting but we understand and respect other points of view. Does anyone see a better way to encode this information **without complicating a lot the implementation**? What are people thoughts on the feature? Thanks in advance, Regards from cloudy London, Pablo Galindo Salgado __

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
Technically the main concern may be the size of the unmarshalled pyc files in memory, more than the storage size of disk. On Fri, 7 May 2021, 23:04 Antoine Pitrou, wrote: > On Fri, 7 May 2021 22:45:38 +0100 > Pablo Galindo Salgado wrote: > > > > The cost of this is havin

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
:35, Gregory P. Smith wrote: > > On Fri, May 7, 2021 at 3:24 PM Pablo Galindo Salgado > wrote: > >> Thanks a lot Gregory for the comments! >> >> An additional cost to this is things that parse text tracebacks not >>> knowing how to handle it and things that log t

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
This is actually a very good point. The only disadvantage is that it complicates the parsing a bit and we loose the possibility of indexing the table by instruction offset. On Fri, 7 May 2021 at 23:01, Larry Hastings wrote: > On 5/7/21 2:45 PM, Pablo Galindo Salgado wrote: > > Given th

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
ier for reporting the exception as the current traceback display removes indentation. On Fri, 7 May 2021 at 23:37, MRAB wrote: > On 2021-05-07 22:45, Pablo Galindo Salgado wrote: > > Hi there, > > > > We are preparing a PEP and we would like to start some early discussion &

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
al So that's an increase of 8.56 % over the original value. This is storing the start offset and end offset with no compression whatsoever. On Fri, 7 May 2021 at 22:45, Pablo Galindo Salgado wrote: > Hi there, > > We are preparing a PEP and we would like to start some early discuss

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
Thanks a lot Gregory for the comments! An additional cost to this is things that parse text tracebacks not knowing > how to handle it and things that log tracebacks generating additional > output. We should provide a way for people to disable the feature on a process as > part of this while they

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
021 at 23:21, Irit Katriel wrote: > > > On Fri, May 7, 2021 at 10:52 PM Pablo Galindo Salgado > wrote: > >> >> The cost of this is having the start column number and end column number >> information for every bytecode instruction >> > > > Is it rea

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
One last note for clarity: that's the increase of size in the stdlib, the increase of size for pyc files goes from 28.471296MB to 34.750464MB, which is an increase of 22%. On Sat, 8 May 2021 at 01:43, Pablo Galindo Salgado wrote: > Some update on the numbers. We have made some dr

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
02:41, MRAB wrote: > On 2021-05-08 01:43, Pablo Galindo Salgado wrote: > > Some update on the numbers. We have made some draft implementation to > > corroborate the > > numbers with some more realistic tests and seems that our original > > calculations were wrong. &

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Pablo Galindo Salgado
as offsets would be quite random (although predominantly in the range 10 - 120). On Sat, 8 May 2021 at 01:56, Pablo Galindo Salgado wrote: > One last note for clarity: that's the increase of size in the stdlib, the > increase of size > for pyc files goes from 28.471296MB to 34.750464MB, w

  1   2   3   >