On Sat, May 8, 2021 at 2:59 PM Gregory P. Smith <g...@krypto.org> wrote:
> > > On Sat, May 8, 2021 at 2:09 PM Pablo Galindo Salgado <pablog...@gmail.com> > wrote: > >> > Why not put in it -O instead? Then -O means lose asserts and lose >> fine-grained tracebacks, while -OO continues to also >> strip out doc strings. >> >> What if someone wants to keep asserts but do not want the extra data? >> > > exactly my theme. our existing -O and -OO already don't serve all user > needs. (I've witnessed people who need asserts but don't want docstrings > wasting ram jump through hacky hoops to do that). Complicating these > options more by combining additional actions on them them doesn't help. > > The reason we have -O and -OO generate their own special opt-1 and opt-2 > pyc files is because both of those change the generated bytecode and > overall flow of the program by omitting instructions and data. code using > those will run slightly faster as there are fewer instructions. > > The change we're talking about here doesn't do that. It just adds > additional metadata to whatever instructions are generated. So it doesn't > feel -O related. > While I'm the opposite. 😄 Metadata that is not necessary for CPython to function and whose primary driver is better exception tracebacks totally falls into the same camp as "I don't need docstrings" to me. > > While some people aren't going to like the overhead, I'm happy not > offering the choice. > > > Greg, what do you think if instead of not writing it to the pyc file > with -OO or adding a header entry to decide to read/write, we place None in > the field? That way we can > > leverage the option that we intend to add to deactivate displaying the > traceback new information to reduce the data in the pyc files. The only > problem > > is that there will be still a tiny bit of overhead: an extra object per > code object (None), but that's much much better than something that scales > with the > > number of instructions :) > > > > What's your opinion on this? > > I don't understand the pyc structure enough to comment on how that works, > Code to read a .pyc file and use it: https://github.com/python/cpython/blob/a0bd9e9c11f5f52c7ddd19144c8230da016b53c6/Lib/importlib/_bootstrap_external.py#L951-L1015 (I'd explain more but it is the weekend and I technically shouldn't be reading this thread 😉). -Brett > but that sounds fine from a way to store less data if these are stored as > a side table rather than intermingled with each instruction itself. *If > anyone even cares about storing less data.* I would not activate > generation of that in py_compile and compileall based on the -X flag to > disable display of tracebacks though. A flag changing a setting of the > current runtime regarding traceback printing detail level should not change > the metadata in pyc files it emits. I realize -O and -OO behave this way, > but I don't view those as a great example. We're not writing new uniquely > named pyc files, I suggest making this an explicit option for py_compile > and compileall if we're going to support generation of pyc files without > column data at all. > > I'm unclear on what the specific goals are with all of these option > possibilities. > > Who non-hypothetically cares about a 22% pyc file size increase? I don't > think we should be concerned. I'm in favor of always writing them and the > 20% size increase that results in. If pyc size is an issue that should be > its own separate enhancement PEP. When it comes to pyc files there is more > data we may want to store in the future for performance reasons - I don't > see them shrinking without an independent effort. > > Caring about additional data retained in memory at runtime makes more > sense to me as ram cost is much greater than storage cost and is paid > repeatedly per process. Storing an additional reference to None on code > objects where a column information table is perfectly fine. That can be a > -X style interpreter startup option. It isn't something that needs to > impacted by the pyc files. Pass that option to the interpreter, and it > just discards column info tables on code objects after loading them or > compiling them. If people want to optimize for a shared pyc situation with > memory mapping techniques, that is also something that should be a separate > enhancement PEP and not involved here. People writing code to use the > column information should always check it for None first, that'd be > something we document with the new feature. > > -gps > > >> >> On Sat, 8 May 2021 at 22:05, Ethan Furman <et...@stoneleaf.us> wrote: >> >>> On 5/8/21 1:31 PM, Pablo Galindo Salgado wrote: >>> >> We can't piggy back on -OO as the only way to disable this, it needs >>> to >>> >> have an option of its own. -OO is unusable as code that relies on >>> "doc" >>> >> strings as application data such as >>> http://www.dabeaz.com/ply/ply.html >>> >> exists. >>> > >>> > -OO is the only sensible way to disable the data. There are two >>> things to disable: >>> > >>> > * The data in pyc files >>> > * Printing the exception highlighting >>> >>> Why not put in it -O instead? Then -O means lose asserts and lose >>> fine-grained tracebacks, while -OO continues to also >>> strip out doc strings. >>> >>> -- >>> ~Ethan~ >>> _______________________________________________ >>> Python-Dev mailing list -- python-dev@python.org >>> To unsubscribe send an email to python-dev-le...@python.org >>> https://mail.python.org/mailman3/lists/python-dev.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-dev@python.org/message/BEE4BGUZHXBTVDPOW5R4DC3S463XC3EJ/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> _______________________________________________ >> Python-Dev mailing list -- python-dev@python.org >> To unsubscribe send an email to python-dev-le...@python.org >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/WK4KXZPOSWYMI3C5AILQCEYVZRCDFL7N/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/Q5RCT3VMQFPZTJJKB666PML4K2KZ2HJW/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3SIFWW2H4GXYDYQ7AYCZWHJZKTP7YJJT/ Code of Conduct: http://python.org/psf/codeofconduct/
