Em sex., 22 de abr. de 2022 às 09:02, Petr Viktorin <encu...@gmail.com> escreveu:
> Hello Fabio, > Let's talk a bit about which API should, exactly, be guaranteed to not > change across minor releases. > So far it looks like: > - PyEval_RequestCodeExtraIndex > - PyCode_GetExtra > - PyCode_SetExtra > - PyFrameEvalFunction > - PyInterpreterState_GetEvalFrameFunc > - PyInterpreterState_SetEvalFrameFunc > > Do any more come to mind? > > The issue with this set is that in 3.11, _PyFrameEvalFunction changes > its signature to take _PyInterpreterFrame rather than PyFrameObject. > Exposing _PyInterpreterFrame would be quite problematic. For example, > since it's not a PyObject, it has its own lifetime management that's > controlled by the interpreter itself,. And it includes several > pointers whose lifetime and semantics also isn't guaranteed (they > might be borrowed, cached or filled on demand). I don't think we can > make any guarantees on these, so the info needs to be accessed using > getter functions. > > There is the function _PyFrame_GetFrameObject, which returns a > PyFrameObject. > I think it would be best to only expose _PyInterpreterFrame as an > opaque structure, and expose PyFrame_GetFrameObject so debuggers can > get a PyFrameObject from it. > Does that sound reasonable? > Humm, now I'm a bit worried... the approach the debugger is using gets the PyFrameObject that's about to be executed and changes the PyFrameObject.f_code just before the execution so that the new code is executed instead. >From what you're saying the PyFrameObject isn't really used anymore (apparently it's substituted by a _PyInterpreterFrame?)... in this case, will this approach still let the debugger patch the code object in the frame before it's actually executed? -- i.e.: the debugger changes the state.interp.eval_frame to its own custom evaluation function, but _PyEval_EvalFrameDefault is still what ends up being called afterwards (it works more as a hook to change the PyFrameObject.f_code prior to execution than as an alternate interpreter). On Thu, Mar 24, 2022 at 8:13 PM Fabio Zadrozny <fabi...@gmail.com> wrote: > > > > > > Em qui., 24 de mar. de 2022 às 15:39, Fabio Zadrozny <fabi...@gmail.com> > escreveu: > >>> > >>> PEP 523 API added more private functions for code objects: > >>> > >>> * _PyEval_RequestCodeExtraIndex() > >>> * _PyCode_GetExtra() > >>> * _PyCode_SetExtra() > >>> > >>> The _PyEval_RequestCodeExtraIndex() function seems to be used by the > >>> pydevd debugger. The two others seem to be unused in the wild. I'm not > >>> sure if these ones should be moved to the internal C API. They can be > >>> left unchanged, since they don't use a type only defined by the > >>> internal C API. > >> > >> Just to note, the pydevd/debugpy debuggers actually uses all of those > APIs. > >> > >> i.e.: > >> > >> > https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L187 > >> > https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L232 > >> > https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L311 > >> > >> The debugger already has workarounds because of changes to evaluation > api over time (see: > https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L491) > and I know 3.11 won't be different. > >> > >> I'm ok with changes as I understand that this is a special API -- as > long as there's still a way to use it and get the information needed (the > debugger already goes through many hops because it needs to use many > internals of CPython -- in every new release it's a **really** big task to > update to the latest version as almost everything that the debugger relies > to make debugging fast changes across versions and I never really know if > it'll be possible to support it until I really try to do the port -- I > appreciate having less things in a public API so it's easier to have > extensions work in other interpreters/not recompiling on newer versions, > but please keep it possible to use private APIs which provides the same > access that CPython has to access things internally for special cases such > as the debugger). > >> > >> Maybe later on that PEP from mark which allows a better debugger API > could alleviate that (but until then, if possible I appreciate it if > there's some effort not to break things unless really needed -- ideally > with instructions on how to port). > >> > >> Anyways, to wrap up, the debugger already needs to be built with > `Py_BUILD_CORE_MODULE=1` anyways, so, I guess having it in a private API > (as long as it's still accessible in that case) is probably not a big issue > for the debugger and having setters/getters to set it instead of relying on > `state.interp.eval_frame` seems good to me. > >> > >> Cheers, > >> > >> Fabio > >> > > > > > > I think the main issue here is the compatibility across the same version > though... is it possible to have some kind of guarantee on private APIs > that something won't change across micro-releases? > > > > I.e.: having the frame evaluation function change across major releases > and having them be reworked seems reasonable, but then having the frame > evaluation be changed across micro-releases wouldn't be. > > > > So, I'm ok in pushing things to the internal API, but then I still would > like guarantees about the compatibility of that API in the same major > release (otherwise those setters/getters/frame evaluation should probably > remain on the public API if the related structure was moved to the internal > API). > > > > Cheers, > > > > Fabio > > _______________________________________________ > > Python-Dev mailing list -- python-dev@python.org > > To unsubscribe send an email to python-dev-le...@python.org > > https://mail.python.org/mailman3/lists/python-dev.python.org/ > > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/DHKE7LVN4R7NQFTBJJHGXI3AJOK6OYIV/ > > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HZIPIKCOEF6UVBD7HM67ARQJUVFVTBEY/ Code of Conduct: http://python.org/psf/codeofconduct/
