Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-spyder-kernels for openSUSE:Factory checked in at 2021-11-29 17:28:28 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-spyder-kernels (Old) and /work/SRC/openSUSE:Factory/.python-spyder-kernels.new.31177 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-spyder-kernels" Mon Nov 29 17:28:28 2021 rev:30 rq:934156 version:2.2.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-spyder-kernels/python-spyder-kernels.changes 2021-10-27 22:21:42.583214161 +0200 +++ /work/SRC/openSUSE:Factory/.python-spyder-kernels.new.31177/python-spyder-kernels.changes 2021-12-02 02:24:49.376697893 +0100 @@ -1,0 +2,9 @@ +Fri Nov 26 17:58:11 UTC 2021 - Ben Greiner <c...@bnavigator.de> + +- Update to version 2.2.0 + * Add the ability to capture segfaults and flush standard streams + from Spyder. + * Add support for jupyter_client 7. +- Drop upstreamed spyder-kernels-pr328-unpin_jupyter-client.patch + +------------------------------------------------------------------- Old: ---- python-spyder-kernels-2.1.3.tar.gz spyder-kernels-pr328-unpin_jupyter-client.patch New: ---- python-spyder-kernels-2.2.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-spyder-kernels.spec ++++++ --- /var/tmp/diff_new_pack.sN38BV/_old 2021-12-02 02:24:49.804696400 +0100 +++ /var/tmp/diff_new_pack.sN38BV/_new 2021-12-02 02:24:49.808696386 +0100 @@ -20,7 +20,7 @@ %define skip_python2 1 %define skip_python36 1 Name: python-spyder-kernels -Version: 2.1.3 +Version: 2.2.0 Release: 0 Summary: Jupyter kernels for Spyder's console License: MIT @@ -28,8 +28,6 @@ URL: https://github.com/spyder-ide/spyder-kernels # PyPI tarballs do not include the tests: https://github.com/spyder-ide/spyder-kernels/issues/66 Source: %{url}/archive/v%{version}.tar.gz#/%{name}-%{version}.tar.gz -# PATCH-FIX-UPSTREAM spyder-kernels-pr328-unpin_jupyter-client.patch -- gh#spyder-ide/spyder-kernels#328 -Patch0: https://github.com/spyder-ide/spyder-kernels/pull/328.patch#/spyder-kernels-pr328-unpin_jupyter-client.patch BuildRequires: %{python_module setuptools} BuildRequires: fdupes BuildRequires: python-rpm-macros ++++++ python-spyder-kernels-2.1.3.tar.gz -> python-spyder-kernels-2.2.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/CHANGELOG.md new/spyder-kernels-2.2.0/CHANGELOG.md --- old/spyder-kernels-2.1.3/CHANGELOG.md 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/CHANGELOG.md 2021-11-23 00:19:57.000000000 +0100 @@ -1,5 +1,40 @@ # History of changes +### New features + +* Add the ability to capture segfaults and flush standard streams from + Spyder. +* Add support for jupyter_client 7. + +## Version 2.2.0 (2021-11-22) + +### Issues Closed + +* [Issue 326](https://github.com/spyder-ide/spyder-kernels/issues/326) - Setting variables in recursive debugging namespaces is broken ([PR 327](https://github.com/spyder-ide/spyder-kernels/pull/327) by [@impact27](https://github.com/impact27)) +* [Issue 325](https://github.com/spyder-ide/spyder-kernels/issues/325) - %timeit has issues with the namespace ([PR 327](https://github.com/spyder-ide/spyder-kernels/pull/327) by [@impact27](https://github.com/impact27)) +* [Issue 316](https://github.com/spyder-ide/spyder-kernels/issues/316) - Disable capture_fd_output in IPykernel 6+ in favor of Wurlitzer ([PR 337](https://github.com/spyder-ide/spyder-kernels/pull/337) by [@ccordoba12](https://github.com/ccordoba12)) + +In this release 3 issues were closed. + +### Pull Requests Merged + +* [PR 337](https://github.com/spyder-ide/spyder-kernels/pull/337) - PR: Disable the new mechanism to forward low-level output in IPykernel 6, by [@ccordoba12](https://github.com/ccordoba12) ([316](https://github.com/spyder-ide/spyder-kernels/issues/316)) +* [PR 336](https://github.com/spyder-ide/spyder-kernels/pull/336) - PR: Don't show traceback when exiting debugger after breakpoint call, by [@ccordoba12](https://github.com/ccordoba12) +* [PR 335](https://github.com/spyder-ide/spyder-kernels/pull/335) - PR: Remove locals from exec in comprehensions (debugger), by [@impact27](https://github.com/impact27) +* [PR 334](https://github.com/spyder-ide/spyder-kernels/pull/334) - PR: Fix missing stderr when the kernel crashes, by [@impact27](https://github.com/impact27) +* [PR 333](https://github.com/spyder-ide/spyder-kernels/pull/333) - PR: Add cast for shape tuple subclass instances when getting values size, by [@dalthviz](https://github.com/dalthviz) +* [PR 332](https://github.com/spyder-ide/spyder-kernels/pull/332) - PR: Remove usage of f_locals.get, by [@impact27](https://github.com/impact27) +* [PR 328](https://github.com/spyder-ide/spyder-kernels/pull/328) - PR: Remove upper constraint on jupyter_client, by [@bnavigator](https://github.com/bnavigator) +* [PR 327](https://github.com/spyder-ide/spyder-kernels/pull/327) - PR: Fix timeit in Pdb and other namespace issues, by [@impact27](https://github.com/impact27) ([326](https://github.com/spyder-ide/spyder-kernels/issues/326), [325](https://github.com/spyder-ide/spyder-kernels/issues/325)) +* [PR 317](https://github.com/spyder-ide/spyder-kernels/pull/317) - PR: Run asyncio and normal handlers, by [@impact27](https://github.com/impact27) ([16183](https://github.com/spyder-ide/spyder/issues/16183)) +* [PR 254](https://github.com/spyder-ide/spyder-kernels/pull/254) - PR: Enable faulthandler from the frontend, by [@impact27](https://github.com/impact27) + +In this release 10 pull requests were closed. + + +---- + + ## Version 2.1.3 (2021-10-02) ### Pull Requests Merged diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/setup.py new/spyder-kernels-2.2.0/setup.py --- old/spyder-kernels-2.1.3/setup.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/setup.py 2021-11-23 00:19:57.000000000 +0100 @@ -36,13 +36,14 @@ REQUIREMENTS = [ + 'decorator<5; python_version<"3"', 'backports.functools-lru-cache; python_version<"3"', 'cloudpickle', 'ipykernel<5; python_version<"3"', 'ipykernel>=5.3.0; python_version>="3"', 'ipython<6; python_version<"3"', 'ipython>=7.6.0; python_version>="3"', - 'jupyter-client>=5.3.4,<7', + 'jupyter-client>=5.3.4', 'pyzmq>=17', 'wurlitzer>=1.0.3;platform_system!="Windows"', ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/_version.py new/spyder-kernels-2.2.0/spyder_kernels/_version.py --- old/spyder-kernels-2.1.3/spyder_kernels/_version.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/_version.py 2021-11-23 00:19:57.000000000 +0100 @@ -8,5 +8,5 @@ """Version File.""" -VERSION_INFO = (2, 1, 3) +VERSION_INFO = (2, 2, 0) __version__ = '.'.join(map(str, VERSION_INFO)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/comms/frontendcomm.py new/spyder-kernels-2.2.0/spyder_kernels/comms/frontendcomm.py --- old/spyder-kernels-2.1.3/spyder_kernels/comms/frontendcomm.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/comms/frontendcomm.py 2021-11-23 00:19:57.000000000 +0100 @@ -15,7 +15,6 @@ import threading import time -import ipykernel from IPython.core.getipython import get_ipython from jupyter_client.localinterfaces import localhost from tornado import ioloop @@ -146,38 +145,35 @@ return handler = self.kernel.shell_handlers.get(msg_type, None) - if handler is None: - self.kernel.log.warning("Unknown message type: %r", msg_type) - else: - try: - if not PY2: - import asyncio - if (not getattr(asyncio, 'run', False) or - ipykernel.__version__[0] < '6'): - # This is required for Python 3.6, which doesn't have - # asyncio.run or ipykernel versions less than 6. The - # nice thing is that ipykernel 6, which requires - # asyncio, doesn't support Python 3.6. - handler(out_stream, ident, msg) - else: - # This is needed for ipykernel 6+ - asyncio.run(handler(out_stream, ident, msg)) - else: - handler(out_stream, ident, msg) - except ValueError as e: - # This avoids showing an unnecessary message about expected - # coroutines. + try: + if handler is None: + self.kernel.log.warning("Unknown message type: %r", msg_type) return - except Exception: - self.kernel.log.error("Exception in message handler:", - exc_info=True) + if PY2: + handler(out_stream, ident, msg) return - sys.stdout.flush() - sys.stderr.flush() - # Flush to ensure reply is sent - if out_stream: - out_stream.flush(zmq.POLLOUT) + import asyncio + + if (getattr(asyncio, 'run', False) and + asyncio.iscoroutinefunction(handler)): + # This is needed for ipykernel 6+ + asyncio.run(handler(out_stream, ident, msg)) + else: + # This is required for Python 3.6, which doesn't have + # asyncio.run or ipykernel versions less than 6. The + # nice thing is that ipykernel 6, which requires + # asyncio, doesn't support Python 3.6. + handler(out_stream, ident, msg) + except Exception: + self.kernel.log.error( + "Exception in message handler:", exc_info=True) + finally: + sys.stdout.flush() + sys.stderr.flush() + # Flush to ensure reply is sent + if out_stream: + out_stream.flush(zmq.POLLOUT) def remote_call(self, comm_id=None, blocking=False, callback=None, timeout=None): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/console/__main__.py new/spyder-kernels-2.2.0/spyder_kernels/console/__main__.py --- old/spyder-kernels-2.1.3/spyder_kernels/console/__main__.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/console/__main__.py 2021-11-23 00:19:57.000000000 +0100 @@ -20,4 +20,12 @@ sys.path.remove(cwd) from spyder_kernels.console import start - start.main() + try: + start.main() + except Exception: + # We have to explicitely write to __stderr__ as stderr might already + # have been replaced. + import traceback + traceback.print_exc(file=sys.__stderr__) + sys.__stderr__.flush() + raise diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/console/kernel.py new/spyder-kernels-2.2.0/spyder_kernels/console/kernel.py --- old/spyder-kernels-2.1.3/spyder_kernels/console/kernel.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/console/kernel.py 2021-11-23 00:19:57.000000000 +0100 @@ -19,7 +19,6 @@ # Third-party imports import ipykernel from ipykernel.ipkernel import IPythonKernel -from ipykernel.zmqshell import ZMQInteractiveShell from traitlets.config.loader import LazyConfigValue # Local imports @@ -30,34 +29,16 @@ from spyder_kernels.utils.mpl import ( MPL_BACKENDS_FROM_SPYDER, MPL_BACKENDS_TO_SPYDER, INLINE_FIGURE_FORMATS) from spyder_kernels.utils.nsview import get_remote_data, make_remote_view +from spyder_kernels.console.shell import SpyderShell +if PY3: + import faulthandler # Excluded variables from the Variable Explorer (i.e. they are not # shown at all there) EXCLUDED_NAMES = ['In', 'Out', 'exit', 'get_ipython', 'quit'] -class SpyderShell(ZMQInteractiveShell): - """Spyder shell.""" - - def ask_exit(self): - """Engage the exit actions.""" - self.kernel.frontend_comm.close_thread() - return super(SpyderShell, self).ask_exit() - - def get_local_scope(self, stack_depth): - """Get local scope at given frame depth.""" - frame = sys._getframe(stack_depth + 1) - if self.kernel._pdb_frame is frame: - # we also give the globals because they might not be in - # self.user_ns - namespace = frame.f_globals.copy() - namespace.update(self.kernel._pdb_locals) - return namespace - else: - return frame.f_locals - - class SpyderKernel(IPythonKernel): """Spyder kernel for Jupyter.""" @@ -99,19 +80,26 @@ 'get_matplotlib_backend': self.get_matplotlib_backend, 'pdb_input_reply': self.pdb_input_reply, '_interrupt_eventloop': self._interrupt_eventloop, + 'enable_faulthandler': self.enable_faulthandler, + "flush_std": self.flush_std, } for call_id in handlers: self.frontend_comm.register_call_handler( call_id, handlers[call_id]) self.namespace_view_settings = {} - self._pdb_obj = None self._pdb_step = None self._mpl_backend_error = None self._running_namespace = None self._pdb_input_line = None + self.faulthandler_handle = None # -- Public API ----------------------------------------------------------- + def do_shutdown(self, restart): + """Disable faulthandler if enabled before proceeding.""" + self.disable_faulthandler() + super(SpyderKernel, self).do_shutdown(restart) + def frontend_call(self, blocking=False, broadcast=True, timeout=None, callback=None): """Call the frontend.""" @@ -127,6 +115,42 @@ callback=callback, timeout=timeout) + def flush_std(self): + """Flush C standard streams.""" + sys.__stderr__.flush() + sys.__stdout__.flush() + + def enable_faulthandler(self, fn): + """ + Open a file to save the faulthandling and identifiers for + internal threads. + """ + if not PY3: + # Not implemented + return + self.disable_faulthandler() + f = open(fn, 'w') + self.faulthandler_handle = f + f.write("Main thread id:\n") + f.write(hex(threading.main_thread().ident)) + f.write('\nSystem threads ids:\n') + f.write(" ".join([hex(thread.ident) for thread in threading.enumerate() + if thread is not threading.main_thread()])) + f.write('\n') + faulthandler.enable(f) + + def disable_faulthandler(self): + """ + Cancel the faulthandling, close the file handle and remove the file. + """ + if not PY3: + # Not implemented + return + if self.faulthandler_handle: + faulthandler.disable() + self.faulthandler_handle.close() + self.faulthandler_handle = None + # --- For the Variable Explorer def set_namespace_view_settings(self, settings): """Set namespace_view_settings.""" @@ -233,7 +257,7 @@ """ from spyder_kernels.utils.misc import fix_reference_name - glbs = self._mglobals() + glbs = self.shell.user_ns load_func = iofunctions.load_funcs[ext] data, error_message = load_func(filename) @@ -263,12 +287,6 @@ return iofunctions.save(data, filename) # --- For Pdb - def is_debugging(self): - """ - Check if we are currently debugging. - """ - return bool(self._pdb_frame) - def _do_complete(self, code, cursor_pos): """Call parent class do_complete""" return super(SpyderKernel, self).do_complete(code, cursor_pos) @@ -279,13 +297,13 @@ Public method of ipykernel overwritten for debugging. """ - if self.is_debugging(): - return self._pdb_obj.do_complete(code, cursor_pos) + if self.shell.is_debugging(): + return self.shell.pdb_session.do_complete(code, cursor_pos) return self._do_complete(code, cursor_pos) def publish_pdb_state(self): """Publish Pdb state.""" - if self._pdb_obj: + if self.shell.pdb_session: state = dict(namespace_view = self.get_namespace_view(), var_properties = self.get_var_properties(), step = self._pdb_step) @@ -295,35 +313,36 @@ """ Handle a message from the frontend """ - if self._pdb_obj: - self._pdb_obj.set_spyder_breakpoints(breakpoints) + if self.shell.pdb_session: + self.shell.pdb_session.set_spyder_breakpoints(breakpoints) def set_pdb_ignore_lib(self, state): """ Change the "Ignore libraries while stepping" debugger setting. """ - if self._pdb_obj: - self._pdb_obj.pdb_ignore_lib = state + if self.shell.pdb_session: + self.shell.pdb_session.pdb_ignore_lib = state def set_pdb_execute_events(self, state): """ Handle a message from the frontend """ - if self._pdb_obj: - self._pdb_obj.pdb_execute_events = state + if self.shell.pdb_session: + self.shell.pdb_session.pdb_execute_events = state def set_pdb_use_exclamation_mark(self, state): """ Set an option on the current debugging session to decide wether the Pdb commands needs to be prefixed by '!' """ - if self._pdb_obj: - self._pdb_obj.pdb_use_exclamation_mark = state + if self.shell.pdb_session: + self.shell.pdb_session.pdb_use_exclamation_mark = state def pdb_input_reply(self, line, echo_stack_entry=True): """Get a pdb command from the frontend.""" - if self._pdb_obj: - self._pdb_obj._disable_next_stack_entry = not echo_stack_entry + if self.shell.pdb_session: + self.shell.pdb_session._disable_next_stack_entry = ( + not echo_stack_entry) self._pdb_input_line = line if self.eventloop: # Interrupting the eventloop is only implemented when a message is @@ -340,7 +359,7 @@ Runs the eventloop while debugging. """ # Only works if the comm is open and this is a pdb prompt. - if not self.frontend_comm.is_open() or not self._pdb_frame: + if not self.frontend_comm.is_open() or not self.shell.is_debugging(): return input(prompt) # Flush output before making the request. @@ -566,16 +585,16 @@ """ ns = {} if self._running_namespace is None: - ns.update(self._mglobals()) + ns.update(self.shell.user_ns) else: + # This is true when a file is executing. running_globals, running_locals = self._running_namespace ns.update(running_globals) if running_locals is not None: ns.update(running_locals) - if self._pdb_frame is not None: - ns.update(self._pdb_locals) - + # Add debugging locals + ns.update(self.shell._pdb_locals) # Add magics to ns so we can show help about them on the Help # plugin if with_magics: @@ -583,7 +602,6 @@ cell_magics = self.shell.magics_manager.magics['cell'] ns.update(line_magics) ns.update(cell_magics) - return ns def _get_reference_namespace(self, name): @@ -592,22 +610,10 @@ It returns the globals() if reference has not yet been defined """ - glbs = self._mglobals() - if self._pdb_frame is None: - return glbs - else: - lcls = self._pdb_locals - if name in lcls: - return lcls - else: - return glbs - - def _mglobals(self): - """Return current globals -- handles Pdb frames""" - if self._pdb_frame is not None: - return self._pdb_frame.f_globals - else: - return self.shell.user_ns + lcls = self.shell._pdb_locals + if name in lcls: + return lcls + return self.shell.user_ns def _get_len(self, var): """Return sequence length""" @@ -668,28 +674,6 @@ except: return None - # --- For Pdb - def _register_pdb_session(self, pdb_obj): - """Register Pdb session to use it later""" - self._pdb_obj = pdb_obj - - @property - def _pdb_frame(self): - """Return current Pdb frame if there is any""" - if self._pdb_obj is not None and self._pdb_obj.curframe is not None: - return self._pdb_obj.curframe - - @property - def _pdb_locals(self): - """ - Return current Pdb frame locals if available. Otherwise - return an empty dictionary - """ - if self._pdb_frame: - return self._pdb_obj.curframe_locals - else: - return {} - # --- For the Help plugin def _eval(self, text): """ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/console/shell.py new/spyder-kernels-2.2.0/spyder_kernels/console/shell.py --- old/spyder-kernels-2.1.3/spyder_kernels/console/shell.py 1970-01-01 01:00:00.000000000 +0100 +++ new/spyder-kernels-2.2.0/spyder_kernels/console/shell.py 2021-11-23 00:19:57.000000000 +0100 @@ -0,0 +1,106 @@ +# -*- coding: utf-8 -*- +# ----------------------------------------------------------------------------- +# Copyright (c) 2009- Spyder Kernels Contributors +# +# Licensed under the terms of the MIT License +# (see spyder_kernels/__init__.py for details) +# ----------------------------------------------------------------------------- + +""" +Spyder shell for Jupyter kernels. +""" + +# Standard library imports +import bdb +import sys + +# Third-party imports +from ipykernel.zmqshell import ZMQInteractiveShell + + +class SpyderShell(ZMQInteractiveShell): + """Spyder shell.""" + + def __init__(self, *args, **kwargs): + # Create _pdb_obj before __init__ + self._pdb_obj = None + super(SpyderShell, self).__init__(*args, **kwargs) + + # ---- Methods overriden by us. + def ask_exit(self): + """Engage the exit actions.""" + self.kernel.frontend_comm.close_thread() + return super(SpyderShell, self).ask_exit() + + def _showtraceback(self, etype, evalue, stb): + """ + Don't show a traceback when exiting our debugger after entering + it through a `breakpoint()` call. + + This is because calling `!exit` after `breakpoint()` raises + BdbQuit, which throws a long and useless traceback. + """ + if etype is bdb.BdbQuit: + stb = [''] + super(SpyderShell, self)._showtraceback(etype, evalue, stb) + + # ---- For Pdb namespace integration + def get_local_scope(self, stack_depth): + """Get local scope at given frame depth.""" + frame = sys._getframe(stack_depth + 1) + if self._pdb_frame is frame: + # Avoid calling f_locals on _pdb_frame + return self._pdb_obj.curframe_locals + else: + return frame.f_locals + + def get_global_scope(self, stack_depth): + """Get global scope at given frame depth.""" + frame = sys._getframe(stack_depth + 1) + return frame.f_globals + + def is_debugging(self): + """ + Check if we are currently debugging. + """ + return bool(self._pdb_frame) + + @property + def pdb_session(self): + """Get current pdb session.""" + return self._pdb_obj + + @pdb_session.setter + def pdb_session(self, pdb_obj): + """Register Pdb session to use it later""" + self._pdb_obj = pdb_obj + + @property + def _pdb_frame(self): + """Return current Pdb frame if there is any""" + if self.pdb_session is not None: + return self.pdb_session.curframe + + @property + def _pdb_locals(self): + """ + Return current Pdb frame locals if available. Otherwise + return an empty dictionary + """ + if self._pdb_frame is not None: + return self._pdb_obj.curframe_locals + else: + return {} + + @property + def user_ns(self): + """Get the current namespace.""" + if self._pdb_frame is not None: + return self._pdb_frame.f_globals + else: + return self.__user_ns + + @user_ns.setter + def user_ns(self, namespace): + """Set user_ns.""" + self.__user_ns = namespace diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/console/start.py new/spyder-kernels-2.2.0/spyder_kernels/console/start.py --- old/spyder-kernels-2.1.3/spyder_kernels/console/start.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/console/start.py 2021-11-23 00:19:57.000000000 +0100 @@ -30,7 +30,6 @@ IPYKERNEL_6 = ipykernel.__version__[0] >= '6' - def import_spydercustomize(): """Import our customizations into the kernel.""" here = osp.dirname(__file__) @@ -237,6 +236,11 @@ lines = sympy_config(mpl_backend) spy_cfg.IPKernelApp.exec_lines.append(lines) + # Disable the new mechanism to capture and forward low-level output + # in IPykernel 6. For that we have Wurlitzer. + if LooseVersion(ipykernel.__version__) >= LooseVersion('6.3.0'): + spy_cfg.IPKernelApp.capture_fd_output = False + # Merge IPython and Spyder configs. Spyder prefs will have prevalence # over IPython ones cfg._merge(spy_cfg) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/console/tests/test_console_kernel.py new/spyder-kernels-2.2.0/spyder_kernels/console/tests/test_console_kernel.py --- old/spyder-kernels-2.1.3/spyder_kernels/console/tests/test_console_kernel.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/console/tests/test_console_kernel.py 2021-11-23 00:19:57.000000000 +0100 @@ -30,7 +30,7 @@ import numpy as np # Local imports -from spyder_kernels.py3compat import PY3, to_text_string +from spyder_kernels.py3compat import PY2, PY3, to_text_string from spyder_kernels.utils.iofuncs import iofunctions from spyder_kernels.utils.test_utils import get_kernel, get_log_text from spyder_kernels.customize.spyderpdb import SpyderPdb @@ -57,9 +57,9 @@ This function was taken from the ipykernel project. We plan to remove it when dropping support for python 2. - Returns + Yields ------- - kernel_manager: connected KernelManager instance + client: jupyter_client.BlockingKernelClient connected to the kernel """ kernel = Popen([sys.executable, '-c', cmd], stdout=PIPE, stderr=PIPE) try: @@ -423,7 +423,7 @@ with setup_kernel(cmd) as client: msg_id = client.execute("import sys; sys_path = sys.path", user_expressions={'output':'sys_path'}) - reply = client.get_shell_msg(block=True, timeout=TIMEOUT) + reply = client.get_shell_msg(timeout=TIMEOUT) # Transform value obtained through user_expressions user_expressions = reply['content']['user_expressions'] @@ -447,7 +447,7 @@ with setup_kernel(cmd) as client: # Remove all variables client.execute("%reset -f") - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Write multiprocessing code to a file code = """ @@ -465,11 +465,11 @@ # Run code client.execute("runfile(r'{}')".format(to_text_string(p))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that the `result` variable is defined client.inspect('result') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] @@ -487,7 +487,7 @@ with setup_kernel(cmd) as client: # Remove all variables client.execute("%reset -f") - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Write multiprocessing code to a file # Runs two times to verify that in the second case it doesn't break @@ -504,14 +504,14 @@ # Run code two times client.execute("runfile(r'{}')".format(to_text_string(p))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) client.execute("runfile(r'{}')".format(to_text_string(p))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that the `x` variable is defined client.inspect('x') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] @@ -527,7 +527,7 @@ with setup_kernel(cmd) as client: # Remove all variables client.execute("%reset -f") - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Write defined variable code to a file code = u"result = 'hello world'; error # make an error" @@ -547,40 +547,40 @@ # Run code file `d` to define `result` even after an error client.execute("runfile(r'{}', current_namespace=False)" .format(to_text_string(d))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that `result` is defined in the current namespace client.inspect('result') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] # Run code file `u` without current namespace client.execute("runfile(r'{}', current_namespace=False)" .format(to_text_string(u))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that the variable `result2` is defined client.inspect('result2') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] # Run code file `u` with current namespace client.execute("runfile(r'{}', current_namespace=True)" .format(to_text_string(u))) - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] # Verify that the variable `result3` is defined client.inspect('result3') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] # Verify that the variable `__file__` is undefined client.inspect('__file__') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert not content['found'] @@ -601,14 +601,14 @@ suppress=True, formatter={'float_kind':'{:0.2f}'.format}) """) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Create a big Numpy array and an array to check decimal format client.execute(""" x = np.random.rand(75000,5); a = np.array([123412341234.123412341234]) """) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Assert that NumPy threshold, suppress and formatter # are the same as the ones set by the user @@ -617,29 +617,29 @@ s = np.get_printoptions()['suppress']; f = np.get_printoptions()['formatter'] """) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Check correct decimal format client.inspect('a') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content']['data']['text/plain'] assert "123412341234.12" in content # Check threshold value client.inspect('t') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content']['data']['text/plain'] assert "inf" in content # Check suppress value client.inspect('s') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content']['data']['text/plain'] assert "True" in content # Check formatter client.inspect('f') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content']['data']['text/plain'] assert "{'float_kind': <built-in method format of str object" in content @@ -667,7 +667,7 @@ with setup_kernel(cmd) as client: # Remove all variables client.execute("%reset -f") - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Write turtle code to a file code = """ @@ -692,11 +692,11 @@ # Run code client.execute("runfile(r'{}')".format(to_text_string(p))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that the `tess` variable is defined client.inspect('tess') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] @@ -708,11 +708,11 @@ # Run code again client.execute("runfile(r'{}')".format(to_text_string(p))) - client.get_shell_msg(block=True, timeout=TIMEOUT) + client.get_shell_msg(timeout=TIMEOUT) # Verify that the `a` variable is defined client.inspect('a') - msg = client.get_shell_msg(block=True, timeout=TIMEOUT) + msg = client.get_shell_msg(timeout=TIMEOUT) content = msg['content'] assert content['found'] @@ -727,7 +727,7 @@ # Get current backend code = "import matplotlib; backend = matplotlib.get_backend()" client.execute(code, user_expressions={'output': 'backend'}) - reply = client.get_shell_msg(block=True, timeout=TIMEOUT) + reply = client.get_shell_msg(timeout=TIMEOUT) # Transform value obtained through user_expressions user_expressions = reply['content']['user_expressions'] @@ -754,9 +754,10 @@ pdb_obj = SpyderPdb() pdb_obj.curframe = inspect.currentframe() pdb_obj.completenames = lambda *ignore: ['baba'] - kernel._pdb_obj = pdb_obj + kernel.shell.pdb_session = pdb_obj match = kernel.do_complete('ba', 2) assert 'baba' in match['matches'] + pdb_obj.curframe = None @pytest.mark.parametrize("exclude_callables_and_modules", [True, False]) @@ -811,20 +812,96 @@ pdb_obj = SpyderPdb() pdb_obj.curframe = inspect.currentframe() pdb_obj.curframe_locals = pdb_obj.curframe.f_locals - kernel._pdb_obj = pdb_obj + kernel.shell.pdb_session = pdb_obj # Create a local variable. - kernel._pdb_obj.default('zz = 10') + kernel.shell.pdb_session.default('zz = 10') assert kernel.get_value('zz') == 10 # Run a list comprehension with this variable. - kernel._pdb_obj.default("compr = [zz * i for i in [1, 2, 3]]") + kernel.shell.pdb_session.default("compr = [zz * i for i in [1, 2, 3]]") assert kernel.get_value('compr') == [10, 20, 30] # Check that the variable is not reported as being part of globals. - kernel._pdb_obj.default("in_globals = 'zz' in globals()") + kernel.shell.pdb_session.default("in_globals = 'zz' in globals()") assert kernel.get_value('in_globals') == False + pdb_obj.curframe = None + pdb_obj.curframe_locals = None + +def test_comprehensions_with_locals_in_pdb_2(kernel): + """ + Test that evaluating comprehensions with locals works in Pdb. + + This is a regression test for spyder-ide/spyder#16790. + """ + pdb_obj = SpyderPdb() + pdb_obj.curframe = inspect.currentframe() + pdb_obj.curframe_locals = pdb_obj.curframe.f_locals + kernel.shell.pdb_session = pdb_obj + + # Create a local variable. + kernel.shell.pdb_session.default('aa = [1, 2]') + kernel.shell.pdb_session.default('bb = [3, 4]') + kernel.shell.pdb_session.default('res = []') + + # Run a list comprehension with this variable. + kernel.shell.pdb_session.default( + "for c0 in aa: res.append([(c0, c1) for c1 in bb])") + assert kernel.get_value('res') == [[(1, 3), (1, 4)], [(2, 3), (2, 4)]] + + pdb_obj.curframe = None + pdb_obj.curframe_locals = None + + +def test_namespaces_in_pdb(kernel): + """ + Test namespaces in pdb + """ + # Define get_ipython for timeit + get_ipython = lambda: kernel.shell + kernel.shell.user_ns["test"] = 0 + pdb_obj = SpyderPdb() + pdb_obj.curframe = inspect.currentframe() + pdb_obj.curframe_locals = pdb_obj.curframe.f_locals + kernel.shell.pdb_session = pdb_obj + + # Check adding something to globals works + pdb_obj.default("globals()['test2'] = 0") + assert pdb_obj.curframe.f_globals["test2"] == 0 + + if PY2: + # no error method in py2 + pdb_obj.curframe = None + pdb_obj.curframe_locals = None + return + + # Create wrapper to check for errors + old_error = pdb_obj.error + pdb_obj._error_occured = False + def error_wrapper(*args, **kwargs): + print(args, kwargs) + pdb_obj._error_occured = True + return old_error(*args, **kwargs) + pdb_obj.error = error_wrapper + + # Test globals are visible + pdb_obj.curframe.f_globals["test3"] = 0 + pdb_obj.default("%timeit test3") + assert not pdb_obj._error_occured + + # Test locals are visible + pdb_obj.curframe_locals["test4"] = 0 + pdb_obj.default("%timeit test4") + assert not pdb_obj._error_occured + + # Test user namespace is not visible + pdb_obj.default("%timeit test") + assert pdb_obj._error_occured + + pdb_obj.curframe = None + pdb_obj.curframe_locals = None + if __name__ == "__main__": pytest.main() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/customize/namespace_manager.py new/spyder-kernels-2.2.0/spyder_kernels/customize/namespace_manager.py --- old/spyder-kernels-2.1.3/spyder_kernels/customize/namespace_manager.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/customize/namespace_manager.py 2021-11-23 00:19:57.000000000 +0100 @@ -12,31 +12,6 @@ from spyder_kernels.py3compat import PY2 -def _get_globals_locals(): - """Return current namespace.""" - if get_ipython().kernel.is_debugging(): - pdb = get_ipython().kernel._pdb_obj - ns_locals = pdb.curframe_locals - ns_globals = pdb.curframe.f_globals - else: - ns_locals = None - ns_globals = get_ipython().user_ns - return ns_globals, ns_locals - - -def _set_globals_locals(ns_globals, ns_locals): - """Update current namespace.""" - if get_ipython().kernel.is_debugging(): - pdb = get_ipython().kernel._pdb_obj - pdb.curframe.f_globals.update(ns_globals) - if ns_locals: - pdb.curframe_locals.update(ns_locals) - else: - get_ipython().user_ns.update(ns_globals) - if ns_locals: - get_ipython().user_ns.update(ns_locals) - - class NamespaceManager(object): """ Get a namespace and set __file__ to filename for this namespace. @@ -61,14 +36,18 @@ Prepare the namespace. """ # Save previous __file__ + ipython_shell = get_ipython() if self.ns_globals is None: if self.current_namespace: - self.ns_globals, self.ns_locals = _get_globals_locals() + # stack_depth = parent of calling function + stack_depth = 2 + self.ns_globals = ipython_shell.get_global_scope(stack_depth) + self.ns_locals = ipython_shell.get_local_scope(stack_depth) if '__file__' in self.ns_globals: self._previous_filename = self.ns_globals['__file__'] self.ns_globals['__file__'] = self.filename else: - ipython_shell = get_ipython() + main_mod = ipython_shell.new_main_mod( self.filename, '__main__') self.ns_globals = main_mod.__dict__ @@ -80,7 +59,7 @@ self._reset_main = True # Save current namespace for access by variable explorer - get_ipython().kernel._running_namespace = ( + ipython_shell.kernel._running_namespace = ( self.ns_globals, self.ns_locals) if (self._file_code is not None @@ -104,14 +83,21 @@ """ Reset the namespace. """ - get_ipython().kernel._running_namespace = None + ipython_shell = get_ipython() + ipython_shell.kernel._running_namespace = None if self._previous_filename: self.ns_globals['__file__'] = self._previous_filename elif '__file__' in self.ns_globals: self.ns_globals.pop('__file__') if not self.current_namespace: - _set_globals_locals(self.ns_globals, self.ns_locals) + # stack_depth = parent of calling function + stack_depth = 2 + ns_globals = ipython_shell.get_global_scope(stack_depth) + ns_locals = ipython_shell.get_local_scope(stack_depth) + ns_globals.update(self.ns_globals) + if ns_locals and self.ns_locals: + ns_locals.update(self.ns_locals) if self._previous_main: sys.modules['__main__'] = self._previous_main diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/customize/spydercustomize.py new/spyder-kernels-2.2.0/spyder_kernels/customize/spydercustomize.py --- old/spyder-kernels-2.1.3/spyder_kernels/customize/spydercustomize.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/customize/spydercustomize.py 2021-11-23 00:19:57.000000000 +0100 @@ -469,9 +469,9 @@ ipython_shell.showtraceback(exception_only=True) except BaseException as error: if (isinstance(error, bdb.BdbQuit) - and ipython_shell.kernel._pdb_obj): + and ipython_shell.pdb_session): # Ignore BdbQuit if we are debugging, as it is expected. - ipython_shell.kernel._pdb_obj = None + ipython_shell.pdb_session = None elif post_mortem and isinstance(error, Exception): error_type, error, tb = sys.exc_info() post_mortem_excepthook(error_type, error, tb) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/customize/spyderpdb.py new/spyder-kernels-2.2.0/spyder_kernels/customize/spyderpdb.py --- old/spyder-kernels-2.1.3/spyder_kernels/customize/spyderpdb.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/customize/spyderpdb.py 2021-11-23 00:19:57.000000000 +0100 @@ -34,6 +34,18 @@ logger = logging.getLogger(__name__) +def uses_comprehension(code): + """Check if given code uses comprehensions.""" + comprehension_statements = ( + ast.ListComp, + ast.SetComp, + ast.GeneratorExp, + ast.DictComp + ) + nodes = ast.walk(ast.parse(code)) + return any(isinstance(node, comprehension_statements) for node in nodes) + + class DebugWrapper(object): """ Notifies the frontend when debugging starts/stops @@ -126,26 +138,21 @@ " command instead") return locals = self.curframe_locals - - # This is necessary to allow running comprehensions with the - # frame locals. It also fallbacks to the right globals if the - # user wants to work with them instead. - # See spyder-ide/spyder#13909. - if not 'globals()' in line: - ns = self.curframe.f_globals.copy() - ns.update(locals) - else: - ns = self.curframe.f_globals + globals = self.curframe.f_globals if self.pdb_use_exclamation_mark: # Find pdb commands executed without ! cmd, arg, line = self.parseline(line) if cmd: cmd_in_namespace = ( - cmd in ns or cmd in builtins.__dict__) + cmd in globals + or cmd in locals + or cmd in builtins.__dict__ + ) # Special case for quit and exit if cmd in ("quit", "exit"): - if cmd in ns and isinstance(ns[cmd], ZMQExitAutocall): + if cmd in globals and isinstance( + globals[cmd], ZMQExitAutocall): # Use the pdb call cmd_in_namespace = False cmd_func = getattr(self, 'do_' + cmd, None) @@ -184,7 +191,34 @@ sys.displayhook = self.displayhook if execute_events: get_ipython().events.trigger('pre_execute') - exec(code, ns, locals) + + # Mitigates a CPython bug (https://bugs.python.org/issue41918) + # that prevents running comprehensions with the frame locals + # in Pdb. + # See https://bugs.python.org/issue21161 and + # spyder-ide/spyder#13909. + if uses_comprehension(line): + # There are three potential problems with this approach: + # 1. If the code access a globals variable that is + # masked by a locals variable, it will get the locals + # one. + # 2. Any edit to that variable will be lost. + # 3. The globals will appear to contain all the locals + # variables. + # 4. Any new locals variable will be saved to globals + # instead + fake_globals = globals.copy() + fake_globals.update(locals) + locals_keys = locals.keys() + # Don't pass locals, solves spyder-ide/spyder#16790 + exec(code, fake_globals) + # Avoid mixing locals and globals + for key in locals_keys: + locals[key] = fake_globals.pop(key, None) + globals.update(fake_globals) + else: + exec(code, globals, locals) + if execute_events: get_ipython().events.trigger('post_execute') finally: @@ -255,8 +289,8 @@ def stop_here(self, frame): """Check if pdb should stop here.""" if (frame is not None - and frame.f_locals.get( - "__tracebackhide__", False) == "__pdb_exit__"): + and "__tracebackhide__" in frame.f_locals + and frame.f_locals["__tracebackhide__"] == "__pdb_exit__"): self.onecmd('exit') return False @@ -504,8 +538,7 @@ Register Pdb session after reset. """ super(SpyderPdb, self).reset() - kernel = get_ipython().kernel - kernel._register_pdb_session(self) + get_ipython().pdb_session = self def do_debug(self, arg): """ @@ -528,8 +561,7 @@ exc_info = sys.exc_info()[:2] self.error( traceback.format_exception_only(*exc_info)[-1].strip()) - kernel = get_ipython().kernel - kernel._register_pdb_session(self) + get_ipython().pdb_session = self def user_return(self, frame, return_value): """This function is called when a return trap is set here.""" @@ -737,10 +769,10 @@ def enter_debugger(filename, continue_if_has_breakpoints, code_format): """Enter debugger. Code format should be a format that accept filename.""" - kernel = get_ipython().kernel - recursive = kernel.is_debugging() + shell = get_ipython() + recursive = shell.is_debugging() if recursive: - parent_debugger = kernel._pdb_obj + parent_debugger = shell.pdb_session sys.settrace(None) globals = parent_debugger.curframe.f_globals locals = parent_debugger.curframe_locals @@ -770,7 +802,7 @@ # Reset parent debugger sys.settrace(parent_debugger.trace_dispatch) parent_debugger.lastcmd = debugger.lastcmd - kernel._register_pdb_session(parent_debugger) + shell.pdb_session = parent_debugger else: # The breakpoint might not be in the cell debugger.run(code) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spyder-kernels-2.1.3/spyder_kernels/utils/nsview.py new/spyder-kernels-2.2.0/spyder_kernels/utils/nsview.py --- old/spyder-kernels-2.1.3/spyder_kernels/utils/nsview.py 2021-10-02 19:10:06.000000000 +0200 +++ new/spyder-kernels-2.2.0/spyder_kernels/utils/nsview.py 2021-11-23 00:19:57.000000000 +0100 @@ -89,6 +89,11 @@ isinstance(item.shape, (tuple, np.integer))): try: if item.shape: + # This is needed since values could return as + # `shape` an instance of a `tuple` subclass. + # See spyder-ide/spyder#16348 + if isinstance(item.shape, tuple): + return tuple(item.shape) return item.shape else: # Scalar value