Re: ipdb throws AttributeError in autoreload.py with 2.2.2

[Jason Johns]

Wow, that was quick! Thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6b7125aa-404e-4678-8c19-578533cb52fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ipdb throws AttributeError in autoreload.py with 2.2.2

[Tom Forbes]

IPython seems to override `__main__`, which I certainly did not expect: 
https://github.com/ipython/ipython/blob/257e4c45563598810b0a771de286004617510fc1/IPython/core/debugger.py#L238
 

 and perhaps 
https://github.com/ipython/ipython/blob/7b42de99c651de35f487adea3f57824ad97bcd74/IPython/testing/globalipapp.py#L115
 
.

I’ve made a ticket here: https://code.djangoproject.com/ticket/30588#ticket 
, and a PR to fix it: 
https://github.com/django/django/pull/11507 


> , and I know the autoreload has been an area of some frustration lately and 
> as such I'm not sure of the best approach for handling this case.

I do feel responsible for the frustration, sorry! My advice to anyone that 
hears a little voice in their head asking “how hard can building an autoloader 
be?”: ignore it. That being said, I think we’ve ironed out the worst of the 
issues, despite this one being pretty bad :/

> On 23 Jun 2019, at 16:46, Jason Johns  wrote:
> 
> Hello,
> 
> About two weeks ago, work updated the base docker image to use django 2.2.2 
> and I noticed an issue right off the bat with using ipdb 0.12 and ipython 
> 7.5.0 (which are also defaults for the docker image).  I really didn't have 
> the time to dig more, and got along well enough with pdb in the meantime.  
> This weekend, I had some free time to poke around, and these are my findings:
> 
> 
> In a DRF serializer helper method, I import ipdb and set a breakpoint.  The 
> API call executes and stops here.  After anywhere from 1 to 3 seconds pause 
> at the breakpoint, I then see the following output from ipdb> and the 
> runserver process exits.
> > /orm/service/api/serializers/helpers/facet_generator.py(62)__make_facet()
>  61 import ipdb; ipdb.set_trace()
> ---> 62 log.info(f'Facet field: {facet_field}')
>  63 log.info(f'Facet values: {facet_values}')
> 
> ipdb> Traceback (most recent call last):
>   File "/orm/manage.py", line 15, in 
> execute_from_command_line(sys.argv)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
> line 381, in execute_from_command_line
> utility.execute()
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
> line 375, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 
> 323, in run_from_argv
> self.execute(*args, **cmd_options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 60, in execute
> super().execute(*args, **options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 
> 364, in execute
> output = self.handle(*args, **options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 95, in handle
> self.run(**options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 102, in run
> autoreload.run_with_reloader(self.inner_run, **options)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 585, in run_with_reloader
> start_django(reloader, main_func, *args, **kwargs)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 570, in start_django
> reloader.run(django_main_thread)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 288, in run
> self.run_loop()
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 294, in run_loop
> next(ticker)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 334, in tick
> for filepath, mtime in self.snapshot_files():
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 350, in snapshot_files
> for file in self.watched_files():
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 249, in watched_files
> yield from iter_all_python_module_files()
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 103, in iter_all_python_module_files
> return iter_modules_and_files(modules, frozenset(_error_files))
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 120, in iter_modules_and_files
> sys_file_paths.append(module.__file__)
> AttributeError: module '__main__' has no attribute '__file__'
> 
> If you suspect this is an IPython bug, please report it at:
> https://github.com/ipython/ipython/issues
> or 

ipdb throws AttributeError in autoreload.py with 2.2.2

[Jason Johns]

Hello,

About two weeks ago, work updated the base docker image to use django 2.2.2 
and I noticed an issue right off the bat with using ipdb 0.12 and ipython 
7.5.0 (which are also defaults for the docker image).  I really didn't have 
the time to dig more, and got along well enough with pdb in the meantime.  
This weekend, I had some free time to poke around, and these are my 
findings:


In a DRF serializer helper method, I import ipdb and set a breakpoint.  The 
API call executes and stops here.  After anywhere from 1 to 3 seconds pause 
at the breakpoint, I then see the following output from ipdb> and the 
runserver process exits.
> /orm/service/api/serializers/helpers/facet_generator.py(62)__make_facet()
 61 import ipdb; ipdb.set_trace()
---> 62 log.info(f'Facet field: {facet_field}')
 63 log.info(f'Facet values: {facet_values}')

ipdb> Traceback (most recent call last):
  File "/orm/manage.py", line 15, in 
execute_from_command_line(sys.argv)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
line 381, in execute_from_command_line
utility.execute()
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/base.py", 
line 323, in run_from_argv
self.execute(*args, **cmd_options)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
 
line 60, in execute
super().execute(*args, **options)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/base.py", 
line 364, in execute
output = self.handle(*args, **options)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
 
line 95, in handle
self.run(**options)
  File 
"/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
 
line 102, in run
autoreload.run_with_reloader(self.inner_run, **options)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 585, in run_with_reloader
start_django(reloader, main_func, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 570, in start_django
reloader.run(django_main_thread)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 288, in run
self.run_loop()
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 294, in run_loop
next(ticker)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 334, in tick
for filepath, mtime in self.snapshot_files():
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 350, in snapshot_files
for file in self.watched_files():
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 249, in watched_files
yield from iter_all_python_module_files()
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 103, in iter_all_python_module_files
return iter_modules_and_files(modules, frozenset(_error_files))
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
line 120, in iter_modules_and_files
sys_file_paths.append(module.__file__)
AttributeError: module '__main__' has no attribute '__file__'

If you suspect this is an IPython bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-...@python.org

You can print a more detailed traceback right now with "%tb", or use 
"%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/IPython/core/history.py", 
line 780, in writeout_cache
self._writeout_input_cache(conn)
  File "/usr/local/lib/python3.7/site-packages/IPython/core/history.py", 
line 764, in _writeout_input_cache
(self.session_number,)+line)
sqlite3.ProgrammingError: SQLite objects created in a thread can only be 
used in that same thread. The object was created in thread id 
139743563077376 and this is thread id 139743865329408.



To be sure it wasn't just the project's fault, I checked out another team's 
project and got the same error.  This seems to be isolated to just ipdb.  
Using pdb does not show the same errors

Then, I pulled the latest from django github and confirmed that it is 
probably related to the changes in iter_modules_and_files 

 between 
2.2.1 and 2.2.2.  Looking at ipdb 
's package structure, I 
think the problem is the usage of __main__.py, because it apparently does 
not