Re: Corrputed stacktrace?

2014-06-04 Thread Nikolaus Rath
Chris Angelico ros...@gmail.com writes:
 On Wed, Jun 4, 2014 at 12:20 PM, Nikolaus Rath nikol...@rath.org wrote:
   File /usr/lib/python3.3/threading.py, line 878 in _bootstrap

 Can you replicate the problem in a non-threaded environment? Threads
 make interactive debugging very hairy.

Hmm. I could try to run the server thread in a separate process. I'll
try that and report back.


Thanks for the suggestion,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
-- 
https://mail.python.org/mailman/listinfo/python-list


Corrputed stacktrace?

2014-06-03 Thread Nikolaus Rath
Hello,

I'm trying to debug a problem. As far as I can tell, one of my methods
is called at a point where it really should not be called. When setting
a breakpoint in the function, I'm getting this:

 /home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py(693)close()
- if not self.md5_checked:
(Pdb) bt
  /usr/lib/python3.3/threading.py(878)_bootstrap()
- self._bootstrap_inner()
  /usr/lib/python3.3/threading.py(901)_bootstrap_inner()
- self.run()
  /usr/lib/python3.3/threading.py(858)run()
- self._target(*self._args, **self._kwargs)
  /usr/lib/python3.3/socketserver.py(610)process_request_thread()
- self.finish_request(request, client_address)
  /usr/lib/python3.3/socketserver.py(345)finish_request()
- self.RequestHandlerClass(request, client_address, self)
  /usr/lib/python3.3/socketserver.py(666)__init__()
- self.handle()
  /home/nikratio/in-progress/s3ql/tests/mock_server.py(77)handle()
- return super().handle()
  /usr/lib/python3.3/http/server.py(402)handle()
- self.handle_one_request()
  /usr/lib/python3.3/http/server.py(388)handle_one_request()
- method()
  /home/nikratio/in-progress/s3ql/tests/mock_server.py(169)do_GET()
- q = parse_url(self.path)
  /home/nikratio/in-progress/s3ql/tests/mock_server.py(52)parse_url()
- p.params = urllib.parse.parse_qs(q.query)
  /usr/lib/python3.3/urllib/parse.py(553)parse_qs()
- encoding=encoding, errors=errors)
  /usr/lib/python3.3/urllib/parse.py(585)parse_qsl()
- pairs = [s2 for s1 in qs.split('') for s2 in s1.split(';')]
  /usr/lib/python3.3/urllib/parse.py(585)listcomp()
- pairs = [s2 for s1 in qs.split('') for s2 in s1.split(';')]
  /home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py(853)close()
- self.fh.close()
 /home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py(693)close()
- if not self.md5_checked:


To me this does not make any sense.

Firstly, the thread that is (apparently) calling close should never ever
reach code in common.py. This thread is executing a socketserver handler
that is entirely contained in mock_server.py and only communicates with
the rest of the program via tcp.

Secondly, the backtrace does not make sense. How can evaluation of 

 pairs = [s2 for s1 in qs.split('') for s2 in s1.split(';')]

in urllib/parse.py() result in a method call in backends/common.py?
There is no trickery going on, qs is a regular string:

(Pdb) up
(Pdb) up
(Pdb) up
(Pdb) l
580 into Unicode characters, as accepted by the bytes.decode() 
method.
581 
582 Returns a list, as G-d intended.
583 
584 qs, _coerce_result = _coerce_args(qs)
585  - pairs = [s2 for s1 in qs.split('') for s2 in s1.split(';')]
586 r = []
587 for name_value in pairs:
588 if not name_value and not strict_parsing:
589 continue
590 nv = name_value.split('=', 1)
(Pdb) whatis qs
class 'str'
(Pdb) p qs
''
(Pdb)

I have also tried to get a backtrace with the faulthandler module, but
it gives the same result:

Thread 0x7f7dafdb4700:
  File /usr/lib/python3.3/cmd.py, line 126 in cmdloop
  File /usr/lib/python3.3/pdb.py, line 318 in _cmdloop
  File /usr/lib/python3.3/pdb.py, line 345 in interaction
  File /usr/lib/python3.3/pdb.py, line 266 in user_line
  File /usr/lib/python3.3/bdb.py, line 65 in dispatch_line
  File /usr/lib/python3.3/bdb.py, line 47 in trace_dispatch
  File /home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py, line 693 in 
clos
  File /home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py, line 853 
in c
  File /usr/lib/python3.3/urllib/parse.py, line 585 in listcomp
  File /usr/lib/python3.3/urllib/parse.py, line 585 in parse_qsl
  File /usr/lib/python3.3/urllib/parse.py, line 553 in parse_qs
  File /home/nikratio/in-progress/s3ql/tests/mock_server.py, line 52 in 
parse_url
  File /home/nikratio/in-progress/s3ql/tests/mock_server.py, line 169 in 
do_GET
  File /usr/lib/python3.3/http/server.py, line 388 in handle_one_request
  File /usr/lib/python3.3/http/server.py, line 402 in handle
  File /home/nikratio/in-progress/s3ql/tests/mock_server.py, line 77 in handle
  File /usr/lib/python3.3/socketserver.py, line 666 in __init__
  File /usr/lib/python3.3/socketserver.py, line 345 in finish_request
  File /usr/lib/python3.3/socketserver.py, line 610 in process_request_thread
  File /usr/lib/python3.3/threading.py, line 858 in run
  File /usr/lib/python3.3/threading.py, line 901 in _bootstrap_inner
  File /usr/lib/python3.3/threading.py, line 878 in _bootstrap


Is it possible that the stack got somehow corrupted?

Does anyone have a suggestion how I could go about debugging this?

I am using Python 3.3.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Corrputed stacktrace?

2014-06-03 Thread Chris Angelico
On Wed, Jun 4, 2014 at 12:20 PM, Nikolaus Rath nikol...@rath.org wrote:
   File /usr/lib/python3.3/threading.py, line 878 in _bootstrap

Can you replicate the problem in a non-threaded environment? Threads
make interactive debugging very hairy.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list