Re: [Zope-dev] Segfault in zope.configuration
On Tue, Apr 6, 2010 at 10:25 PM, Martin Aspeli optilude+li...@gmail.com wrote: So this is still in pyexpat C code as far as I can tell. :-( This is saddening. I've not managed a 64-bit sandbox, which I suspect is what I really need to debug that. Will shoot for this weekend, since last didn't pan out. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Tue, Apr 6, 2010 at 10:25 PM, Martin Aspelioptilude+li...@gmail.com wrote: So this is still in pyexpat C code as far as I can tell. :-( This is saddening. But on the other hand, your dedication in helping me find a fix is heartening. ;) I've not managed a 64-bit sandbox, which I suspect is what I really need to debug that. Will shoot for this weekend, since last didn't pan out. I have a slight suspicion that lxml is involved somewhere. I've managed to make it crash reliably simply by doing a parse of an XML file, and I think such a parse may be happening as a side effect of a module import caused by that very ZCML file. That doesn't really explain why pdb dies when it dies, but I don't know how the C/Python boundary is working here (especially since I can't really see the C code). I'm re-building for the n-th time, so I'll check again tomorrow when that's done. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Wed, Apr 7, 2010 at 9:29 AM, Martin Aspeli optilude+li...@gmail.com wrote: I have a slight suspicion that lxml is involved somewhere. I've managed to make it crash reliably simply by doing a parse of an XML file, and I think such a parse may be happening as a side effect of a module import caused by that very ZCML file. If it can be nailed on lxml, I'll be happy. :-) Not because I want lxml to be at fault so much as I want Expat not to be. Invoking lxml as a side effect of import seems like it should be fairly easy to identify and comment out for testing purposes; if the ZCML configuration can then complete, you've implicated your suspect. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Fri, 02 Apr 2010 11:29:06 +0800 Martin Aspeli optilude+li...@gmail.com wrote: Hello, The weird thing is that it's not parsing of any file: it happens during a standard configure.zcml (for the collective.xdv package), but z3c.autoinclude itself is invoked from another ZCML file, so it must've been able to read that. Any tips on how to debug or similar experiences would be appreciated! Can you reproduce this, reliably? Yes. I already got segfaults myself with zope.component in Zope 2.11 when doing race conditions: two threads registering the same adapter at the same time. It corrupted Python memory leading to segfault few seconds after. Maybe somehow you have the same thing. I don't image writing a test for it, so I just adding a lock around all registration steps in the ZCA, and it worked fine. Best regards, Sylvain, -- Sylvain Viollon -- Infrae t +31 10 243 7051 -- http://infrae.com Hoevestraat 10 3033GC Rotterdam -- The Netherlands ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Martin Aspeli wrote:
Hi,
I'm not sure if this is a Python issue or a zope issue. We're getting a
segfault on 64-bit SuSE Linux (SLES 11), originating from
z3c.autoinclude, which in turn called zope.configuration'sinclude /
implementation. This calls expat, which then crashes (no error, log
message, or core file, but it has all the markings of a segfault) during
parsing of the file.
The weird thing is that it's not parsing of any file: it happens during
a standard configure.zcml (for the collective.xdv package), but
z3c.autoinclude itself is invoked from another ZCML file, so it must've
been able to read that.
I tried running it through gdb, and got:
(gdb) run
Starting program: /home/osc/python-env/bin/python2.6 ./bin/instance1 fg
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
2010-04-07 09:38:22 INFO ZServer HTTP server started at Wed Apr 7
09:38:22 2010
Hostname: 0.0.0.0
Port: 8801
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
Program exited normally.
.. which is not very helpful.
The status code is indeed zero.
(python-env)o...@lwpn-osb-webback-2:~/osc ./bin/instance1 fg2010-04-07
09:39:21 INFO ZServer HTTP server started at Wed Apr 7 09:39:21 2010
Hostname: 0.0.0.0
Port: 8801
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
(python-env)o...@lwpn-osb-webback-2:~/osc echo $?
0
Next, I tried to pdb from roughly where I know it dies. Sorry about the
long text here. It's a bit weird, but I'll explain what's happening.
(python-env)o...@lwpn-osb-webback-2:~/osc ./bin/instance1 fg2010-04-07
09:47:37 INFO ZServer HTTP server started at Wed Apr 7 09:47:37 2010
Hostname: 0.0.0.0
Port: 8801
This is where the brekapoint was set, in z3c.autoinclude's
includePlugins / handler. This looks for packages with a particular
entry point and loads their meta.zcml, then configure.zcml. The problem
happens during configure.zcml loading.
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(104)includePluginsDirective()
- for filename in zcml_to_look_for:
snip /
We've now stepped into the function that does the real work:
- def includeZCMLGroup(_context, info, filename, override=False):
(Pdb) l
10from z3c.autoinclude.plugin import PluginFinder
11
12import logging
13log = logging.getLogger(z3c.autoinclude)
14
15 -def includeZCMLGroup(_context, info, filename, override=False):
16includable_zcml = info[filename]
17# ^^ a key error would mean that we are trying to include a
group of ZCML
18#with a filename that wasn't ever searched for. that
*should* be an error
19
20zcml_context = repr(_context.info)
snip /
This quickly gets to here, where it's loading a package and calling
zope.configuration's include / handler.
(Pdb) l
22for dotted_name in includable_zcml:
23log.debug('including file %s from package %s at %s',
filename, dotted_name, zcml_context)
24
25for dotted_name in includable_zcml:
26includable_package = resolve(dotted_name)
27 -if override:
28includeOverrides(_context, filename, includable_package)
29else:
30include(_context, filename, includable_package)
31
32
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(30)includeZCMLGroup()
- include(_context, filename, includable_package)
It's not na override, so we end up in zope.configuration now.
--Call--
/home/osc/osc/eggs/zope.configuration-3.6.0-py2.6.egg/zope/configuration/xmlconfig.py(487)include()
- def include(_context, file=None, package=None, files=None):
(Pdb) n
/home/osc/osc/eggs/zope.configuration-3.6.0-py2.6.egg/zope/configuration/xmlconfig.py(493)include()
- if files:
(Pdb) l
488 Include a zcml file
489
490 See examples in tests/text_xmlconfig.py
491
492
493 - if files:
494 if file:
495 raise ValueError(Must specify only one of file or files)
496 elif not file:
497 file = 'configure.zcml'
498
(Pdb) l
499 # BBB 2006/12/19 -- to be removed after 12 months
500 # This is a backward-compatibility support for old site.conf
501
502 if package and (package.__name__ == 'zope.app'):
503
Re: [Zope-dev] Segfault in zope.configuration
Martin Aspeli wrote:
At this point, something is printed to the console. collective.wtf is a
dependency of lw.portal, and its ZCML is being included from lw.portal.
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
Okay, I found out what it does next: It now processes z3c.autoinclude's
includeDependencies /, which basically just goes through the
dependencies in setup.py's install_requires and tries to load a
configure.zcml for each. It successfully does that for a number of
dependencies, up until it hits collective.xdv (same package I saw the
problem with when I set up a minimal buildout that just had Plone and
collective.xdv in it).
The pdb log is:
(Pdb) l
43
44def includeDependenciesDirective(_context, package):
45
46import pdb; pdb.set_trace()
47
48 -if api.dependencies_disabled():
49log.warn('z3c.autoinclude.dependency is disabled but is
being invoked by %s' % _context.info)
50return
51
52dist = distributionForPackage(package)
53info =
DependencyFinder(dist).includableInfo(['configure.zcml', 'meta.zcml'])
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(52)includeDependenciesDirective()
- dist = distributionForPackage(package)
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(53)includeDependenciesDirective()
- info = DependencyFinder(dist).includableInfo(['configure.zcml',
'meta.zcml'])
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(55)includeDependenciesDirective()
- includeZCMLGroup(_context, info, 'meta.zcml')
(Pdb) pp info
{'configure.zcml': ['collective.xdv', 'five.grok', 'z3c.jbot'],
'meta.zcml': ['five.grok', 'z3c.jbot']}
(Pdb) l
50return
51
52dist = distributionForPackage(package)
53info =
DependencyFinder(dist).includableInfo(['configure.zcml', 'meta.zcml'])
54
55 -includeZCMLGroup(_context, info, 'meta.zcml')
56includeZCMLGroup(_context, info, 'configure.zcml')
57
58def includeDependenciesOverridesDirective(_context, package):
59
60if api.dependencies_disabled():
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(56)includeDependenciesDirective()
- includeZCMLGroup(_context, info, 'configure.zcml')
(Pdb) s
--Call--
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(15)includeZCMLGroup()
- def includeZCMLGroup(_context, info, filename, override=False):
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(16)includeZCMLGroup()
- includable_zcml = info[filename]
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(20)includeZCMLGroup()
- zcml_context = repr(_context.info)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(25)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(26)includeZCMLGroup()
- includable_package = resolve(dotted_name)
(Pdb) l
21
22for dotted_name in includable_zcml:
23log.debug('including file %s from package %s at %s',
filename, dotted_name, zcml_context)
24
25for dotted_name in includable_zcml:
26 -includable_package = resolve(dotted_name)
27if override:
28includeOverrides(_context, filename, includable_package)
29else:
30include(_context, filename,
Re: [Zope-dev] Segfault in zope.configuration
A: Because it destroys the flow of the conversation Q: Why is it bad? A: No, it's bad. Q: Should I top post in replies to mailing lists? On Fri, Apr 02, 2010 at 12:26:59PM +0800, Tim Hoffman wrote: Hi Martin Given that its failing on a call to libc I would do a quick run with strace and ltrace. That was my segfault, not Martin's. I suspect hardware, as it's very non-deterministic (a small pure-Python script that runs every 5 minutes segfaults once in a few months). (In your case ltrace will be probably be more useful.) You will be able to see the calls into the libraries and you should see immediately if your segfault is inside expat or not. (ltrace output will be very long ;-) On Fri, Apr 2, 2010 at 11:29 AM, Martin Aspeli optilude+li...@gmail.com wrote: Marius Gedminas wrote: On Thu, Apr 01, 2010 at 06:07:26PM +0800, Martin Aspeli wrote: I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration'sinclude / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. On recent Ubuntu versions segfaults get recorded in dmesg and end up in /var/log/kern.log like this: Mar 22 02:10:03 fridge kernel: [7804719.781375] python[10447]: segfault at 7fff84a7c000 ip 7fff83aae7cb sp 7fff84a7bd98 error 7 in libc-2.9.so[7fff83a2b000+168000] I assume that's a standard kernel feature now (we've 2.6.28, x86_64). Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Fri, Apr 02, 2010 at 11:29:06AM +0800, Martin Aspeli wrote: I'd be reaching for python/expat debug symbols and gdb, probably. I'm not sure how to do that. My gdb fu is *very* limited. Have you used pdb? gdb is mostly the same. gdb python and when it crashes, type 'bt' at the (gdb) prompt. The C stack trace reads backwards when compared to Python tracebacks: the main function is at the bottom, the innermost one at the top. I also assume I'd need to re-build python and/or expat for this? Not necessarily. Even bare gdb will tell you which library was it that got the segfault. If you want more, you can usually install debug symbols for Python and libexpat into /usr/lib/debug and re-run gdb, without having to rebuild your own python/libexpat. I don't know about SLES. I've tried Googling and only found http://en.opensuse.org/Packaging/Debuginfo which appears to be a page for packagers, and links to http://fedoraproject.org/wiki/StackTraces for end-user how to get a useful stack trace tutorial. AFAICs if you have a 'debuginfo-install' command (part of yum-utils) on your server, you should be golden. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
sorry On Fri, Apr 2, 2010 at 9:07 PM, Marius Gedminas mar...@gedmin.as wrote: A: Because it destroys the flow of the conversation Q: Why is it bad? A: No, it's bad. Q: Should I top post in replies to mailing lists? On Fri, Apr 02, 2010 at 12:26:59PM +0800, Tim Hoffman wrote: Hi Martin Given that its failing on a call to libc I would do a quick run with strace and ltrace. That was my segfault, not Martin's. I suspect hardware, as it's very non-deterministic (a small pure-Python script that runs every 5 minutes segfaults once in a few months). (In your case ltrace will be probably be more useful.) You will be able to see the calls into the libraries and you should see immediately if your segfault is inside expat or not. (ltrace output will be very long ;-) On Fri, Apr 2, 2010 at 11:29 AM, Martin Aspeli optilude+li...@gmail.com wrote: Marius Gedminas wrote: On Thu, Apr 01, 2010 at 06:07:26PM +0800, Martin Aspeli wrote: I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration'sinclude / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. On recent Ubuntu versions segfaults get recorded in dmesg and end up in /var/log/kern.log like this: Mar 22 02:10:03 fridge kernel: [7804719.781375] python[10447]: segfault at 7fff84a7c000 ip 7fff83aae7cb sp 7fff84a7bd98 error 7 in libc-2.9.so[7fff83a2b000+168000] I assume that's a standard kernel feature now (we've 2.6.28, x86_64). Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAku17BQACgkQkVdEXeem149bGwCfYKlubUF9+xxJ7oOM0hhRc6o9 BOMAniwIKQNDr+h224tdW7nonPoYtctr =7p1x -END PGP SIGNATURE- ___ Zope-Dev maillist - zope-...@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Thu, Apr 1, 2010 at 6:07 AM, Martin Aspeli optilude+li...@gmail.com wrote: Any tips on how to debug or similar experiences would be appreciated! If you're on some Unix flavor, you should be able to deconstruct the return code from the runzope process to determine if the app was killed by a signal and, if so, which. If it's a segfault, you should be looking for signal.SIGSEGV. I'd be very interested in hearing if Expat is implicated. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Thu, Apr 1, 2010 at 6:07 AM, Martin Aspelioptilude+li...@gmail.com wrote: Any tips on how to debug or similar experiences would be appreciated! If you're on some Unix flavor, you should be able to deconstruct the return code from the runzope process to determine if the app was killed by a signal and, if so, which. If it's a segfault, you should be looking for signal.SIGSEGV. I can't check until after the long weekend now, but I'm pretty sure it is. I'd be very interested in hearing if Expat is implicated. I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Thu, Apr 1, 2010 at 7:29 AM, Martin Aspeli optilude+li...@gmail.com wrote: I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. If you can point me at the ZCML file you were trying to parse (or email it to me privately if it's not in a public repository), I'd be glad to see what I can turn up. I'll also need to know what version of Python you're using. Thanks! -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Thu, Apr 1, 2010 at 7:29 AM, Martin Aspelioptilude+li...@gmail.com wrote: I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. If you can point me at the ZCML file you were trying to parse (or email it to me privately if it's not in a public repository), I'd be glad to see what I can turn up. The one that blew up is http://svn.plone.org/svn/collective/collective.xdv/trunk/collective/xdv/configure.zcml, at least that's what z3c.autoinclude was calling the include directive handler on. I'll also need to know what version of Python you're using. It's a fully patched SLES11. I think it's 2.6.4 (it's definitely 2.6), though unfortunately I don't have access here on the weekend. I've used virtualenv --no-site-packages to get a clean Python (SLES installs all kinds of crap in the global site), and then used that to run a buildout. Not sure how SuSE compiles their Python. One thing to note is that SLES has /usr/lib and /usr/lib64. Everything used in our build is (or should be to my knowledge) 64-bit, so linked against the 64-bit versions. Then again, I've had a lot of trouble getting things to build properly on SLES. The buildout's got client-specific stuff in it, but a minimal one would basically just install Plone + collective.xdv in a plone.recipe.zope2instance with the KGS at http://good-py.appspot.com/release/dexterity/2.0-next?plone=4.0b1-1 Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Aspeli wrote: Fred Drake wrote: On Thu, Apr 1, 2010 at 7:29 AM, Martin Aspelioptilude+li...@gmail.com wrote: I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. If you can point me at the ZCML file you were trying to parse (or email it to me privately if it's not in a public repository), I'd be glad to see what I can turn up. The one that blew up is http://svn.plone.org/svn/collective/collective.xdv/trunk/collective/xdv/configure.zcml, at least that's what z3c.autoinclude was calling the include directive handler on. I'll also need to know what version of Python you're using. It's a fully patched SLES11. I think it's 2.6.4 (it's definitely 2.6), though unfortunately I don't have access here on the weekend. I've used virtualenv --no-site-packages to get a clean Python (SLES installs all kinds of crap in the global site), and then used that to run a buildout. Not sure how SuSE compiles their Python. One thing to note is that SLES has /usr/lib and /usr/lib64. Everything used in our build is (or should be to my knowledge) 64-bit, so linked against the 64-bit versions. Then again, I've had a lot of trouble getting things to build properly on SLES. The buildout's got client-specific stuff in it, but a minimal one would basically just install Plone + collective.xdv in a plone.recipe.zope2instance with the KGS at http://good-py.appspot.com/release/dexterity/2.0-next?plone=4.0b1-1 One fault-isolation strategy would be to try building Python from source into someplace like /opt/Python-2.6.5, and see if the segfault recurs under it. /me is deeply suspicious of *any* distro-provided python, ever. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAku04yYACgkQ+gerLs4ltQ4x0gCg2Ak5GIrBGggpXYpnCzogWSlD tugAoNWewaihiPqG06QjnOlKkUFjWO8v =Umi6 -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Thu, Apr 1, 2010 at 2:17 PM, Tres Seaver tsea...@palladion.com wrote: /me is deeply suspicious of *any* distro-provided python, ever. I'm also suspicious of 64-bit builds, given that I'm not using one on my dev machine. I've picked up the OpenSuSE installation ISOs; hopefully that's close enough to the SLES distributions that I'll be able to construct useful VMs. Martin: If I can't manage a useful 64-bit VM, I'll probably send you a script to try out. It'll likely be this weekend before I can really dive in. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
On Thu, Apr 01, 2010 at 06:07:26PM +0800, Martin Aspeli wrote: I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration's include / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. On recent Ubuntu versions segfaults get recorded in dmesg and end up in /var/log/kern.log like this: Mar 22 02:10:03 fridge kernel: [7804719.781375] python[10447]: segfault at 7fff84a7c000 ip 7fff83aae7cb sp 7fff84a7bd98 error 7 in libc-2.9.so[7fff83a2b000+168000] I assume that's a standard kernel feature now (we've 2.6.28, x86_64). The weird thing is that it's not parsing of any file: it happens during a standard configure.zcml (for the collective.xdv package), but z3c.autoinclude itself is invoked from another ZCML file, so it must've been able to read that. Any tips on how to debug or similar experiences would be appreciated! Can you reproduce this, reliably? I'd be reaching for python/expat debug symbols and gdb, probably. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Aspeli wrote: Fred Drake wrote: On Thu, Apr 1, 2010 at 7:29 AM, Martin Aspelioptilude+li...@gmail.com wrote: I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. If you can point me at the ZCML file you were trying to parse (or email it to me privately if it's not in a public repository), I'd be glad to see what I can turn up. The one that blew up is http://svn.plone.org/svn/collective/collective.xdv/trunk/collective/xdv/configure.zcml, at least that's what z3c.autoinclude was calling the include directive handler on. I'll also need to know what version of Python you're using. It's a fully patched SLES11. I think it's 2.6.4 (it's definitely 2.6), though unfortunately I don't have access here on the weekend. I've used virtualenv --no-site-packages to get a clean Python (SLES installs all kinds of crap in the global site), and then used that to run a buildout. Not sure how SuSE compiles their Python. One thing to note is that SLES has /usr/lib and /usr/lib64. Everything used in our build is (or should be to my knowledge) 64-bit, so linked against the 64-bit versions. Then again, I've had a lot of trouble getting things to build properly on SLES. The buildout's got client-specific stuff in it, but a minimal one would basically just install Plone + collective.xdv in a plone.recipe.zope2instance with the KGS at http://good-py.appspot.com/release/dexterity/2.0-next?plone=4.0b1-1 One fault-isolation strategy would be to try building Python from source into someplace like /opt/Python-2.6.5, and see if the segfault recurs under it. I actually tried this before the virtualenv, and couldn't get Python to build cleanly. I forget the details right now (it's been a frustrating week, server wise), but trying to build it again is definitely on my list. /me is deeply suspicious of *any* distro-provided python, ever. Me too. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Marius Gedminas wrote: On Thu, Apr 01, 2010 at 06:07:26PM +0800, Martin Aspeli wrote: I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration'sinclude / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. On recent Ubuntu versions segfaults get recorded in dmesg and end up in /var/log/kern.log like this: Mar 22 02:10:03 fridge kernel: [7804719.781375] python[10447]: segfault at 7fff84a7c000 ip 7fff83aae7cb sp 7fff84a7bd98 error 7 in libc-2.9.so[7fff83a2b000+168000] I assume that's a standard kernel feature now (we've 2.6.28, x86_64). Interesting. I'll check. The weird thing is that it's not parsing of any file: it happens during a standard configure.zcml (for the collective.xdv package), but z3c.autoinclude itself is invoked from another ZCML file, so it must've been able to read that. Any tips on how to debug or similar experiences would be appreciated! Can you reproduce this, reliably? Yes. I'd be reaching for python/expat debug symbols and gdb, probably. I'm not sure how to do that. My gdb fu is *very* limited. I also assume I'd need to re-build python and/or expat for this? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Thu, Apr 1, 2010 at 2:17 PM, Tres Seavertsea...@palladion.com wrote: /me is deeply suspicious of *any* distro-provided python, ever. I'm also suspicious of 64-bit builds, given that I'm not using one on my dev machine. I've picked up the OpenSuSE installation ISOs; hopefully that's close enough to the SLES distributions that I'll be able to construct useful VMs. Martin: If I can't manage a useful 64-bit VM, I'll probably send you a script to try out. It'll likely be this weekend before I can really dive in. I really appreciate the help! As I said, I can't actually ssh into the machine from home, but once I'm back in the office on Tuesday (Monday evening your time, I guess) I'll be try out anything you suggest. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Hi Martin Given that its failing on a call to libc I would do a quick run with strace and ltrace. (In your case ltrace will be probably be more useful.) You will be able to see the calls into the libraries and you should see immediately if your segfault is inside expat or not. (ltrace output will be very long ;-) T On Fri, Apr 2, 2010 at 11:29 AM, Martin Aspeli optilude+li...@gmail.com wrote: Marius Gedminas wrote: On Thu, Apr 01, 2010 at 06:07:26PM +0800, Martin Aspeli wrote: I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration'sinclude / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. On recent Ubuntu versions segfaults get recorded in dmesg and end up in /var/log/kern.log like this: Mar 22 02:10:03 fridge kernel: [7804719.781375] python[10447]: segfault at 7fff84a7c000 ip 7fff83aae7cb sp 7fff84a7bd98 error 7 in libc-2.9.so[7fff83a2b000+168000] I assume that's a standard kernel feature now (we've 2.6.28, x86_64). Interesting. I'll check. The weird thing is that it's not parsing of any file: it happens during a standard configure.zcml (for the collective.xdv package), but z3c.autoinclude itself is invoked from another ZCML file, so it must've been able to read that. Any tips on how to debug or similar experiences would be appreciated! Can you reproduce this, reliably? Yes. I'd be reaching for python/expat debug symbols and gdb, probably. I'm not sure how to do that. My gdb fu is *very* limited. I also assume I'd need to re-build python and/or expat for this? Martin ___ Zope-Dev maillist - zope-...@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
