Re: [Zope-dev] [Vote] PEP308 voting began

2003-03-04 Thread Matthew T. Kromer
Guido van Rossum wrote:

IMO TALES should solve this for itself by introducing an if/then/else
expression form rather than depending on Python.  If you can have a
not:.. expression, surely you can have an if:..:then:..:else:..
expression.
--Guido van Rossum (home page: http://www.python.org/~guido/)
 

Yes, I'd be interested in seeing some kind of expression superset 
operator in TALES such that you could use some boolean logic in 
expressions which had subexpressions of different types (ie path 
expressions vs python expressions).  Currently the punt is to go out 
to Python for any logic other than the path expression OR syntax.

e.g.

   tal:define=variable tales: path: path_component | string: foo

An inline if/else might be C like

   tal:define=variable tales: local_var ? path: path_componenta : 
string: foo

where tales: is simply whatever the meta-expression handler name is.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Zope 2.6.1 beta 2 deferred until January

2002-12-24 Thread Matthew T. Kromer
With the holiday schedule and people being on vacation, we have decided 
to defer issuing a Zope 2.6.1 beta 2 release until after the first of 
January, to avoid the necessity of creating a beta 3 release (because of 
any potential bugs in beta 2 not found due to people being away.)

Have a Merry Christmas and a Happy New Year.  


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Infuriating ZClass registry Heisenbug

2002-12-12 Thread Matthew T. Kromer
If you feel like it, the change that is biting you is probably in here:

http://cvs.zope.org/Zope/lib/python/ZODB/ 
coptimizations.c.diff?r1=1.17r2=1.17.60.1only_with_tag=Zope-2_6-branch

I looked and there are some changes to the persistent_id code, but I  
lack sufficient background with this module to have something leap out  
at me and say aha!


On Wednesday, December 11, 2002, at 11:13 PM, Shane Hathaway wrote:

On Wed, 11 Dec 2002, Evan Simpson wrote:


I think I've fixed my ZGlobals, but there's definitely a problem in
coptimizations.  I made Connections.py use the Python persistent_id,
restarted twice, and ZGlobals was fine on the second restart.


Ah-ha, you now have a reproducible test case.  The coptimizations code  
has
been undergoing beautification lately; apparently this particular
ZClass-related functionality was not retained correctly.  This ought  
to be
a priority for Zope 2.6, I think.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )


Re: [Zope] Re: [Zope-dev] Zope Book call for assistance

2002-12-09 Thread Matthew T. Kromer
Jeffrey P Shell wrote:


But SQL Method DTML is very very very very nice.  It has a lot of type 
enforcement/safety measures (ie - autoquoting SQL Strings, ensuring 
that a 'sqlvar type=float' operation is inserting a float); a lot of 
*very* nice features for generating 'where' clauses (the sqltest 
'optional' flag and the smart 'dtml-and and dtml-or' tags that 
won't render if an optional 'sqltest' preceding them was not 
rendered); the 'sqltest multiple' feature is especially nice:

dtml-sqlgroup where
dtml-sqltest foo type=nb multiple optional
/dtml-sqlgroup

If foo is a blank string or empty list, that will render nothing.

If foo is a single string, that renders::

where foo = 'bar'

But if foo is a list of strings, that will render::

where foo in ('bar', 'baz')

Doing that programatically in Python is counterintuitive and awkward 
(just as it was before the specialized 'dtml-sql___' tags in DTML).  
For simple queries, doing it in the host programming language is not 
bad.  But for complex queries, it's very awkward to generate SQL.  
It's almost as bad as generating HTML inside of a programming language 
- it becomes difficult to maintain.


Yes, I think SQL methods are going to stick around.  The downside is 
there are some things that they SHOULD do that they dont, and that DTML 
doesnt (to my knowledge) facilitate.

For example, dtml-sqltest foo type=nb ought to be able to check with 
the underlying DA, and ask that DA to help it format its parameters. 
Currently the render() method used by DTML seems to be presumed to be a 
string, but what you want back is

   (foo=:foo, ('foo',foo))  # or whatever  all this foo foo sounds 
like a poodle

so that any bind variables can be handled more efficiently by the DA. 
Since each DA handles bind variables differently, it has to be involved 
in the process to return a string with substitution text, and the value 
to be substituted later.   The DA's query method currently takes only a 
string, but it should take a string and a concatenation of bind variables.

I remember looking at it and not wanting to get into trying to track 
down where DTML would have to change to allow nonlinear results.

There may well be something in DTML processing that would make this 
simple, I'm not very well versed on DTML processing internals.


--
Matt Kromer
Zope Corporation  http://www.zope.com/ 



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Unicode Diffs

2002-09-25 Thread Matthew T. Kromer

kent sin wrote:

I have a quick hack of Zope to use utf-8 as default
encoding.

This will allow all languages co-exist, making a
heaven for different language users. Also, this help
to easy Zope 2.6 for non latin users.

I did not test this very throughly, but it seems not
broken any thing yet. 

Please consider seriously make this into zope 2.6. 

I know this is very late, but the current 2.6 is very
bad to non-latin users.

Rgs,

Kent Sin

  


Please take a look at the proposals on dev.zope.org, and file this idea 
as a proposal.  In this way we can evaluate the solution with regards to 
the problem and the implementation.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] plone.org - chrism_installer_branch

2002-09-17 Thread Matthew T. Kromer

alan runyan wrote:

Hi.

just wanted to drop you guys a line and say that
chrism_installer_branch will save the world.
But I have some questions:

Chris said its based on HEAD.  Whats the 
difference between HEAD and Zope-2_6-branch?
I probably shouldnt be running HEAD in production?
I am having severe leakage here are some refoucnts:

DateTime - 62263
IOBucket - 32641
PloneFolder - 22766
User - 22196
PersistentMapping - 12846
CMFDefault.Document.Document - 12820

I have have the python security machinery turned on.

should I move to Zope2.6 branch instead of trying to
fight? what would I look at next?

~runyaga
 
  


Zope-2_6-branch broke off from the head about 10 days ago; so that the 
head is now somewhat 'undefined.'  While fixes will be applied to both 
2.6 and the head, 2.6 is the point where stability will be focused on.

What it looks like to me btw is that your Documents are holding on to 
the root of the ZODB perhaps -- do you do anything that would cause that 
to be stored?  


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Some Zope distributions are missing Data.fs.in

2002-09-12 Thread Matthew T. Kromer

Stefan H. Holek wrote:

Shane,

You are talking about the Zope core unit tests, aren't you? Can you be
equally sure you do not break third party tests people wrote for their own
products and packages? Please keep in mind that ZC is not the only one
using the Testing package to write their unit tests.

I am wondering why such a change would be necessary at all. If it is just
for the heck of it I politely ask to reconsider that decision. ;-)
  


It is our intent to remove Data.fs.in from distributions; any third 
party tests relying on this file existing should be altered to not rely 
upon it.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Some Zope distributions are missing Data.fs.in

2002-09-11 Thread Matthew T. Kromer

James Johnson wrote:

 Hi all,
I'm using win2k and zope2.5.1 and python 2.1.2.  And I don't have 
 the file Data.fs.in.  I have the others files.   Am I suppose to have 
 this file on win32.
 tia,
 - James


You don't need it; it merely provides a bootstrap of what Zope does on 
startup if there is no Data.fs anyway.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Zope Binary Release Changes for Zope 2.6

2002-08-20 Thread Matthew T. Kromer

Zope is currently released in three binary formats:

Windows - i386

 Solaris 2.6 - SPARC

 Linux RH 5.2 - i386

All binary releases come with python 2.1.3, built without large file 
support.

We are proposing to change the binary distributions to newer releases, 
at least on the Solaris and Linux platforms.  

The new Solaris binary would be Solaris 2.8.

The new Linux binary would be RedHat 7.2 based (i.e. glibc 2.1).

It is very likely that the supplied Pythons would be built with large 
file support *enabled* for these platform, unless some unforseen 
compatibility issue arose.

If there are requirements for keeping the binary distributions based on 
the older software, we need to understand these requirements.  Please 
follow up with this thread on the [EMAIL PROTECTED] list if you wish to 
participate.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Refresh trashes acquisition

2002-07-24 Thread Matthew T. Kromer

Ross Boylan wrote:

I find that when I refresh my product it destroys some of the
containment relationships.  Things start failing, and as far as I can
tell the only recovery is to completely rebuild the object.

This is a big problem, and if anyone could explain what is going on
and how to avoid it I would appreciate it.

Here is the result of aq_chain before the refresh:
[RankQuestion instance at 8dd5620,
OriginalSuborder instance at 8c9f8d8, EMailBallot instance at
8dfc870, __FactoryDispatcher__ instance at 8e73770,
ProductDispatcher instance at 83f0618, Folder instance at 8d5733

and after
[RankQuestion instance at 8dee2d0]
  

[...]

  


Hmm, while what you're referring to is refresh in the sense of a 
product reload, it's important to first state that *no* acquisition is 
expected to survive between transactions.

Each transaction must separately acquire its objects; if you try to pass 
wrapped objects between transactions or threads you're going to end up 
in trouble.

Secondly, when a product is refreshed, the extensionclass module will 
probably create new base classes for any ExtensionClass derived class 
file.  I dont *think* you can get much mileage out of knowing the id 
(address) of a class now, but be aware that a refresh will likely cause 
that address to change.  I have seen some bizarre errors where module 
constants changed because of a reload, and is tests broke because 
is tests the object address.

None of this is explicitly what you're asking about -- but I think what 
you're trying to do is in violation of the no wrapped objects should be 
stored past a transaction boundary rule.

However, I may be misreading your question too -- which is why I mention 
those two rules above so YOU can see if they make sense in your context.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ZCatalogs with SearchIndexes in ZODB - how to fix?

2002-07-24 Thread Matthew T. Kromer

Anthony Baxter wrote:

I've got a Data.fs which has been gradually moved from Zope version to
zope version (it's quite possible that the same Data.fs was originally a
Data.bbb!)

We've been getting the SearchIndex deprecation warning for some time, and
I'd like to fix it - as far as I can tell, it's caused by some ZCatalog 
objects in the ZODB that were created back when using SearchIndex was the 
hip'n'groovy way to impress the other catalog objects. 

Short of manually deleting and re-creating all the ZCatalog objects in the
ZODB, is there a way to get these puppies updated to get rid of the warnings?

Thanks,
Anthony
  


Well, the *sneaky* way to do it would be a funky one-time mod to the 
object's __setstate__ method -- but I caution against excessive 
sneakiness as potentially very frustrating to get right.

My inclination would be to write a one time search and replace method 
which crawled the ZODB looking for such objects and replaced them with 
newer versions.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Confusing segfault for Zope2 head on RH7.3

2002-07-23 Thread Matthew T. Kromer

Matthew T. Kromer wrote:


 On *my* RedHat 7.3 system, I cannot reproduce your problem.

 However, you do note that exception 0x6e727574 is ASCII for 'nrut' or 
 'turn' backwards.
 I dont know if that's meaningful or not.  I usally watch out for ASCII 
 in things that should otherwise be addresses.


FYI -- I fixed this in the ISO_8859_1 splitter.  The problem was a 
section of code attempting to use the character mu (a character with 
the high bit set) as an offset into an array.  It was cast to an int, 
rather than an unsigned char, so it went negative, and the code stored 
0x01 into an area it shouldn't have -- in this case, on top of the 
linkage table entry for PyExc_IndexError.


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Restricted Compiler issues on FreeBSD? (Core Dumps...waaa!)

2002-07-17 Thread Matthew T. Kromer

jeffrey wrote:

Jens Vagelpohl writes:

  

have you read about the python crash-bug stemming from tiny thread 
stack sizes on freebsd and applied the patch? the mailing list 
archives should tell you what to do.




Ahh, thanks!  I think I found the right patch and all.  Won't get to find
out if it all works until tomorrow.  (After the keynote of course ;)
Hopefully this nails the issue.  It's been beating me up for a while
now (primarily with older code, strangely enough).
  


Jeffrey,

Make sure you recompile your python scripts too (update to Zope 2.4.4 or 
Zope 2.5.1) as the older restricted python compiler generated bad 
constructs when dealing with large compound static constructs (ie lists 
of lists etc).  Pythons prior to 2.1.3 lacked checks to assert that the 
data structures were properly generated, and would go off the deep end 
once they hit them.  Python  2.1.3 will recognize bad structures and 
stop (albeit with messages that are cryptic to someone who isn't aware 
of the potential pitfalls).

I dont remember off the top of my head what the magic recompile all 
url is, but it's there, and it should show up in the debug log when you 
restart and it notices that the magic number on python scripts has changed.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Confusing segfault for Zope2 head on RH7.3

2002-07-17 Thread Matthew T. Kromer

Barry A. Warsaw wrote:

Working on the Zope 2.7 branch I noticed a very strange crash while
running the test suite.  I re-checked out the Zope 2 cvs head, rebuilt
everything from scratch and I still get the crash.  I'm about out of
ideas so I thought I'd post what I know here in case anyone else can
confirm or give clues.

[...]

The top of the backtrace is:

#0  PyErr_SetString (exception=0x6e727574, 
string=0x402660dc Splitter index out of range) at Python/errors.c:69

  



On *my* RedHat 7.3 system, I cannot reproduce your problem.

However, you do note that exception 0x6e727574 is ASCII for 'nrut' or 
'turn' backwards.
I dont know if that's meaningful or not.  I usally watch out for ASCII 
in things that should otherwise be addresses.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope/Python crashing

2002-05-20 Thread Matthew T. Kromer

Myroslav Opyr wrote:


 For FreeBSD there is another pthreads related bug in Python. And 
 quick-patch from 
 https://sourceforge.net/tracker/?func=detailaid=554841group_id=5470atid=305470 
 with THREAD_STACK_SIZE=0x2 solves the problem. But it is quick and 
 dirty hack. Will it be solved in next bugfix release of Python? 
 Whom/where I have to contact to have progress regarding the issue?

 I'll post comment into SF tracker because site which crashed 
 previously works fine now (for 5 days already)... Is that sufficient? 
 This dirty patch is not acceptable and clean solution have to take 
 place. Should I take part and try to develop one?


Well, the way you start threading with pthreads requres a single 
initializer for the stack size; once you have initialized the threading 
environment you cannot change the stacksize later, to the best of my 
knowledge.

I suppose you could finesse a new python startup argument, or 
environment variable check.

I think the SF tracker is the best way to approach that.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope/Python crashing

2002-05-17 Thread Matthew T. Kromer

Ahsan Imam wrote:

Hello,

I was going through the zope archive mail and saw message posted by
Matthew T. Kromer regarding crashing problems with Python 2.1.2. It
instructed to increase the 
#define PyTrash_UNWIND_LEVEL from 50 500. 

I just downloaded python 2.1.3. Do I still have to edit the object.h
file before I compile it.  

Thanks
  


No, Python 2.1.3 doesn't need to be patched like that -- that was just a 
temporary suggestion while we isolated where the problem was.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Can I get rid of standard_template.pt?

2002-05-13 Thread Matthew T. Kromer

Stefan H. Holek wrote:

 --On Freitag, 10. Mai 2002 08:27 -0700 Andy McKay [EMAIL PROTECTED] wrote:

 But... when you restart Zope it recreates them all again for you :)
 However you can delete the Examples and not have them come back ;)


 I think it was Matt who pointed out to me that I could create harmless
 substitute objects (say, Files) for the things I do not want autocreated.

 -) Delete session_data_manager
 -) Add File named session_data_manager
 -) Voila, core sessions disabled

 This will however not help you if your intention is to unclutter the 
 root folder...

 Regards,
 Stefan


My apologies for jumping in to the discussion a few days late, but I saw 
my name ;)

I have mixed feelings on some of the reserved names in Zope.

On the one hand, I certainly understand that people want to be able to 
customize *everything* and that means being able to delete things etc 
from the root folder.

On the other hand, we want to have a known 'stable base' which is pretty 
much guaranteed to exist for product writers, to facilitate 
interoperability.  

What we don't have is a good way of saying I *know* I can shoot myself, 
but I swear if I do I wont bleed on your carpet.  

So far, my preferences are for the latter; that is, I would prefer a 
well-defined area that has known contents.  Like them or not, that 
includes the standard_* pages as well as various other objects.  In the 
Zope 2 context, being able to know things exist by name without having 
to go through gyrations to confirm them is highly useful.  To my mind, 
this utility outweighs the clutter aspect.  Said differently, the 
small benefit of the common names multiplied by the broad general user 
base has a higher value of the large benefit (to the beholder) of the 
aesthetic root folder multiplied by a small user base.

And, as Andy has pointed out, you can easily go tweak the reserved names 
out of the source.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: [Zope Crash Solution] Positive Report

2002-03-28 Thread Matthew T. Kromer

Stefano Noferi wrote:

Hi Zopers!

Thanks all for your time, especially to Matthew and Leonardo.
My patched Python/Zope systems run well now in multi-threads mode.
I tested with very hard Python Scripts too and my report is at all positive.

My relief is great, like many others zopers, I think ; )

Thanks again and Happy Easter!

Stefano

FYI, Pythonlabs thinks there is more than one bug involved.  The 
relevant report at sourceforge is:


http://sourceforge.net/tracker/?func=detailatid=105470aid=535905group_id=5470


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Fwd: [Zope-dev] Re: more on the segfault saga]

2002-03-27 Thread Matthew T. Kromer

Martijn Jacobs wrote:

Hello Leo!

Tell us if your segfaults go away and whether Zope performance is
impacted by it.


The segfault is AWAY! I have an uptime of 2 hours now, it has never been
so long since it's in use! I don't notice any performance differences,
but I also have to say that I can't measure it really. The more
important is that it doesn't crash anymore!

Is this a *final* solution for the crashing? Or is this just a temporary
solution to make sure where the problem is?

BTW : Thank Matt for the patch!


regards,

Martijn.


It's going to be a temporary work around, but we're going to build Zope 
2.4.4 final and Zope 2.5.1 final around it; I'll be building beta 
releases this Friday with the patch included.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Crashing problem? Python workaround?!

2002-03-27 Thread Matthew T. Kromer

Hey fellow Zopistas:

If you're still experiencing crashing problems with Python 2.1.2, and 
Zope, please consider trying the following test:

 From a clean Python build tree, modify Include/object.h line 587.

#define PyTrash_UNWIND_LEVEL 50

make this read

#define PyTrash_UNWIND_LEVEL 500

Rebuild Python, and Zope, with the new patched Python.  Test it.

If this makes your problem go away, please post here to the list.

Technical details:  this patch prevents Python from storing objects in 
an internal trashcan of objects which need to be deleted, but aren't 
-- a way to avoid C stack recursion.  The trashcan code is not safe from 
the normal garbage collector; it steals a few fields in the object that 
the GC depends on (the reference count and the object type field, in 
fact) and so items in the trashcan will cause the garbage collector to 
crash with a segmentation fault.

Objects only go into the trashcan when a recursive delete is detected, 
with a depth defined by PyTrash_UNWIND_LEVEL.  Bumping this number up to 
a large number has the effect of disabling the trashcan module, at the 
expense of the C stack (because all of those recursive deletes CAN 
happen then.)

Pythonlabs found this today after much assistance from the folks on 
#zope on the irc.openprojects.net system; particularly, Leonardo Rochael 
Almeida's assistance has been extremely valuable in isolating this problem.

This is an intermediate workaround until Pythonlabs develops an official 
fix.  However, by using this workaround, you do not need the specially 
modified ZServerPublisher module I coded.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: more on the segfault saga

2002-03-19 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:


The official unofficial Zope place on irc is #zope at
irc.openprojects.net. Lots of cool and very knowledgeable people hang
out there.


OK, I put up a set of patches and a rather frazzled looking README for a 
profiler patch to Python at

http://www.zope.org/Members/matt

You want the C profiler patch; you have to build your OWN python 2.1.2 
and it will probably only work under Linux -- dont bother with Windows, 
parts of the code use mmap() for speed and Windows doesn't provide mmap.

There's a README document inside that has some rather vague and minimal 
installation instructions.  This is very definately use-at-your-own-risk 
stuff.  I'm posting notice here because others are interested in trying 
to help diagnose the Zope crashing problem so this serves as a reminder 
of where something is as it sits in your inbox waiting for bits to decay.

Here's the readme in its entirety:


To activate python tracing

Rebuild a clean python 2.1.2 with the two patches (included) applied.

Patch 1 is for the garbage collector module, it installs a segfault handler
which allows for an environment variable  CRASHCMD  to be present to
tell python what to do in the event of a segfault.

Patch 2 is a patch to ceval.c which builds in addtional tracing.

The cprof module must be built; a simple

make -f Makefile.pre.in PYTHON=/path/to/rebuilt/python2.1.2

will build the cprof module.


Once built, test the cprof module


/path/to/rebuild/python2.1.2

 import cprof
 cprof.activate()
 cprof.dump(filename)

and the filename specified should be created.  For the curious, the pb.py
program will play back the trace file to get data out of it.

 PATCHING ZOPE TO USE THIS 

Replace Zope's ZServer/PubCore/ZServerPublisher file with the included one.
Patch the line that contains the gdb command to point to your rebuilt 
python.

Copy the file gdb.cmd to where you start Zope.

Copy the file cprof.so to lib/python in your Zope directory

Start Zope.  Wait.  GDB will be invoked to gather crash data, save the
gdb output if possble (keep stdout from gdb).


Unfortunately, the README forgets to mention that you need to run Zope 
under the patched python.  Whoops.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: more on the segfault saga

2002-03-15 Thread Matthew T. Kromer

Hi Martijn,

We're basically just trying to construct traps to try to identify a 
smoking gun.  The upside is, if it works, we'll be able to fix the bug 
very quickly.  However, its based on assumptions about the exact nature 
of the bug -- so each trap I write essentially is making a hypothesis 
and then testing it.

Because Leo can get the crash very quickly, if you have a difficult time 
reproducing it, you don't need to spend a lot of effort trying to keep 
up with my traps.


On Friday, March 15, 2002, at 06:19 AM, Martijn Jacobs wrote:


 Hello Leo, Matt, Brian,

 I'm on it. Will send results when they're available. If anyone wants
 to talk to me during the period, I'll be on IRC.

 If you need any assistance for anything, I'm at your service
 Which channel/server are you on IRC?

 Did somebody  succeed reproducing the crash? We try the best we can to
 make a reproducable testcase, but Zope doesn't want to crash here... The
 clients who use the production Zope which crashes are all using Active
 Desktop (I know :( ), could that be of any matter?
 Technically it's insane if it does matter, but you never know...

 I'm out of capabilities right now, don't know what to do anymore, so I
 hope the bug will be found soon.


 regards,

 martijn










 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] more on the segfault saga

2002-03-14 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

On Wed, 2002-03-13 at 21:30, Matthew T. Kromer wrote:

On Wednesday, March 13, 2002, at 10:40 AM, Leonardo Rochael Almeida 
wrote:

What about patching Python to report the freed objects like you
mentioned on IRC? Also, how about turning on some flags in
gc.seg_debug()? Do you think we might be able to glance something by
seeing what objects where logged as freed or by storing them in
gc.garbage?


setting gc.set_debug(gc.DEBUG_LEAK) floods your stderr in a way you can
only believe by seeing it. And it didn't give me any clue. the last
object freed was an instance method. Most everything running inside Zope
is an instance method or another...


OK, I'm attaching a patch to Python's Modules/gcmodule.c which should 
set a trap for where the garbage collector trips over bad data; this 
will grab the bad data and send it to stderr so I can build a better trap.

This is ONLY step one in tracking this down.  You will have to rebuild 
Python to activate this patch; and all it basically is doing is setting 
a SIGSEGV handler; and setting up a small trace area for the GC to 
record data in to, so at the time the SIGSEGV comes in, it can print out 
what the last thing was the code was doing.

This is ONLY going to tell me that the GC tripped over something, but it 
WILL at least tell me what object it is scanning, that object's refcount 
(which I bet is zero, and forms the basis for a better trap) and the 
object's type and traverse pointers.

The traverse pointer should NOT be null.  If it is, then thats something 
wrong with gc being called for that type.

If you apply this patch, run Zope with a python with this patch applied 
with stderr saved to a file.  send me the file, and then you can revert 
to running zope w/o the patch.

When the patch triggers, it will exit Python immediately with exit code 
999 after it prints its information.


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




--- Modules/gcmodule.c.orig Thu Mar 14 10:35:21 2002
+++ Modules/gcmodule.c  Thu Mar 14 11:14:13 2002
 -22,6 +22,8 
 #include Python.h
 
 #ifdef WITH_CYCLE_GC
+#include signal.h
+#include stdarg.h
 
 /* magic gc_refs value */
 #define GC_MOVED -1
 -34,6 +36,7 
 static PyGC_Head generation2 = {generation2, generation2, 0};
 static int generation = 0; /* current generation being collected */
 
+
 /* collection frequencies, XXX tune these */
 static int enabled = 1; /* automatic collection enabled? */
 static int threshold0 = 700; /* net new containers before collection */
 -60,12 +63,82 
DEBUG_SAVEALL
 static int debug;
 
+
+static int CRASHTRAP = 0;
+static int CRASHFLAG = 0;
+static char *CRASHTYPE = NULL;
+static int CRASHLOG[16];
+
 /* list of uncollectable objects */
 static PyObject *garbage;
 
 /* Python string to use if unhandled exception occurs */
 static PyObject *gc_str;
 
+static void CRASH_trip(int i, siginfo_t *siginfo, void *p) {
+
+   int n;
+
+   fprintf(stderr,CRASH %d at %08x\n, (int) siginfo-si_signo,
+   (unsigned int) siginfo-si_addr);
+
+   if (CRASHFLAG == 0) {
+   fprintf(stderr,\tCrash handler not activated for this!\n);
+   } else {
+   fprintf(stderr,\tCrash type %s\n, CRASHTYPE ? CRASHTYPE : (none));
+   fprintf(stderr,\tCrash log: %d values: , CRASHLOG[0]);
+   for (n = 0; n  CRASHLOG[0]; n++) {
+   fprintf(stderr, %08x, (unsigned int) CRASHLOG[n+1]);
+   }
+   fprintf(stderr,\n);
+   }
+   exit(999);
+}
+
+static void CRASH_activate(void) {
+
+   struct sigaction sa;
+   struct sigaction oldsa;
+
+   sa.sa_sigaction = CRASH_trip;
+   sigemptyset(sa.sa_mask);
+   sa.sa_flags = SA_SIGINFO;
+
+   if (CRASHTRAP == 0) {
+   sigaction(SIGSEGV, sa, oldsa); 
+   CRASHTRAP = 1;
+   }
+
+   CRASHFLAG = 1;
+   CRASHTYPE = NULL;
+   CRASHLOG[0] = 0;
+
+}
+
+static void CRASH_deactivate(void) {
+   CRASHFLAG = 0;
+}
+
+static void CRASH_type(char *s) {
+   CRASHTYPE = s;
+}
+
+static void CRASH_record(int n, ...) {
+   va_list ap;
+   int i;
+
+   va_start(ap, n);
+
+   for (i = 0; i  n; i++) {
+   CRASHLOG[i+1] = va_arg(ap, int);
+   }
+
+   va_end(ap);
+
+   CRASHLOG[0] = n;
+}
+
+
 /*** list functions ***/
 
 static void
 -164,13 +237,29 
 subtract_refs(PyGC_Head *containers)
 {
traverseproc traverse;
+   PyObject *obj;
+
+
PyGC_Head *gc = containers-gc_next;
+
+   CRASH_activate();
+   CRASH_type(subtract_refs);
+
for (; gc != containers; gc=gc-gc_next) {
+   obj = (PyObject *)PyObject_FROM_GC(gc);
+   CRASH_record(4, obj,
+   obj != 0 ? obj-ob_refcnt : 0,
+   obj != NULL ? obj-ob_type : NULL,
+   obj != NULL  obj-ob_type != NULL ?
+   obj-ob_type

Re: [Zope-dev] more on the segfault saga

2002-03-14 Thread Matthew T. Kromer

Dieter Maurer wrote:

Leonardo Rochael Almeida writes:
  In any event, Martijn Jacobs (a.k.a. instability case #3 :-) sees his
  crashes in pure dtml methods, which could mean that PythonScripts are
  inocent in this case... or not, since the segfault hits inside the gc,
  which might be collecting something completely unrelated to the current
  requests.
Just a wild guess: is the GC guaranteed to be thread safe?


Dieter


I'm fairly sure it is; certainly, there's an activity flag which should 
prevent the collector from being reentered.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] more on the segfault saga

2002-03-14 Thread Matthew T. Kromer

Attached is another diagnostic patch which you might apply to Python. 
 If you apply this patch, you WILL need to rebuild Zope to include it.

What it will do is complain to stderr if an object is INCREF'd from 
refcount 0.  It also silences the complaint for the one area which I 
know revives dead objects.

This patch will probably cause a crash after an erroneous incref-from-0 
is detected, since it doesnt actually DO the incref in that case.

The intent is to find a case in the code where an object is held between 
threads; one thread decrefs to zero, the other thread increfs, causing a 
revive -- but too late to save the patient.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




--- Include/object.h.orig   Thu Mar 14 16:44:36 2002
+++ Include/object.hThu Mar 14 16:54:29 2002
 -442,7 +442,7 
 #define _Py_NewReference(op) ((op)-ob_refcnt = 1)
 #endif
 
-#define Py_INCREF(op) ((op)-ob_refcnt++)
+#define Py_INCREF(op) ((op)-ob_refcnt  0 ? (op)-ob_refcnt++ : 
+fprintf(stderr,Eeek! Increfing an object from refct 0 at %s:%d\n,__FILE__,__LINE__) 
+)
 #define Py_DECREF(op) \
if (--(op)-ob_refcnt != 0) \
; \
--- Objects/classobject.c.orig  Thu Mar 14 17:04:40 2002
+++ Objects/classobject.c   Thu Mar 14 17:01:36 2002
 -535,7 +535,8 
 #endif
 #else /* !Py_TRACE_REFS */
/* Py_INCREF boosts _Py_RefTotal if Py_REF_DEBUG is defined */
-   Py_INCREF(inst);
+   /* Py_INCREF(inst); */
+   inst-ob_refcnt++;  /* we dont want to trap this one */
 #endif /* !Py_TRACE_REFS */
 
/* Save the current exception, if any. */



Re: [Zope-dev] more on the segfault saga

2002-03-14 Thread Matthew T. Kromer

Matthew T. Kromer wrote:

 Attached is another diagnostic patch which you might apply to Python. 
 If you apply this patch, you WILL need to rebuild Zope to include it.

 What it will do is complain to stderr if an object is INCREF'd from 
 refcount 0.  It also silences the complaint for the one area which I 
 know revives dead objects.

 This patch will probably cause a crash after an erroneous 
 incref-from-0 is detected, since it doesnt actually DO the incref in 
 that case.

 The intent is to find a case in the code where an object is held 
 between threads; one thread decrefs to zero, the other thread increfs, 
 causing a revive -- but too late to save the patient.


extensionclass also brings back the dead; the following patch to Zope's 
extensionclass will turn off the warning when it happens when you apply 
the previous patch that I sent out that complains when an object is 
incref'd from a refcount of zero.


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




Index: lib/Components/ExtensionClass/src/ExtensionClass.c
===
RCS file: /cvs-repository/Zope/lib/Components/ExtensionClass/src/ExtensionClass.c,v
retrieving revision 1.46.36.1
diff -u -r1.46.36.1 ExtensionClass.c
--- lib/Components/ExtensionClass/src/ExtensionClass.c  4 Oct 2001 14:25:19 -  
 1.46.36.1
+++ lib/Components/ExtensionClass/src/ExtensionClass.c  14 Mar 2002 22:43:10 -
 -3047,8 +3047,9 
   fprintf(stderr,Deallocating a %s\n, self-ob_type-tp_name);
 #endif
 
+  self-ob_refcnt++;
   PyErr_Fetch(t,v,tb);
-  Py_INCREF(self); /* Give us a new lease on life */
+ /* Py_INCREF(self);   /* Give us a new lease on life */
 
   if (subclass_watcher 
  ! PyObject_CallMethod(subclass_watcher,destroying,O,self))



Re: [Zope-dev] more on the segfault saga

2002-03-13 Thread Matthew T. Kromer


On Tuesday, March 12, 2002, at 05:08 PM, Leonardo Rochael Almeida wrote:


 Matthew, thanks for taking the time to gdb the beast with me. Did you
 come up with any instrumentation I should add to Python or Zope to get
 what it is that Python is trying to resease twice? If you want, I can
 arrange a MySQL-less period during production so that we can capture the
 crash in a cleaner environment.



Sorry, I only dug out what I had and made THAT work; it was a 
single-thread profiler.  It will require some modding to turn it into a 
useful debug tool instead, and I've been busy with other things.


 Speaking of instrumentation, since the gremlin seems to be threading
 related (it stops with '-t 1'), it might be useful to serialize certain
 parts of the execution path with semaphors, like the path to the
 restoration or the execution of PythonScripts, the path to the execution
 of SQL queries, etc. this means running with small locks in certain
 sections instead of the big '-t 1' lock (which is not really a lock, but
 you get the picture :-).


Well, if you have the energy to try serializing some of of the base 
parts of the code, by all means, go ahead.   I can't even begin to guess 
where the problem is though.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.4.4b1 and Python 2.1.2 crash again

2002-03-12 Thread Matthew T. Kromer

what happens when you run single threaded?

what happens when you do

import.gc
gc.disable()

in z2.py?


On Tuesday, March 12, 2002, at 08:18 AM, Dario Lopez-Kästen wrote:

 hm... i am now suffering from core dumps every 2-30 minutes...

 anyone has gotten any further with tracking these down?

 /dario

 - 
 Dario Lopez-Kästen Systems Developer  Chalmers Univ. of Technology
 [EMAIL PROTECTED]  ICQ will yield no hitsIT Systems  Services



 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.4.4b1 and Python 2.1.2 crash again

2002-03-08 Thread Matthew T. Kromer

Ivo van der Wijk wrote:

On Tue, Mar 05, 2002 at 04:49:47PM +0100, Stefano Noferi wrote:

Hi,

With zope 2.4.4b1 and python 2.1.2 my system auto-restart (crash) running
python script again..
It's the Restricted Python problem resolved?


I also still experience instabilities with 2.4.4b1/2.1.2. No coredumps,
no traces in stupid.log or detail.log - I'm quite lost here..

Matt's crash howto doesn't help either.

   Ivo


For what its worth, I'll be on IRC today to assist with diagnosis if you 
have the opportunity to join irc.openprojects.net #zope.



-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] MonkeyPatching in the Core (was: Zope 2.6 planning)

2002-03-04 Thread Matthew T. Kromer

With respect to the overhead of inserting things into modules in the 
source code, Fred Drake (I think!) pointed out to me that constructs in 
the form

if __debug__: ...

are automatically removed by the parser when Python is run with the -O flag.

Note that I am *not* sure how Zope performs when run with Python -O. 
 However, assuming it does run OK, then it makes sense to put all sorts 
of conditional things into code which are disabled for performance when 
-O is passed on the command line to Python.

Also note that I guarantee Zope will not run with Python -OO, as this 
removes doc strings, which the Publisher relies upon.

At the time, I was pushing more for a slightly more complex form of run 
time diagnostic control without the overhead of continuous symbol 
lookup, e.g. ifcondition(bitmask): such that either all the bits in the 
bitmask were enabled in the current debugging context, or at least 
some of them were.

Then again, I effectively want exits in Python's ceval.c at various 
locations to be able to do magic without the burden of more 
interpretation.  For instance, there are actually some clever things you 
can do with the new C level profiler, but 'stealing' the profiler to do 
other magic is not really the right approach.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6 project updated

2002-03-04 Thread Matthew T. Kromer

Brian Lloyd wrote:



http://dev.zope.org/Wikis/DevSite/Projects/Zope2.6/ProposedFeatures

I tried to capture who volunteered for what, but please look 
this over and let me know if I have you volunteered for something 
that you didn't mean to volunteer for :) You can also fix it 
yourself if you like (its a wiki).


To stir things up some more: for my part, I don't like seeing we need 
feature X without the corresponding I'll volunteer to write feature 
X.  Items that dont get contributed probably are not going to survive. 
  It's probably also worthwhile to consider how do I make the least 
intrusive contribution as well, such that we don't end up with 
highly-conflicting contributions.   Products form fairly unintrusive 
contributions.  Changes to ZServer cause headaches.  :)  I *really like* 
bugfixes or feature fixes though.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: [Zope-Coders] Zope 2.6 planning - call for contributors!

2002-03-01 Thread Matthew T. Kromer

If I had to make a wild guess, I would say 2.6 might land in the 
May-June timeframe.

I would anticipate a drive for finalization in May, with the release 
cycle in June.

That's not official, just a good guess based on our historical release 
patterns.


On Friday, March 1, 2002, at 06:56 AM, Chris Withers wrote:

 Toby Dickenson wrote:

 The 2.6 timeframe should allow me to find some time to integrate my 
 Unicode
 support (in ZPublisher, ZServer, and DTML), if there is agreement that 
 this
 is the right thing to do.

 I'd +1 that in a big way :-)

 Especially combined with landing all Andreas' cool ZCatalog unicode 
 work...

 cheers,

 Chris

 ___
 Zope-Coders mailing list
 [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-coders


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Zope Hotfix 2002-03-01 (Ownership Roles Enforcement)

2002-03-01 Thread Matthew T. Kromer


This hotfix addresses an important security issue that may affect some 
users of Zope versions 2.2.0 through 2.5.x

The issue involves the checking of security for objects with proxy 
roles. The context of the owner user that created the object with proxy 
roles was not being taken into account when determining access to the 
object with proxy roles. This flaw could allow users defined in 
subfolders of a site with sufficient privileges to access objects at 
higher levels in the site that they would not normally be able to access.

We highly recommend that any Zope site running Zope 2.2.0 through Zope 
2.5.x have this hotfix product installed to mitigate the issue. Zope 
2.5.1 and 2.4.4 will contain a fix for the issue, at which time the 
hotfix can be removed.


  DOWNLOAD

Download this hotfix from


http://www.zope.org/Products/Zope/Hotfix_2002-03-01/Hotfix_2002-03-01.tgz

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] still segfaults, this time no ZMySQLDA

2002-02-28 Thread Matthew T. Kromer


On Wednesday, February 27, 2002, at 06:49 PM, Leonardo Rochael Almeida 
wrote:

 So, what should I try now?


Did -t 1 help?  If not, do you feel comfortable hooking up gdb  and 
getting a traceback?


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] still segfaults, this time no ZMySQLDA

2002-02-28 Thread Matthew T. Kromer


On Thursday, February 28, 2002, at 02:09 PM, Leonardo Rochael Almeida 
wrote:

 Funny enough, running with -t 1 solved it.

 /me starts looking sideways to ZMySQLDA again, since it's not the new
 beta version...


Good!

Based on the fact that I'm not being innundated with I'm still 
crashing! complaints, I'm going to *guess* that -t 1 isolated the bad 
module, which was probably being activated even though you didn't think 
it was.  Maybe something traversed to it, or activated it in some other 
way.

I don't think there is a thread-sensitive gremlin in Zope 2.5, but I'm 
asking all parties to keep an eye open for a vanilla 2.5 (or 2.4.4) 
crash that -t 1 solves.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] still segfaults, this time no ZMySQLDA

2002-02-27 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

On Tue, 2002-02-26 at 21:55, Matthew T. Kromer wrote:

in lib/python/Products/PythonScripts/PythonScripts.py, try changing 
Script_magic to be 4, and see if that helps.


Nope, time between restart after the change and 1st crash was aprox.
3m15s.


Rats!  I was hoping that would flush things out, but wasn't expecting it 
to work.

FYI, some of this is written up in

  http://www.zope.org/Members/matt/StabilityHOWTO


hmm, I didn't know that page. It mentions trying with the -t 1. In this
case, what is the real total number of threads? is it one for Medusa and
one for handling Zope requests or is it just one overall? In this case I
might be able to run it under Valgrind after all.

Should I bother trying a different ExtensionClass.h?

Cheers, Leo


With -t 1, you're still running with more than one real thread, but only 
1 worker thread is processing Zope requests.  Most of the time when 
there is a threading issue, the worker Zope threads are the ones that do 
the potentially thread-unsafe work, so running with -t 1 will prevent a 
thread switch from occurring while an exposure could take place.

You don't need to bother with the different ExtensionClass.h; the magic 
in my special version is NOT necessary to apply, and should not make 
your system any more stable.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] still segfaults, this time no ZMySQLDA

2002-02-26 Thread Matthew T. Kromer

in lib/python/Products/PythonScripts/PythonScripts.py, try changing 
Script_magic to be 4, and see if that helps.

FYI, some of this is written up in

  http://www.zope.org/Members/matt/StabilityHOWTO


On Tuesday, February 26, 2002, at 07:15 PM, Leonardo Rochael Almeida 
wrote:

 On Tue, 2002-02-26 at 20:11, Leonardo Rochael Almeida wrote:
 [...]
 The options I think I have left:
 * Try the new ExtensionClass.h

 Which one, by the way? the one in HEAD, the one in 2.5 branch or a
 different one altogether? Does it have any impact in the objects in the
 ZODB? Can I run a ZEO Client with it against a ZEO Server with a
 different ExtensionClass.h?

 * Run Zope under a memory analiser such as Valgrind (which might not be
 feasible if the runtimes are too slow):
 http://devel-home.kde.org/~sewardj/

 Scratch that. Valgrind doesn't do threads. Anyone knows any
 alternatives?

 --
 Ideas don't stay in some minds very long because they don't like
 solitary confinement.


 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: [Zope] IBM DB2/CLI Database Adapter

2002-02-15 Thread Matthew T. Kromer

[Removed extra Zope list CC]

Robin, if you haven't already been directed towards mxODBC, I suggest you
give that a try.


- Original Message -
From: Robin Smidsrod [EMAIL PROTECTED]
To: Zope User List [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Wednesday, February 13, 2002 10:31 AM
Subject: [Zope] IBM DB2/CLI Database Adapter


 Anybody know about the current status of this DA?

 I'm currently locked into using Perl ZDBI_DA to get access
 to a DB2 datasource. I've had some problems
 with connections being reset and the DBI_DA lacks a table browser.

 Any information would be appreciated.

 -- Robin


 ___
 Zope maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope-dev )



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope builds (new Intel C++ compiler)

2002-02-12 Thread Matthew T. Kromer


On Tuesday, February 12, 2002, at 07:17 AM, seb bacon wrote:

 However, I do build Python with gcc 3.1 (cvs) and Python 2.1.2 and with
 various compile options can get a 17% speedup.

 Could you share the compile options with us?

 seb



OK, but be warned -- gcc cvs isnt guaranteed stable, so often I have to 
hand tweak things (rebuilding certain modules with different options, 
etc.)

First pass through:  edit Makefile to include the following flags to OPT=
-fomit-frame-pointer -march='ppro' -fssa -fprofile-arcs

make

run the test suite; run pystone with the produced python

make clean

edit Makefile to change -fprofile-arcs to be -fbranch-predictions

make

make test

edit the Makefile to remove the fancy optimization stuff

make install

You may be able in certain circumstances to be able to use 
-fschedule-insns and -fschedule-insns2.

I dont think -fssa works but I include it anyway as long as the modules 
pass test.

Gcc 3.0X doesn't seem to be able to make a good build with these 
options; gcc 3.1 (from cvs) does, but YMMV -- don't blame me or ask me 
for help building python with fancy options, please.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] copy paste 'leakage'

2002-02-08 Thread Matthew T. Kromer

Martijn Faassen wrote:

Hey,

I'm running into a weird problem I'm not sure how to tackle. I've
noticed that under some circumstances it takes a long time to copy
and paste a ParsedXML object. This seems to happen in a clean Zope,
at least in the Zope root, though it doesn't seem to happen in folders.
I've also had it happen in Folderish objects in an app, though *sometimes*
the problem wouldn't occur. Quite peculiar.

I noticed that it wasn't only slow; it was generated huge amounts of
useless volume for the ZODB as seen in the database management screen;
objects a couple of K big generally cause a meg or two of leak when
the copy process is 'slow'.

I concluded the copy process is this slow because Zope is somehow
copying far more data than intended.

But I don't know what; I need to get a clue in order to tackle this problem,
and this is as far as I've gotten. I've speculated somekind of weird
acquisition wrapper issues may be involved (certainly possible with ParsedXML),
which cause other objects to be copied, or possibly the same XML nodes
to be copied, multiple times, even though I can't find them. But perhaps
someone else has run into this problem before and can give me some idea
on where to start looking.

Regards,

Martijn


Martijn, under Zope 2.5, there's a utilites/ZODBTools/fstest.py program 
which can be used to take a look at your Data.fs file -- if you run it 
 with -v -v (two verbose flags) it will print a line for each object in 
every transaction.

If you look at the tail end of the output, you should be seeing the 
transactions performed last.  You should be able to do some degree of 
differenentiation based on that to figure out the volume of objects 
being copied in the FileStorage.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Zope 2.5. cAccessControl.c tentative patch

2002-02-01 Thread Matthew T. Kromer

FYI, those people who had to set ZOPE_SECURITY_POLICY to PYTHON are 
advised to apply the following Patch for 2.5's cAccessControl.c module 
and see if it helps.

As far as I can see, it solves the Zope 2.5 problems reported by Martijn 
Faassen with ParsedXML.


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 



Index: cAccessControl.c
===
RCS file: /cvs-repository/Zope/lib/python/AccessControl/cAccessControl.c,v
retrieving revision 1.13
diff -u -r1.13 cAccessControl.c
--- cAccessControl.c11 Jan 2002 17:14:27 -  1.13
+++ cAccessControl.c1 Feb 2002 17:27:31 -
@@ -1703,6 +1703,7 @@
Py_DECREF(r);
r = self-__roles__;
if (r == NULL) goto err;
+   Py_INCREF(r);
}
 
/*|



Re: [Zope-dev] Running out of options

2002-01-31 Thread Matthew T. Kromer

Erik Stephens wrote:

Hello,

This is regarding the Zope restarting problem that's been talked about
recently.  We've tried TONS of different configurations:

  Python 2.1.x (with threads and with  without pymalloc)
  Zope 2.4.x (built wo_pcgi  binary installs)
  ZMySQLDA (with  without the Kromer patch)
  MySQLdb (built against the different flavors of Python)

Here's our latest attempt at a stable Zope:

  - Python 2.1.2 (with threads and with pymalloc)
  - Zope 2.4.4b1 (wo_pcgi build)
  - ZMySQLDA (latest CVS checkout with and without the Kromer patch
applied to db.py)
  - MySQLdb (latest CVS checkout build against Python 2.1.2)
  - TinyTable product (one of our patches applied)
  - A customizations product for removing specific restrictions on
through-the-web components.

Does anyone know for sure if this is a problem with ZMySQLDA or with
Zope?

We've been hearing talk that once 2.4.4 is released, all will be good
in the hood.  Are we missing the boat somewhere?  Can anyone confirm
that a similar setup fixed their Zope restart problems?

Are there any magic option values (num threads, disable watchdog) we
can be passing to z2.py that might help stop the bleeding?

If it is confirmed that 2.4.4 is still hosed, then what is the
recommended course of action?  Down-grade to Zope 2.3.3?  Cross our
fingers and wait?  Which is more stable right now, Zope 2.5.0 or Zope
2.4.4?

Thank you,
Erik


Try starting with -t 1 (single-threaded mode) and see if that helps.  It 
won't actually hurt your performance very badly unless you are compute 
or I/O intensive.  Andy Dustman has a better fix than mine in the works; 
he posted something about it on the main Zope list today.

BTW, you said in your message above you use --with-pymalloc.  That's 
almost for sure going to cause problems with extension classes.  Use 
--without-pymalloc so that the custom allocator gets left out.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] segfaults persist in 2.4.4b1

2002-01-31 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

Hi,

I previously had a segfaulting Zope 2.4.3 installation, whose segfaults
I traded for a leak when I disabled the cycle-gc.

I just installed Zope 2.4.4b1 (binary) hoping it'd fix the segfaults so
that Zope could go back to not leaking, but they persist. Disabling gc
in this new version stops the segfaults, but restarts the leak, just
like before. Basically, it feels like the problems are the same for me.

I'm now in the process of recompiling python 2.1.2 --without-pymalloc to
see if it solves the problem but I suspect it'll only make Zope crash
less instead of stop crashing (like it did last time).

If anyone has any tip (or an automatic stack dump generator for running
zope processes)


Well, I think I mentioned this before, but you do NOT want 
--with-pymalloc on in your Python build.  I'm just about 100% positive 
that ExtensionClass derived objects will not play well with pymalloc.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Urgent help required

2002-01-31 Thread Matthew T. Kromer

Andy McKay wrote:

It uses the configuration set on the ODBC DSN I think. At least that is what
appears to happen.


AFAIK ZODBCA is only single threaded though, ZmxODBCA should be a better bet.

There is a Zope-DB list BTW and Matt Mr Zope RDBMS guru Kromer hangs out there
a lot.
--
  Andy McKay


I *definately* recommend ZmxODBCA over what we have -- we have some SWIG 
generated code that's old and crufty -- it works, but it isn't being 
maintained.   I do not have the time to vest in creating a new ODBC 
adapter for Zope.


-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] segfaults persist in 2.4.4b1

2002-01-31 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

On Thu, 2002-01-31 at 15:11, Matthew T. Kromer wrote:

Well, I think I mentioned this before, but you do NOT want 
--with-pymalloc on in your Python build.  I'm just about 100% positive 
that ExtensionClass derived objects will not play well with pymalloc.


Indeed you have, I just mentioned it because I was mentioning my next
steps and I suspect *the Python that comes with binary Zope 2.4.4b1* is
compiled with it.

   []'s Leo


The linux and solaris binaries should not; the Windows binary ... might. 
 For the Windows binary, we use the regular PythonLabs distributed 
Python212.exe files.  And I dont actually know if it uses pymalloc or not.

-- 
Matt Kromer
Zope Corporation  http://www.zope.com/ 




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] OracleStorage Used in a Production Environment

2002-01-30 Thread Matthew T. Kromer

Eric Roby wrote:

Is anyone successfully using OracleStorage in a production system or
(better yet) as a storage server in a ZEO configuration ?  It doesn't
appear that any work has been done on it for nearly 18 months.  Will
OracleStorage work with the DCOracle2 product in support of 9i ?

Thanks in advance

Eric


__



Hi Eric,

There's a modified version of OracleStorage in DCOracle2 -- thats why 
you don't see any activity on the other module.

I've loaded an extremely large data set into it in in-house testing, and 
there are one or two clients out there using it, but I cannot identify 
them to you.

With a well-tuned Oracle, I find that Oracle Storage is about 5 times 
slower than File Storage -- which is actually fantastic, given that its 
a lot easier to append onto the end of a file than it it to update 
tables in a RDBMS, so that a factor of 5x is really quite good!  I 
actually think you can do better than 5x, but it requires having a very 
IO-balanced system -- the one I tested on was Oracle 9i using 3 drives 
to separate out data areas.

However, keep in mind that Oracle Storage still stores pickled data into 
Oracle -- it doesn't do any object-relational mapping for you or 
anything like that; so you can't do operations on Zope data easily from 
within Oracle (although it's notionally possible to integrate a 
de-pickler as a PL/SQL loadable module so that you could actually 
inspect a pickle from inside PL/SQL.)

I have used this in a ZEO envrionment; it works as you would expect (you 
pop out the ZEO server's FileStorage and replace it with an 
OracleStorage).  Because the ZEO clients will cache objects if they have 
the ZEO_CLIENT environment variable set, I normally focus on write speed 
as being the dominant storage limiting factor, not read speed.

And of course, as the obligatory plug: Zope Corporation is always 
willing to undertake specific support engagements should you have the 
requirement to have on-call support.  Our web page with an overview of 
this is http://www.zope.com/Services/SupportContracts should you have an 
interest.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] OracleStorage Used in a Production Environment

2002-01-30 Thread Matthew T. Kromer

Eric Roby wrote:

Mathew

Thanks for the quick response.  My previous Zope project worked out great.
I feel very comfortable with the speed and stability of ZODB.  However, I
was only managing 40,000+ objects.  My new project has the potential of
exceeding a million objects.  That plus the fact that they (the customer)
have already invested in 9i and an Oracle DB administrator.  So I am
compelled to explore this as an alternative to ZODB.

Are there any implications to using OracleStorage with regard to ZCatalog
and acessing other Oracle tables via Z SQL Methods ?

Again ... Thanks

Eric



Well, OracleStorage works just like a FileStorage, except that it saves 
pickled objects in Oracle rather than in a big disk file.  So, ZCatalog 
doesn't have any bearing on it -- most ZCatalog objects are actually 
quite small -- about 500 or so bytes.  You also can still continue to 
have Zope clients access Oracle directly, even though the same oracle 
instance is serving them via ZEO and OracleStorage.

Generally, the only advantage to OracleStorage is psychological; some 
people get a better feeling if their data is stored in Oracle.  Is it 
better?  That's a judgement call.  Oracle can do certain things for you 
that FileStorage can't (remote replication etc) but in practice people 
are looking for the answer to what's the best bucket for me to carry 
around data in?  Certainly you can make the case that FileStorage is 
simpler, and arguably easier to back up.  I would only advocate 
OracleStorage if FileStorage is inadequate.   40,000 objects is not a 
lot of objects for FileStorage.  A million objects?  Well, the risk 
factor is making sure your large file support works :)  A good backup 
utility for Oracle may be able to back up data incrementally -- most 
filesystem backup utilites back up whole files on update, rather than 
backing up just the changed portions of a file -- so a huge database 
might be easier to back up to tape with Oracle than with FileStorage.

The inappropriate thing to do is allow a system with 1,000,000 objects 
to touch  all of the objects in a single operation (like Find from the 
ZMI tab).

Also, keep in mind that this is still ZODB -- just a different storage. 
 Most of the time, ZODB and FileStorage are used interchangably, but 
they aren't identical.  ZODB is more of an API specification that a 
storage must meet, so FileStorage is a specific implementation (as is 
OracleStorage.)


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Error 11

2002-01-25 Thread Matthew T. Kromer

Ahsan Imam wrote:

Setting the variable did not help.
I downloaded and built Python 2.1.2 yesterday.

Products:
zope-tinytable-0.8.2-2
zope-zmysqlda-2.0.7-5

Chris McDonough [EMAIL PROTECTED] 01/24/02 12:22 PM 

Sigh.  Can you try to start Zope with the environment variable
ZOPE_SECURITY_POLICY=PYTHON set to see if the problem continues?

Are you using any database adapters or other C extensions?

Are you sure that Zope is using the Python 2.1.2 that you installed?


Try applying this patch to db.py in ZMySQLDA.

It makes it so MySQLDA database connections reopen themselves when the 
DA object migrates between threads.


--- ../ZMySQLDA-orig/db.py  Thu Sep  6 23:45:35 2001
+++ db.py   Tue Jan  8 14:24:52 2002
@@ -104,6 +104,7 @@
 import string, sys
 from string import strip, split, find, upper, rfind
 from time import time
+import thread
 
 hosed_connection = (
 CR.SERVER_GONE_ERROR,
@@ -180,11 +181,12 @@
 self.connection=connection
 self.kwargs = kwargs = self._parse_connection_string(connection)
 self.db=apply(self.Database_Connection, (), kwargs)
-   self.transactions = self.db.server_capabilities  CLIENT.TRANSACTIONS
+   self.transactions = self.db.server_capabilities  CLIENT.TRANSACTIONS
 if self._try_transactions == '-':
 self.transactions = 0
 elif not self.transactions and self._try_transactions == '+':
 raise NotSupportedError, transactions not supported by this server
+self._v_tid = thread.get_ident()
 
 def _parse_connection_string(self, connection):
 kwargs = {'conv': self.conv}
@@ -275,6 +277,10 @@
 desc=None
 result=()
 db=self.db
+# Because we might not be thread-safe across threads...
+if not self._v_tid == thread.get_ident():
+db=self.db=apply(self.Database_Connection, (), self.kwargs)
+self._v_tid = thread.get_ident()
 try:
 for qs in filter(None, map(strip,split(query_string, '\0'))):
 qtype = upper(split(qs, None, 1)[0])
@@ -297,7 +303,7 @@
 except OperationalError, m:
 if m[0] not in hosed_connection: raise
 # Hm. maybe the db is hosed.  Let's restart it.
-   db=self.db=apply(self.Database_Connection, (), self.kwargs)
+db=self.db=apply(self.Database_Connection, (), self.kwargs)
 return self.query(query_string, max_rows)
 
 if desc is None: return (),()



Re: [Zope-dev] Error 11

2002-01-24 Thread Matthew T. Kromer


On Thursday, January 24, 2002, at 12:57 PM, Ahsan Imam wrote:

 Hey Chris,

 1) Installed Python 2.1.2 (with threads and without pymalloc)
 2) Got the latest cvs built of zope 2.4.
 3) Then compiled zope wo pcgi.
 4) Even tried with the Zope 2.5 beta.

 Zope still keeps restarting every 10 to 15 minutes.


Can you tell me what products you have installed?  Any database adapters 
or other custom products?


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: Python 2.1.2 does not fix the signal 11 crashing problem !!!!!!!!!!!

2002-01-21 Thread Matthew T. Kromer

Harald Koschinski wrote:

 Anthony Baxter wrote:

Harald Koschinski wrote

Hallo,

after switching to python 2.1.2 compiled with GC 
 (intranet:/usr/local/src/Python-2.1.2 # ./configure --without-pymalloc)
the crashes are back again :-(((


Which version of Zope? 

 2.4.3

Does it have the RestrictedCompiler fix?

 Good question - next question please :-(((

 I am again a little bit confused - I thought that the problem is only 
 in the GC module of Python because zope is running without any crashes 
 when I disable GC.


As far as I am aware, Zope 2.4 includes it's own version of the
Compiler code, and the current release of 2.4 still has the stacksize
bug.

Anthony


 Matthew: Can you give the final answer ???



2.4.4 (which should be mostly the equivalent of the Zope-2_4-branch in 
our CVS) hasn't been extensively tested.  There are a LOT of things that 
I'm aware of that _can_ cause crashes, ie in MySQL when the database 
object switches threads, it can get unhappy (I think something in MySQL 
may have thread-local state).

Unfortunately, I can't speculate on exactly WHICH problem is causing the 
seg faults, because they don't occur in-house (except for the one 
compiler bug which we were able to reproduce.)

I am notionally toying with the idea of spending some evening hours 
attempting to come up with some kind of Linux-enabled crash capturing 
package, but this is not necessarily going to be trivial, and I dont 
know exactly how I want it to work.  (Ie the proper way is to be able to 
validate all pointers etc, from the stack frame trace on a sig 11 signal 
handler etc.. but that may require some assembly glue).



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: [Zope] Error on install

2002-01-18 Thread Matthew T. Kromer

Eron Lloyd wrote:

Chris (or anybody else), 

Have you tried to install Beta4 using the linux binary package yet? I'm 
trying to write a patch against it but the installation does in fact bomb out 
each time I try to run it, even freshly untarred. The source TGZ works fine 
and dandy, however. It seems compilezpy.py has been dramatically changed 
since Beta3, as if to try and surpress the stdout messages. I'm going to try 
and break it down and find out where the exceptions are flagging, and also 
refactor my patch so it doesn't depend on the changes being made to 
compilezpy.py. I don't like wrapping a gag around code if I don't have to, 
and I'd rather this stuff be written to an install log or something.

I-see-a-fishbowl-proposal-for-a-new-installer-ly yours,

Eron


Hi Eron,

One of the engineers snuck in a last minute change to fix the install 
process to not install if compilation of any module fails.  However, the 
test suite for Python itself has some modules that don't compile, and 
they're causing the newly fixed installer to complain.

In the file inst/compilezpy.py, you can delete the section that says 
(starting at about line 45):

if not success:


to the end of the file and rerun the install program.

This one was particularly aggravating because the beta 4 release is only 
supposed to include bugfixes, not 'features'.  Clearly, I distingush 
between a bug fix which fixes an existing problem causing Zope to not 
work vs a feature which causes an otherwise working Zope to cease 
functioning.  :)


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: no more SIG11 --without-pymallog --without-cycle-gc (was Re: [Zope-dev] Bravery Zope 2.5 Testers)

2002-01-16 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

On a related note, when is Zope 2.4.4 comming out?



Post Python 2.1.2 final.  :)  Probably very shortly therafter.  


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.5.0b4 problems w/ ZEO

2002-01-15 Thread Matthew T. Kromer

Andrew Sydelko wrote:

Trying to start a ZEO client using Zope 2.5.0b4 doesn't
work:

--
2002-01-15T20:13:20 INFO(0) client Trying to connect to server: ('192.168.1.52', 8900)
--
2002-01-15T20:13:20 INFO(0) ClientStorage Connected to storage
--
2002-01-15T20:13:20 ERROR(200) ZODB Couldn't load state for 
'\x00\x00\x00\x00\x00\x00\x00\x02'
Traceback (innermost last):
  File /data/www/Zope.8/src/Zope-2.5.0b4-src/lib/python/ZODB/Connection.py, line 471, 
in setstate
TypeError: ('__init__() takes exactly 3 arguments (1 given)', extension class 
Products.Sessions.SessionDataManager.SessionDataManagerTraverser at 88ba2f8, ())

--
2002-01-15T20:13:20 PANIC(300) z2 Startup exception
--
2002-01-15T20:13:20 ERROR(200) ZODB Couldn't load state for 
'\x00\x00\x00\x00\x00\x00\x00\x02'
Traceback (innermost last):
  File /data/www/Zope.8/src/Zope-2.5.0b4-src/lib/python/ZODB/Connection.py, line 471, 
in setstate 
TypeError: ('__init__() takes exactly 3 arguments (1 given)', extension class 
Products.Sessions.SessionDataManager.SessionDataManagerTraverser at 88ba2f8, ())

Traceback (innermost last):
  File /data/www/Zope.8/Zope/z2.py, line 495, in ?
  File string, line 1, in ?
  File /data/www/Zope.8/src/Zope-2.5.0b4-src/lib/python/Zope/__init__.py, line 71, in 
?  
  File /data/www/Zope.8/src/Zope-2.5.0b4-src/lib/python/OFS/Application.py, line 245, 
in initialize
  File /data/www/Zope.8/src/Zope-2.5.0b4-src/lib/python/ZODB/Connection.py, line 471, 
in setstate
TypeError: ('__init__() takes exactly 3 arguments (1 given)', extension class 
Products.Sessions.SessionDataManager.SessionDataManagerTraverser at 88ba2f8, ())

--
2002-01-15T20:13:21 PROBLEM(100) zdaemon zdaemon: Tue Jan 15 15:13:21 2002: The kid, 
12557, died on me.


Any ideas or suggestions?

--andy.


Hi Andy,

in this particular case, ZEO is a red herring...  In beta 4 (and 
curiously, not on the HEAD--I'm sure I'll hear a mea maxima culpa soon 
from someone) the base class for the SessionDataManagerTraverser was 
made to be a persistent object.  Since the base class changed, the ZODB 
is now having fits trying to reconcile the change in base classes.

Since the session data manager object gets created automatically by Zope 
when it doesn't exist at system startup, you can make a new one (which 
inherits from Persistent) by:

1) Edit .../Products/Sessions/SessionDataManager and change the 
SessionDataManagerTraverser to not be persistent by duplicating the 
class line, commenting out one, and removing the Persistent word from 
the other

2) Restart Zope

3) Delete the session data manager from the root object

4) Stop Zope

5) Undo the editing change you did in step 1

6) Restart Zope

The problem will now have been addressed in the ZODB.  HOWEVER, since 
you mention you are running ZEO, all clients will have to have the same 
code base to be able to function.  You won't be able to do this without 
upgrading all of your code at the same time.





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] sessionDataManager should create it's temp/xxxx

2002-01-09 Thread Matthew T. Kromer

Chris McDonough wrote:

 OIC.  Thanks!

 This patch probably makes sense for some class of usage.  I'm a little 
 skittish because I think doing magic (e.g. if I dont find a data 
 container where I said there was one, make one in  /temp_folder) 
 might tend to bite folks.  Also, doing writes on reads (explicitly 
 commiting a transaction in the middle of a request-bounded 
 transaction) is probably a bad idea because it might cause half of 
 something to be committed inadvertently.

 The right solution for this is to create some sort of analogue of 
 rc.d/init.d for Zope where you can hook Zope startup and thus 
 repopulate the temp_folder with the separate data containers then.  I 
 thought of this a while back but lacked the time to implement it.

 - C


Actually, we did have the Temporary Folder have the capability to 
automatically import its contents at system startup, but that got tossed 
out as being function inappropriate to a temporary storage container by 
executive fiat.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ghosts of removed products haunting ZSQL Methods?

2002-01-08 Thread Matthew T. Kromer

John Ziniti wrote:

 I used to have the Product TrackerBase installed
 and have removed it.  Now, however, attempts to
 access the Advanced tab of ZSQL Methods is
 broken because a chain of events attempt to get
 info about a the non-existent Product:

 lib/python/Shared/DC/ZRDB/dtml/advanced.dtml calls
 manage_product_zclass_info

 ../ZRDB/DA.py.manage_product_zclass_info (line 546)
 calls self.aq_acquire('_getProductRegistryData')('zclasses')

 One of these items is the dict:
   {'id': 'Tracker', 'meta_type': 'Tracker',
 'product': 'Tracker', 'meta_class': ZClass instance at d55af0}

 Then, line 548 calls if hasattr(z._zclass_,'_p_deactivate').

 The attempt to access z._zclass_ fails because this
 product doesn't exist.

 1) the call to z._zclass_ doesn't actuallt do anything
except: ## Eek, persistent

 2) Is this schizophrenic state my fault?

 3) How to rectify the problem (Make *all* of Zope
know that TrackerBase is gone)?


Hi John,

I'm ready and eager to blame Z classes for everything :)

In this case, lets try to force ZClasses to fix themselves up by going 
into Zope at the python command line prompt

% cd lib/python
% python2.1.1
  import Zope
  app = Zope.app()
  app.fixupZClassDependencies(rebuild=1)
  get_transaction().commit()

Hopefully that will do it for you.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] FreeBSD, Zope, and Python 2.1

2002-01-04 Thread Matthew T. Kromer

For what it's worth:

Today I helped Jens track down a problem under FreeBSD where Zope was 
crashing rendering a page which rendered successfully on other systems. 
 It turns out that the default stack size for a thread under FreeBSD is 
64K, and that's not enough stack space to render this particular page of 
his (a Tracker page) -- most likely due to changes with how the 
RestrictedPython compiler is used/invoked by DTML processing, since this 
represents a big change from Zope 2.3.

The solution we used was modifying Python/thread_pthread.h and the 
PyThread_start_new_thread function; specifically, instead of using a 
(pthread_attr_t *) NULL, we made a new attribute instead:

pthread_attr_t pta;

pthread_attr_init(pta);

pthread_attr_setstacksize(pta, (117));

and using pta in the thread creation function rather than NULL.

Recompiling Python 2.1 with changes gives it a 128K stack per thread, 
which seemed sufficient to render the Tracker page in question.

Does anyone know of a better way to tune thread stack sizes on FreeBSD?


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Zope 2.4 crashes -- possible fix identified, other solutions also suggested

2001-12-20 Thread Matthew T. Kromer

Well, if you want to grab what is probably going to turn in to Python 
2.1.2 from CVS, you can get the release21-maint branch from 
:pserver:[EMAIL PROTECTED]:/cvsroot/python package 
python.

This already has the important change to ceval.c in it; but I'm not sure 
that the rest of the changes for the 2.1.2 release have been finalized.

I think Anthony Baxter is going to try to put a beta / release candidate 
of 2.1.2 up real soon now, but I can't speak for him :)


On Wednesday, December 19, 2001, at 10:39 PM, Joseph Wayne Norton wrote:


 Matt -

 If possible, I would prefer to use a source Python 2.1.2 release with
 a source zope 2.4.4 ? bugfix release (or create my own from the 2.4
 cvs branch) once the fixes are complete.  We do not want to put a 2.5
 release in production at this time.

 Thanks for the update.

 regards,

 - j

 At Wed, 19 Dec 2001 09:25:08 -0500,
 Matthew T. Kromer wrote:

 The ExtensionClass.h patch has NOT been merged into the branches and 
 trunk
 yet pending further review.


 just my 2 cents, but we have been using this in production for about 1
 week or more without any troubles.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] disabling gc does not necessarily hide memorycorruption

2001-12-20 Thread Matthew T. Kromer

Well the ExtensionClass.h fix I think is completely harmless (assuming 
you recompile every ExtensionClass based C module) but I'm not convinced 
it's necessary, either.  It's more of an insurance policy -- if every C 
module plays by the rules and checks the type flags field of the type 
object first, then the ExtensionClass.h file fix isn't necessary.  Of 
course, anytime someone says well if every... I get suspicious.

Those people who have reported that it helps their system stability are 
probably benefiting because stack stomps (from the two known 
stack-clobbering bugs in Python 2.1) end up leaving pointers to 
extensionclass objects in places they don't belong (ie places which have 
already CHECKED the flags, and thus don't check them again).  Those C 
modules then use a NULL pointer check, which properly halts chasing of 
chains with the ExtensionClass.h patch, where it would have led to 
another wild pointer without.  Its the same reason disabling GC also 
helps some people; it reduces the activities that follow bad pointers.

Actually, I was kind of hoping Martijn Faassen would pipe up and say I 
applied the restricted python patches you've already put up on the 
Zope-2_4-branch, and my problems with ParsedXML went away! since he's 
one of the folks that did NOT benefit from applying the ExtensionClass.h 
patch.

On Wednesday, December 19, 2001, at 03:44 PM, Chris McDonough wrote:

 I'm not sure the ExtensionClass fix is baked yet, and there's not yet an
 overwhelming consensus that it helps stability, not to mention that I'm 
 not
 sure how to install it, so no. ;-)

 I think you can use the trunk, no branch required.  Hopefully the fixes 
 will
 be merged soon.

 - Original Message -
 From: Leonardo Rochael Almeida [EMAIL PROTECTED]
 To: Zope Developers list [EMAIL PROTECTED]
 Sent: Wednesday, December 19, 2001 3:22 PM
 Subject: Re: [Zope-dev] disabling gc does not necessarily hide
 memorycorruption


 Oh, and which branch should I get the tarball from?

 On Wed, 2001-12-19 at 18:20, Leonardo Rochael Almeida wrote:


 On Wed, 2001-12-19 at 18:02, Martijn Pieters wrote:
 [...]
 You can download files from CVS as a tarball. Just go to
 http://cvs.zope.org/Zope/lib/python/RestrictedPython and use the link
 at the
 bottom.

 Should I apply the ExtensionClass.h fix from CVS mentioned here in the
 list too?

 --
 Ideas don't stay in some minds very long because they don't like
 solitary confinement.

 --
 Ideas don't stay in some minds very long because they don't like
 solitary confinement.


 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )



 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Zope 2.4 crashes -- possible fix identified, other solutions also suggested

2001-12-18 Thread Matthew T. Kromer

Jeremy Hylton wrote:

MTK == Matthew T Kromer [EMAIL PROTECTED] writes:


  MTK A side effect of shutting off the garbage collector is that you
  MTK can have some storage leaks.  We're working on being able to
  MTK re-enable the garbage collector so that you don't exhaust
  MTK memory over time.

Do you have any more idea about what shutting the garbage collector
off achieves?  In practice, the garbage collector's most common effect
is to turn latent bugs into manifest bugs; a bug has trashed part of
memory and the garbage collector just happens to find it first.  If
you turn GC off in these cases, you run a little longer, but you're
running with corrupted memory.

Jeremy



Well, one suspicion I have is that (aside from memory corruption caused 
by the compiler sack size bugs and the frame setup bug in 2.1 when 
handling exceptions) ExtensionClasses are providing bogus data to 
modules which aren't checking the flags to see if the GC fields are 
populated.

Some of the people who have tried the modified extensionclass.h which 
pads out the type object to align it with the 2.1 type object THINK they 
have seen a reduction in crashes, but these same folks also have not 
applied fixes for the two known bugs.

Soo... if shutting off GC extends time between crashes for some folks 
from every 15 minutes to 3 times a day, my advise is to shut off GC.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Zope 2.4 crashes -- possible fix identified, other solutions also suggested

2001-12-18 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

On Tue, 2001-12-18 at 13:44, Matthew T. Kromer wrote:

Soo... if shutting off GC extends time between crashes for some folks 
from every 15 minutes to 3 times a day, my advise is to shut off GC.


Now I can really confirm that gc.disable() is enough to avoid the
crashes (no need to recompile python --without-gc).

And as far as we could notice, it didn't start leaking like crazy. We
haven't been able to identify any noticeable increase in memory
consumption. If it is leaking we will need some more time to notice it.

Cheers, Leo


Keep in mind that the leaks you may experience are directly related to 
what code you run, and whether or not they introduce cycles.   Some of 
the restricted python compiler code did/does create cycles under the 
assumption that the GC would break them.  But if you dont lean on code 
that uses RestrictedPython too much, you can live with slow leaks elsewhere.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Bravery Zope 2.5 Testers

2001-12-14 Thread Matthew T. Kromer

Harald Koschinski wrote:

 Matthew T. Kromer wrote:

 OK, I am formulating a hypothesis that if you replace 
 ExtensionClass.h in Zope 2.4 and Zope 2.5 with the following one:

 
http://cvs.zope.org/~checkout~/Zope/lib/Components/ExtensionClass/src/ExtensionClass.h?rev=1.16.140.2content-type=text/plain
 


 (Which goes in lib/Components/ExtensionClass/src/ExtensionClass.h btw)

 and rebuild zope  python2.1 wo_pcgi.py

 and rebuld any other modules you may have which depend on extension 
 class

 that some of the segfaults could go away.

 I have one person testing this right now, and am looking for others 
 to prove it. 


 My System is very oft crashing - here is the log

 
---
 

 2001-12-03T13:54:09 ERROR(200) zdaemon zdaemon: Mon Dec  3 14:54:09 
 2001: Aiieee! 24162 exited with error code: 11
 2001-12-04T06:49:04 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:49:04 
 2001: Aiieee! 21743 exited with error code: 11
 2001-12-04T06:50:52 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:50:52 
 2001: Aiieee! 27028 exited with error code: 11
 2001-12-04T06:51:15 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:51:15 
 2001: Aiieee! 27083 exited with error code: 11
 2001-12-04T06:51:45 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:51:45 
 2001: Aiieee! 27118 exited with error code: 11
 2001-12-04T06:53:36 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:53:36 
 2001: Aiieee! 27154 exited with error code: 11
 2001-12-04T06:54:23 ERROR(200) zdaemon zdaemon: Tue Dec  4 07:54:23 
 2001: Aiieee! 27228 exited with error code: 11
 2001-12-04T07:18:04 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:18:04 
 2001: Aiieee! 27260 exited with error code: 11
 2001-12-04T07:38:51 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:38:51 
 2001: Aiieee! 27775 exited with error code: 11
 2001-12-04T07:42:09 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:42:09 
 2001: Aiieee! 28198 exited with error code: 11
 2001-12-04T07:44:59 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:44:59 
 2001: Aiieee! 28311 exited with error code: 11
 2001-12-04T07:45:50 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:45:50 
 2001: Aiieee! 28421 exited with error code: 11
 2001-12-04T07:51:43 ERROR(200) zdaemon zdaemon: Tue Dec  4 08:51:43 
 2001: Aiieee! 28461 exited with error code: 11
 2001-12-04T13:52:36 ERROR(200) zdaemon zdaemon: Tue Dec  4 14:52:36 
 2001: Aiieee! 2215 exited with error code: 11
 2001-12-04T14:07:49 ERROR(200) zdaemon zdaemon: Tue Dec  4 15:07:49 
 2001: Aiieee! 15784 exited with error code: 11
 2001-12-05T07:31:39 ERROR(200) zdaemon zdaemon: Wed Dec  5 08:31:39 
 2001: Aiieee! 20380 exited with error code: 11
 2001-12-05T07:48:50 ERROR(200) zdaemon zdaemon: Wed Dec  5 08:48:50 
 2001: Aiieee! 21874 exited with error code: 11
 2001-12-05T07:56:16 ERROR(200) zdaemon zdaemon: Wed Dec  5 08:56:16 
 2001: Aiieee! 22040 exited with error code: 11
 2001-12-05T08:02:15 ERROR(200) zdaemon zdaemon: Wed Dec  5 09:02:15 
 2001: Aiieee! 22131 exited with error code: 11
 2001-12-05T08:03:26 ERROR(200) zdaemon zdaemon: Wed Dec  5 09:03:26 
 2001: Aiieee! 22240 exited with error code: 11
 2001-12-05T08:10:29 ERROR(200) zdaemon zdaemon: Wed Dec  5 09:10:29 
 2001: Aiieee! 22281 exited with error code: 11
 2001-12-05T08:11:06 ERROR(200) zdaemon zdaemon: Wed Dec  5 09:11:06 
 2001: Aiieee! 22417 exited with error code: 11
 2001-12-05T08:41:42 ERROR(200) zdaemon zdaemon: Wed Dec  5 09:41:42 
 2001: Aiieee! 22445 exited with error code: 11
 2001-12-05T09:24:19 ERROR(200) zdaemon zdaemon: Wed Dec  5 10:24:19 
 2001: Aiieee! 22886 exited with error code: 11
 2001-12-05T10:18:56 ERROR(200) zdaemon zdaemon: Wed Dec  5 11:18:56 
 2001: Aiieee! 23468 exited with error code: 11
 2001-12-05T11:02:32 ERROR(200) zdaemon zdaemon: Wed Dec  5 12:02:32 
 2001: Aiieee! 24061 exited with error code: 11
 2001-12-05T11:45:35 ERROR(200) zdaemon zdaemon: Wed Dec  5 12:45:35 
 2001: Aiieee! 24940 exited with error code: 11
 2001-12-05T12:48:49 ERROR(200) zdaemon zdaemon: Wed Dec  5 13:48:49 
 2001: Aiieee! 25060 exited with error code: 11
 2001-12-05T13:16:23 ERROR(200) zdaemon zdaemon: Wed Dec  5 14:16:23 
 2001: Aiieee! 25837 exited with error code: 11
 2001-12-05T13:24:00 ERROR(200) zdaemon zdaemon: Wed Dec  5 14:24:00 
 2001: Aiieee! 28674 exited with error code: 11
 2001-12-05T13:31:58 ERROR(200) zdaemon zdaemon: Wed Dec  5 14:31:58 
 2001: Aiieee! 28822 exited with error code: 11
 2001-12-05T13:40:20 ERROR(200) zdaemon zdaemon: Wed Dec  5 14:40:20 
 2001: Aiieee! 29166 exited with error code: 11
 2001-12-05T14:48:50 ERROR(200) zdaemon zdaemon: Wed Dec  5 15:48:50 
 2001: Aiieee! 30233 exited with error code: 11
 2001-12-06T07:48:48 ERROR(200) zdaemon zdaemon: Thu Dec  6 08:48:48 
 2001: Aiieee! 316 exited with error code: 11
 2001-12-06T07:52:41 ERROR(200) zdaemon zdaemon: Thu Dec  6 08:52:41 
 2001: Aiieee! 552 exited with error code: 11
 2001-12-06T08:02

[Zope-dev] Zope 2.4 crashes -- possible fix identified, other solutions also suggested

2001-12-14 Thread Matthew T. Kromer

We've found a problem in the restricted python compiler on Python 2.1 
which can significantly undercalculate the stack size of a dynamically 
generated python code object (such as those used for DTML and Python 
scripts in Zope).

What that means is that when such an object overflows its stack, it 
begins clobbering things in memory.  As the amount of entropy in the 
system increases, it heads for a crash.

We've got a patch into the Zope 2.4 branch and 2.5 branch which corrects 
a large number of misestimations (although we believe there are a few 
cases yet unhandled) and has resulted in a fix for our test case 
reproducing the problem.

You can download an on-the-fly tarball from our cvs.zope.org server, 
the URL for this is

http://cvs.zope.org/Zope/Zope.tar.gz?tarball=1only_with_tag=Zope-2_4-branch

We intend on integrating a few further refinements early next week and 
putting out an official release at that time; however, applying the new 
code earlier may significantly reduce your exposure to the bug.

Also, we believe that anyone using Python 2.1 with garbage collection 
turned on should recompile Python to turn off garbage collection, using 
the flags --without-pymalloc and --without-cycle-gc to avoid additional 
exposures caused by these options.

We are evaluating a modification at this time to z2.py which would also 
disable garbage collection if it is enabled; users who cannot rebuild 
Python could modify z2.py to include the additonal lines

import gc
gc.disable()

to disable garbage collection at runtime.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris

2001-12-12 Thread Matthew T. Kromer

Joseph Wayne Norton wrote:

Matt -

Well, your patch seems fine in our testing environment.
Unfortunately, we do not see any restarts in the testing environment
... always in production.  I had to rebuild our entire software base
because we are using other products that use extensions class and they
are not included under the main zope installation.  It caused a bus
error the first time (with only running wo_pcgi.py).

As I mentioned in my prior e-mail, I modified the patch slightly to
exactly match the struct in Python's object.h.  Please review this
patch.  I will apply the patch in production tomorrow morning, 12/13,
(Japan Standard Time or GMT+9) and monitor the system.  If zope does
not restart during the day, then I think you have fixed the problem.

I'm using Zope 2.4.3 and Python 2.1.1 with pymalloc disabled on the
solaris platform.

thanks and regards,

- joe n.

p.s. I looked **briefly** at the Zope 2.5 source and this patch will
not be compatible since there doesn't seem to be a standard among the
different extension classes on whether to include or not include the
COUNT_ALLOCS define.  The cAccessControl class seems to be the
exception.


My fingers and toes are crossed for you ;)

I've actually built 2.5 with the modified extensionclass.h and it seems 
to build OK and it runs and passes all of its unit tests. Thats not 
proof one way or another, but...

Sorry our turnaround times are so laggy; thats the downside of 
diagnosing a problem on the other side of the globe.






___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] DA connection pooling thoughts

2001-12-12 Thread Matthew T. Kromer

Brad Clements wrote:

Sorry if this is FAQ..

I've been reading the petrilli's DA how-to, and thinking about Thunked_TM.

For Interbase, I'm thinking of trying a DB connection pool that threads can 
checkout.

That is, each DB connection is serialized to be used by one thread only, but there 
can 
be multiple DB connections in a Zope process. We'd maintain a pool of connections 
from 1 to n.. and each thread making a DA request checks out a DB connection from 
the pool (if available) and uses it..

I'm trying to figure out of Zope already does this, that is, has one DB connection 
per 
thread if I took out Thunked_TM from the DB itself.


Sortof.  Basically, yes.

Generally, the DA is going to get a connection object when it is opened. 
 Since each working thread in Zope will get its own PRIVATE copy of the 
DA object, each working thread will have its own connection to the 
database.  It is theoretically possible to share DA connections between 
threads (on purpose, or by accident) but you have to work at it and 
write some Python glue to make it happen.

However, if the underlying adapter is not reentrant, removing Thunked_TM 
will cause the adapter to be reentered for sure, which will probably 
crash the adapter.


Or is it that there are multiple DA's, and the DA uses on internal DB for all the 
DA's ?

If Zope will re-entrantly call a DA and it's corresponding DB without Thunked_TM in-
place, then having a connection pool (up to one for each thread) might be 
worthwhile.. I 
remember seeing something on the list about per-thread state information being 
available somewhere.. an oid jar or something?

Anyone have pointers?

Thanks


Most of the time when Thunked_TM is used, the underlying adapter can 
only have one operation in flight at any given time; ie it is not thread 
safe.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris

2001-12-12 Thread Matthew T. Kromer

Florent Guillaume wrote:

(gdb) print *((PyObject *) gc)-ob_type
$1 = {ob_refcnt = 18213696, ob_type = 0x2d70b0, ob_size = 0, 
  tp_name = 0x1 T, tp_basicsize = 1328272, tp_itemsize = 4156348, 
  tp_dealloc = 0x125865c, tp_print = 0x3c1b04, tp_getattr = 0,
tp_setattr = 0, 
  tp_compare = 0x29, tp_repr = 0x3adeb0, tp_as_number = 0xf66198, 
  tp_as_sequence = 0xdf3fa0, tp_as_mapping = 0x0, tp_hash = 0x1, 
  tp_call = 0x144490 PyMethod_Type, tp_str = 0x3f0a1c, 
  tp_getattro = 0x125865c, tp_setattro = 0x3c1b04, tp_as_buffer = 0x0,

  tp_flags = 158561192, tp_doc = 0x29 , tp_traverse = 0x4c4f4144, 
  tp_clear = 0xd908c0, tp_richcompare = 0x1151300, tp_weaklistoffset =
0}

[...]

gdb) x 0x4c4f4144
0x4c4f4144: Cannot access memory at address 0x4c4f4144.



0x4c4f4144 is big-endian ascii for LOAD. Things were corrupted
before...


Florent


Yes, the whole block is bad, so it probably isn't really a Python type 
object.  The refcount is a bit high, the name is really low (0x01!) the 
basicsize and itemsize are extremely large, the compare function is too 
low, the hash function is too low -- ie it isn't a type object.  

So, I may have been telling him to get the wrong thing; the source code 
that he faulted in reads:

/* Subtract internal references from gc_refs */
static void
subtract_refs(PyGC_Head *containers)
{
traverseproc traverse;
PyGC_Head *gc = containers-gc_next;
for (; gc != containers; gc=gc-gc_next) {
/* The next line is the line that was active at the time of his fault */
traverse = PyObject_FROM_GC(gc)-ob_type-tp_traverse;
(void) traverse(PyObject_FROM_GC(gc),
   (visitproc)visit_decref,
   NULL);
}
}

And PyObject_FROM_GC(gc) is either (gc) or ((PyObject *)(((PyGC_Head 
*)gc)+1)) depending on on whether or not WITH_CYCLE_GC is defined.  I 
took the easy route and asked Joe to assume that the former was true.
If the latter is true, then the type object is shifted upwards in memory 
by three words; the new first three fields are gc_next, gc_prev, and 
gc_refs.

That means every value in the type header is off by three fields, if it 
isn't aligned, meaning the real type object would be:

gc_next = 0x115eb40
gc_prev = 0x2d70b0
gc_refs = 0
ob_refcnt = 0x1
ob_type = 0x144490 (which we actually know is PyMethod_Type -- yay)
ob_size = 0x3f6bbc (which is too large for my comfort)
tp_name = 0x12865c (valid pointer but we dont know what it is)
tp_basicsize=0x3c1b04 (seems high again, but is 0x350b8 less than ob_size)
tp_itemsize = 0
tp_dealloc = 0
tp_print = 0x29 (boo!)
tp_getattr = 0x3adeb0
tp_setattr = 0xf66198
tp_compare = 0xdf3fa0
tp_repr = 0
tp_as_number = 1 (boo!)
tp_as_sequence = 0x144490 PyMethod_Type (boo!)

etc...

even shifting THESE values by 1 (assuming the compiler takes PyGC_Head 
which is three words and pads it up to 4 words for alignment) puts 
garbage values like 0x29 in tp_dealloc.

Ergo, I'm pretty confident that the gc pointer itself is bad.

If I was just a *wee* bit more familiar with how Solaris loaded 
segments, I'd be able to glean some more information from the addresses 
(ie are they code or data segment pointers).  Normally I like seeing 
OS's use the high nybble or byte of an address as a segment number to 
make that sort of diagnosis easier.

It actually looks like page zero is MAPPED on Solaris (I didnt think it 
was) which in my book is a baaad thing since it means a null pointer CAN 
be dereferenced.








___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Bravery Zope 2.5 Testers

2001-12-12 Thread Matthew T. Kromer

OK, I am formulating a hypothesis that if you replace ExtensionClass.h 
in Zope 2.4 and Zope 2.5 with the following one:

http://cvs.zope.org/~checkout~/Zope/lib/Components/ExtensionClass/src/ExtensionClass.h?rev=1.16.140.2content-type=text/plain

(Which goes in lib/Components/ExtensionClass/src/ExtensionClass.h btw)

and rebuild zope  python2.1 wo_pcgi.py

and rebuld any other modules you may have which depend on extension class

that some of the segfaults could go away.

I have one person testing this right now, and am looking for others to 
prove it.

I have *not* been able to reproduce any crashing problems (even Martijn 
Faassens one) so I can't tell what may be going wrong for sure.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris

2001-12-11 Thread Matthew T. Kromer

Hi Joe,

The problem you're seeing is that the fault is happening on a different
thread than the receiver of the signal; that truss syntax is interesting
though (I have an old SPARC around to test on but its painfully slow) so I'm
wondering if first you needed to do an 'info thread' in gdb and then a
'thread N' to switch to the real crashing thread before getting the
backtrace.


- Original Message -
From: Joseph Wayne Norton [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Tuesday, December 11, 2001 2:20 AM
Subject: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on
solaris



 Hello.

 We are facing zope restarts on the solaris 5.6 platform with zope
 2.4.3 and python 2.1.1.  I put together a script based some
 information on an old posting to the apache mailing list.  The
 following shell/perl script allows one to get a core file from a dying
 zope child process and also allow the zope to restart without any side
 effects.


 The script 

 #!/bin/sh
 PATH=$PATH:/usr/local/bin
 export PATH
 cd /tmp
 for PID in `ps -u zfs -f -o pid,comm,args | fgrep z2.py | cut -d' ' -f1`
 do
 export PID
 truss -f -l -t\!all -S SIGSEGV,SIGILL -p $PID 21 \
 | perl -pe 'system(gcore $ENV{'PID'}  sleep 5  kill -9
$ENV{'PID'}), exit($ENV{'PID'}) if /(SIGSEGV|SIGILL)/;' 
 done


 Step 1:  modify script to match your environment.

 Step 2: execute script

 Step 3: wait for core file to be dumped in /tmp.

 Step 4: analyze with gdb where $PID is the pid of the dumped process

 #bash gdb /path/to/bin/python /tmp/core.$PID

 #0  0xef5b9810 in _lwp_sema_wait ()
 (gdb) where
 #0  0xef5b9810 in _lwp_sema_wait ()
 #1  0xef647ea0 in _park ()
 #2  0xef647b84 in _swtch ()
 #3  0xef6468a4 in cond_wait ()
 #4  0xef6467c8 in _ti_pthread_cond_wait ()
 #5  0x50220 in PyThread_acquire_lock (lock=0xd9d878, waitflag=1)
 at Python/thread_pthread.h:313
 #6  0x51f18 in lock_PyThread_acquire_lock (self=0xda39b8, args=0x0)
 at ./Modules/threadmodule.c:67
 #7  0x35db4 in fast_cfunction (func=0xda39b8, pp_stack=0xed40f828,
 na=0)
 at Python/ceval.c:2994
 #8  0x33ca0 in eval_code2 (co=0x267848, globals=0x51ec4, locals=0x0,
 args=0x0,
 argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
 at Python/ceval.c:1951

 :
 :


 It seems that we are facing trouble due to the thread library on
 solaris (unless the truss command has introduced a side-effect).

 Anyone else facing similiar troubles?   or maybe I should post
 this to a python mailing list.

 - joe



 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Medusa Asyncore patch

2001-12-11 Thread Matthew T. Kromer


People having problems with medusa on Linux and Solaris (particularly 
solaris) can try running this patch:

http://www.zope.org/Members/matt/patchMedusa-asyncore.py

Run it from your Zope root.  It will copy asyncore.py from the python 
lib and patch it using the patch command.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ZBabel Object incompatibility with Zope 2.5 b2

2001-12-11 Thread Matthew T. Kromer

Joachim Werner wrote:


class Rewrapper(Base):

def __init__(self, path):
self._path = path

def __of__(self, parent):
ob = parent
for p in self._path:
ob = getattr(ob, p)   = HERE WE GET A RECURSION ERROR
return ob



OK,

I think this is somthing similar to what Martijn Faassen was seeing 
where its causing an acquisition wrapper to be wrapped with itself.

I'm thinking about it right now.

As an experiment, try setting ZOPE_SECURITY_POLICY=PYTHON before 
starting Zope and see if you still get the error.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



EINTR ... was Re: [Zope-dev] browser closing connection

2001-12-10 Thread Matthew T. Kromer



Also, for the record we usually get a bunch of these quite often:

2001-11-04T09:04:33 ERROR(200) ZServer uncaptured
 python exception, closing channel zhttp_channel connected
 XXX.XXX.XXX.XXX:2181 at fb4edc channel#: 2286 requests:4
 (socket.error:(32, 'Broken pipe')

[/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/asynchat.py|initiate_send|21
4] [/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/http_server.py|send|414]
[/usr/local/zope/sw/Python2.1.1/lib/python2.1/asyncore.py|send|330])


We were seeing the same error (asyncore.py|send|330, etc) on solaris.






For what its worth, I tracked this down in the sources and confirmed 
that in Zope 2.3, we shipped a modified asyncore.py with Medusa that 
handled EINTR, but in Zope 2.4 we used stock Python's asyncore which 
does NOT handle EINTR being returned from select().  IMHO, the 
distributed Python 2.1 asyncore behavior is incorrect.

I've attached a diff of a portion of the differences (manually edited to 
take out other patches).

I suspect this patch never got integrated due to ugliness of while 1

Also, the what should this be comment relates to NT's error numbers. 
 Visual C++ has an errno.h that lists EINTR as 4 -- And winsock.h 
defines WSAEINTR as 10004 (ie add 10,000 to the errno).  SO that number 
should be 10004, not 0 for correctness on Windows.


--- /usr/local/python-2.1.1/lib/python2.1/asyncore.py   Fri Nov  9 16:28:15 2001
+++ asyncore.py Sun Oct  1 11:58:56 2000
@@ -59,8 +39,10 @@
 ECONNRESET  = 10054
 ENOTCONN= 10057
 ESHUTDOWN   = 10058
+EINTR   = 0 # what should this be?
 else:
-from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, ENOTCONN, 
ESHUTDOWN
+from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET
+from errno import ENOTCONN, ESHUTDOWN, EINTR
 
 try:
 socket_map
@@ -83,7 +65,13 @@
 r.append (fd)
 if obj.writable():
 w.append (fd)
-r,w,e = select.select (r,w,e, timeout)
+
+while 1:
+try: r,w,e = select.select (r,w,e, timeout)
+except select.error, v:
+if v[0] != EINTR: raise
+else: break
+
 
 if DEBUG:
 print r,w,e



Re: With Oracle as well Re: [Zope-dev] More signal 11 restarts....

2001-12-07 Thread Matthew T. Kromer

Dario Lopez-Kästen wrote:


Well, sorry to disapoint everybody, but we have the same signal 11 restarts
here.


Oh sure, go spoil my blame it on the other guy theory.

We are using DCO2 latest from CVS and have _very_ high Oracle database
usage.

We have yesterday changed from our solaris box to a linux box and
performance has increased dramatically (the linux box ia a 1.8 GHz P3 :).


That's to be expected with the clock speed differences.  Unless you use 
sun's CC, you get fairly poor SPARC code out of gcc, IMHO.


also the threading problems we previously had seem to  have dissapeared.


Yah!  I think that had to do with the rather *stupid* act of 
forgetfulness on my part to re-enable python threading around execute().



Now, what can we do to pin down the problem. Is there anyone else that is a
heavy databse user on similar circumstances that can share information?


I may see what I can do to try to write a script to be able to invoke 
gdb in the event of a crash.  Stay tuned.


I am starting to suspect that there is some kind of DA problem here...


Actually, since its a mysterious sig 11, it's a C module someplace... 
there is probably ONE module which is referring to an object after it 
has been deallocated.


Also, for the record we usually get a bunch of these quite often:

2001-11-04T09:04:33 ERROR(200) ZServer uncaptured
  python exception, closing channel zhttp_channel connected
  XXX.XXX.XXX.XXX:2181 at fb4edc channel#: 2286 requests:4
  (socket.error:(32, 'Broken pipe')

[/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/asynchat.py|initiate_send|21
4] [/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/http_server.py|send|414]
[/usr/local/zope/sw/Python2.1.1/lib/python2.1/asyncore.py|send|330])


We were seeing the same error (asyncore.py|send|330, etc) on solaris.

Any thoughts


Well, that means the browser user clicked 'stop'  -- Medusa is just 
telling you the channel went away on it.  Thats normal when the browser 
chops the tcp connection.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] maybe we could script it (Re: hooking up gdb)

2001-12-06 Thread Matthew T. Kromer


- Original Message -
From: Leonardo Rochael Almeida [EMAIL PROTECTED]

 Speaking of options, is linux-binary-Zope-2.4.3-python still being
 compiled with --pymalloc? As I mentioned before, our segfaults reduced
 drastically WITHOUT it.

The answer is Yes, I think :)

I built the linux binaries for the Zope distribution of Python 2.1 some time
ago and I was already aware of problems with pymalloc, so I certainly didn't
enable it; and I think 2.1 did not enable it by default.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope server speed

2001-12-06 Thread Matthew T. Kromer

Hi Alessandro,

Zope and Python combined aren't too bad if you write some wrappers that help
you manipulate data; the problem comes about when you try to do large naive
queries -- there is a LOT of overhead in taking Oracle data, encapsulating
it in Python objects, and then wrapping that in the Zope machinery.

I have a sample database in Oracle of US Zip codes -- it has about 45,000
records in it.  I can pull this into DCOracle2 on the C adapter side
(dco2.so) in less than a second.  However, wrapping each result cell in a
python object blows that up into about 9 seconds.  If I were to iterate over
the whole thing in Zope (which I haven't timed) it would get a LOT longer,
because Zope is enforcing policy on every object accessed.

So, one of the ways to make that go faster is to use python modules
'outside' of Zope (ie external methods) to perform large calculations which
takes a ZOracleDA connection object to get a cursor object on the database,
does the query directly, and formats and returns the results.

I realize this isn't ideal, but Zope is not optimized for the case of
extract thousands of records from the database and format them in a huge
table.  Zope likes extract hundreds of records from the database and
format them in a big table instead.

With each release, Zope has more features that enable you to program through
the web, but when you're doing performance intensive calculations, it's
usually time to write a python module and plug it in to zope (via an
external method, or by writing a product).  If you have the time to learn, I
suggest the product route as being more satisfying, but it can be more
frustrating as well.


- Original Message -
From: MANOTTI Alessandro [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Thursday, December 06, 2001 8:52 AM
Subject: [Zope-dev] Zope server speed



  HI all,
 
  I have a problem:
 
  my company is looking for an application server. I know Zope (I
presented
  it), and a friend of mine presented Macromedia ColdFusion.
 
  We made some tests about the speed of the systems, arghh!!! Zope is VERY
  slow compared to ColdFusion!
 
  Some examples:
 
  I used Zope with DCOracle2 drivers to connect and extract 257000 (yes!
it
  is a test... two hundred fiftyseven thousand records!) and Zope took
more
  than 10 minutes, since I stopped it after that time (call SQL and print
  data via dtml-var ...  TAG).
 
  The same PC, the same server, ColdFusion took about 2 minutes to
complete
  the operation!!!
 
  Coldfusion is more than 5 times faster than Zope!!!
 
  PHASE 2:
 
  I said: ok, maybe Zope is slower that Coldfusion in TAG formatting...
and
  I tried to get the records and perform a calculation, without no
output...
 
  Sigh! the results are the same!
 
  I am depressed, since I talk about Zope very well in my company, but
  now...
 
  Coldfusion and Zope are similar, so we shall select ColdFusion to
develop
  web applications.
 
  SIGH !!!
 
  Can I try something to increase Zope power?!
 
 
  
 
  Alessandro Manotti
  Presidente dell'Associazione RIUSA
 
  Sito web: http://riusa.apritisesamo.net
  email:[EMAIL PROTECTED]
  mailing-list: [EMAIL PROTECTED]
 
 

 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: hooking up gdb (was Re: [Zope-dev] More signal 11 restarts....)

2001-12-06 Thread Matthew T. Kromer

Andy Dustman wrote:


If you don't think a core dump is going to be useful, gdb isn't going to
be either.


Well, the problem is on Linux, the core file is from the process that 
received the SIG11, not the one that caused it, in most cases (due to 
the way Linux implements threads).  To the best of my knowledge, the 
core does NOT contain the necessary registers of the remaining threads; 
thus gdb can't show you which thread dumped core.

However, attaching gdb to the running zope usually does work, since gdb 
can inspect the processes when they are running to get the thread 
information.





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] More signal 11 restarts....

2001-12-05 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:


It's not fixed, but we managed to make it bearable so as not to lose the
client.

by replacing LoginManager with exUserFolder we managed to bring down the
zope restart time from 5 min (we have a HUGE Data.fs) to 20 secs, and by
installing ZEO we brought down the restart time to between 2 and 8 secs.
By increasing the caching of requests we managed to increase the time
between restart from 15 min to 2 hours. That and a nicely formated
Apache error page for Proxy Errors, for the lucky bastards who happen to
hit the server in the exactly 2 seconds of restart, managed to calm down
the client enough for us to breath.

I'll try the requestprofiler tip Chris gave, but I don't have much hope,
since we use mysql for authentication and the site is authenticated
mostly everywhere.

We are seriously considering droping mysql for postgresql.

   Cheers, Leo



Leo,

Are you comfortable with hooking up gdb to Zope to try to catch this?  I 
suspect, but do not know, that the MySQL python adapter is probably  not 
doing something right w.r.t. memory management.  Unfortunately, it is 
probably also the case that the problem only occurs with high-volume 
traffic -- particularly if it is a timing related bug.

We have not been able to reproduce this problem in any deterministic way 
-- and the only people who seem to have it are those who are heavy MySQL 
users; it makes me think there is something in the adapter which is not 
behaving the same way under Python 2.1 than it did under Python 1.5.2. 
 I have not looked at the adapter, so I'm making a few guesses as what 
is going wrong.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: hooking up gdb (was Re: [Zope-dev] More signal 11 restarts....)

2001-12-05 Thread Matthew T. Kromer

Leonardo Rochael Almeida wrote:

Well, one of the things I was going to ask next was for some help doing
postmortem.

We aren't getting any core files, even after setting ulimit correctly
(although we could be setting it uncorrectly. I'll look into that
further). Anyway, someone else in this list said that core dumps for
threaded apps in Linux were mostly useless, so we aren't investing much
energy in it anyway.

With the short restart times we have, I'd prever a solution that didn't
involve keeping a dead site dead for too long (as in, debugging with
gdb). We are working in a ZEO scheme that would switch over the
accelerator to proxy another zeo client, but we are not there yet.

It would be ideal if we could instruct python to grab the SIG11, invoke
gdb, get a C stacktrace for all threads and let Zope die in peace. If it
all happend in a few seconds, we will still keep the client happy.

So, to answer your question, yes, I am confortable hooking up gdb. I'd
just prefer if it could be done in as little time as possible.

   Cheers, Leo



Well largely, ALL I want is the backtrace -- and I'm wondering if I 
could cobble something together that could get it.  The problem is it 
needs to look at the symbol table, and I dont know how to get at that 
via C -- ie, gdb doesnt have an interface that I know of that you can 
link in to grab a stack trace and exit.

Its been a while since I prowled the gdb source.  I may not be able to 
do anything automatic like you want -- but I sure wish that tool was 
available!



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.3 and MySQL problems

2001-12-04 Thread Matthew T. Kromer

Radu Buzea wrote:

I have a problem with a Zope 2.3.0 (source release, Python 1.5.2 [GCC 2.96
2731 (experimental)]) on Linux and MySQL.

I'm using ZMySQLDA to connect to a MySQL database. I use a single database
table (witch has over 3500 records right now). There are over 1000 records
posted every day. The server is an AMD at 250 Mhz with 64 MB RAM. This
server was on-line since April 2001 without any problems but for 3 days ago,
when I started to use the MySQL database, the server stops sometimes
(crash!!!).

Anybody can give me some suggestions ore some explanations. What can be
wrong?
Thanks!

Radu Buzea
([EMAIL PROTECTED])


Hi Radu,

This seems to be a common problem ... I don't know if its Zope or the 
MySQL adapter that's crashing.  However, a large percentage of the 
people who are reporting these mysterious crashes are using Zope and 
MySQL together.

I am personally suspicious that they may be using a precompiled MySQL 
adapter that doesn't match the python they are using -- because of 
differences in the base object or type fields.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] active listening or parts thereof...

2001-11-30 Thread Matthew T. Kromer

seb bacon wrote:

What we need, as Paul suggested about zope-web, is a set of community
members who are able and willing to contribute 10 hours per week.  I
think there are very few such people.  I would love to, but I simply
can't.  The best way of getting such people is to cast the community
web wider, and draw more people in.  The best way of doing this is
make zope.org *really good*, I reckon.  I for one will be
pontificating on zope-web next week.


You know what an interesting about this whole community involvement 
discussion is that most of ZC monitors the lists on an ad-hoc basis. 
 Thus, there's a general notion of who on the lists is aware of what's 
going on; should an active list participant ever DIRECTLY send an e-mail 
to someone in ZC it would probably get responded to in a very prompt 
fashion.  Ergo, there are already in place the necessary parts to raise 
issues to a higher awareness; in my mind, the only fallacy is the 
assumption that merely posting to the list will guarantee that the 
people that need to see it will see it.

I would suggest that those community members who think an issue is 
important, privately pick one or two people inside ZC to directly 
involve with an issue; that person in ZC may say I dont have time right 
now but will generally be more likely to take a constructive approach 
if they feel an issue has already passed an important hurdle by an 
active participant bringing it to their attention.

This is NOT an offer on my part to get sent more home mortgage 
refinancing SPAM.  I'm not even volunteering to be a community liason. 
 I am suggesting that there are more ways to solve problems than pure 
consensus, and that initiative can make all the difference in the world.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Resize a image stored in a Oracle's BLOB Field

2001-11-27 Thread Matthew T. Kromer
I'm not really sure when this was asked ( I dont think it was '99) :)

The img> tag is in the containing document, not the actual method which returns the image...  Somewhere else you have an img src="myZopeUrlWhichReturnsAnImage"> which is where you slap the height and width tags.


On Friday, November 12, 1999, at 07:40 AM, Juan Javier Carrera Obrero wrote:

Hello,
 
Thanks, I have displayed the images which them was stored in a Oracle's BLOB field, with:
 
dtml-call expr="RESPONSE.setHeader('content-type','image/jpeg')">
dtml-in SQLMethod_with_a_Only_Row>
dtml-var expr="BLOB_FIELD.read()">
/dtml-in>
However, I would like to resize this image. I can not do it with the "height" and "width" attributes, because the image is obtained from a Oracle's BLOB field, and no from image object (i.e.: I can not use dtml-var "ImageObject.tag(border='5', align='left')">).
 
How can I do it ?
 
Thanks again !
 


Re: [Zope-dev] More signal 11 restarts....

2001-11-27 Thread Matthew T. Kromer

The only real suggestion I have is to attach the debugger to a running 
thread and hope it hits the fault while the debugger is attached.  Linux 
core files are difficult to impossible to debug when threading is 
active.  The 2.4 kernels may have addressed this some but I dont know 
that gdb has caught up.


On Monday, November 26, 2001, at 12:47 PM, Daniel Duclos wrote:


   I have a zope that is dumping signal 11 every 40 minutes or so. I 
 have tried
 recompile python 2.1.1 with-threads without-pymalloc, recompile Zope 
 with it,
 recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, 
 upgraded to
 Zope 2.4.3, all this on a Debian Linux box.
   Nothing changed... still restarting...
   Anybody, please, has any ideia on this matter?
   Please, let me know if there's any relevant info that I forgot to 
 mention
 abot my case!
   Thanks in advance!!

 --
 daniel lobato duclos -- [EMAIL PROTECTED] -- 
 http://www.hiperlogica.com.br
 -

 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Images in Oracle's BLOB Field

2001-11-08 Thread Matthew T. Kromer

Juan Javier Carrera Obrero wrote:

 Hello,

  

 I have stored images in a Oracle's BLOB field. I have tried to display 
 them from Zope in a dtml-in method, but I can not show them.

  

 Anybody knows to how do it ?

  

 Thanks.-

  

 Note: I am using Zope 2.4.1, Python 2.1 and Oracle 8i (with the 
 DCOracle2 Adapter).

  


Hi Juan,

Sorry I haven't read the list in a while!

It seems to me that the way you do this is by explicitly using the 
LOB.read() mechanism...

ie

dtml-with mySQLqueryWhichReturnsALOB mapping
dtml-var expr=LOB.read()
/dtml-with

Except that you have to be tricky and make sure there are no newlines 
and so on, so you end up doing like the following (untested, but close):

dtml-call expr=RESPONSE.set_header('content-type','image/jpeg')
 dtml-with ...
 next tag
 you see the pattern?



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.4.2 DTML Method Bug

2001-11-01 Thread Matthew T. Kromer

Casey Duncan wrote:

One of my products exposed a bug in the 2.4.2 version of DTMLMethod.py. It is 
minor and can be worked around, but I wanted to point it out:

Line 203 of DTMLMethod.py now contains:

del self.__dict__['validate']

which is part of a try...finally statement.

It seems that the validate key is not always present in the object at that 
point, specifically if you recursively call an object in a different context 
then it was originally called. as in:

dtml-with name=something
  dtml-return name=this
/dtml-with

This piece of code resulted in a KeyError on validate in my product code, 
which had previously worked fine. Perhaps another try statement should be 
wrapped around this del statement?

Hi Casey,

I think that is fixed but I'm not positive that its in 2.4.2  -- I know 
its on my 2_4 branch; I think we just barely missed this for 2.4.2 -- I 
see the change going into the log about a week later.

I'll ask Brian if we're going to put out a 2.4.3 to include the fix.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.4.2 Error

2001-10-23 Thread Matthew T. Kromer

E. Seifert wrote:

Hi list,

I just installed the 2.4.2 update over my Linux Zope 2.4.1 and I get the
following traceback:

Traceback (innermost last):
[...]



(Object: standard_html_header)
  File /usr/local/zope/lib/python/Products/Renderable/Renderable.py, line
17, in __str__
(Object: standard_site_menue)
  File /usr/local/zope/lib/python/OFS/DTMLMethod.py, line 195, in __call__
(Object: render)
  File /usr/local/zope/lib/python/DocumentTemplate/DT_String.py, line 546,
in __call__
(Object: render)
  File /usr/local/zope/lib/python/DocumentTemplate/DT_In.py, line 770, in
renderwob
(Object: objectValues(['SimpleNav Element']))
  File /usr/local/zope/lib/python/Products/Renderable/Renderable.py, line
17, in __str__
(Object: ab)
  File /usr/local/zope/lib/python/OFS/DTMLMethod.py, line 203, in __call__
(Object: render)
KeyError: validate

Is there someone who can help me?

TIA
Erich


Hi Erich,
 
Sorry it took me so long to read Zope-dev and see this question.

There's a section in lib/python/OFS/DTMLMethod.py which was changed in 
2.4.2 to accelerate DTML security processing, but it doesn't look like a 
subsequent patch to make it handle reentry was routed to 2.4 yet. 
 (Actually, I think it's waiting in a dev branch, because it's not on 
the trunk yet either.)  For the time being, what you can do is comment 
out both lines in DTMLMethod.py which read

self.__dict__['validate'] = security.DTMLValidate

and

   del self.__dict__['validate']

It looks like the renderable product you're using is causing the 
DTMLMethod to be reentered, which is tripping out this bug (ie you can 
only delete validate once -- the second time it fails).  


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] [Fwd: Re: [Zope] Python 2.1 for debian ?]

2001-10-03 Thread Matthew T. Kromer

Jim Washington wrote:


 Would it be smart to include the python header files in the zope 
 binary distributions?  That would seem to solve a few problems for the 
 individuals using them.

 -- Jim Washington

The next binary release will go out the door with Python headers 
installed.   We've already set it up for the packager to do that.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] [Fwd: Re: [Zope] Python 2.1 for debian ?]

2001-10-03 Thread Matthew T. Kromer


On Wednesday, October 3, 2001, at 10:53 AM, Leonardo Rochael Almeida 
wrote:



 Matthew T. Kromer wrote:

 The next binary release will go out the door with Python headers 
 installed.   We've already set it up for the packager to do that.


 Does that mean that using the binary Zope bundled python to call 
 setup.py or using the binary Zope bundled Makefile.pre.in will 
 correctly build C modules and install them correctly inside the binary 
 Zope python tree?!

Yes, I think at least the first part is true, such that setup.py build 
will work -- the ability of install to function is dependent on your 
permissions, of course.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Problems with Oracle DA and Dates

2001-10-02 Thread Matthew T. Kromer

Geir Bækholt wrote:

Hello Dario, 

Just noticed behaviour similar to this a couple of days ago , but
haven't had the time to file a report on it.. - We managed to narrow
it down a bit , tho' :

In our installation ; Zope silently restarted , quite quickly , and
almost unnoticeable to our editors.
This was triggered either when we passed one of the
oracle-date-objects returned by DCO2 to DateTime(), or when we
believed them to be DateTime objects and tried to run DateTime methods
on them...

I was in a production environment , so i had to fix the errors before
going to work on narrowing down the bug. Selecting your dates as
strings with TO_CHAR(datefield) in your SQL will be a safeguard
against the restarting/crashing/whatever_bad_things_might_happen , but
will give you boring strings instead of date-objects.. -ok , it is a
stopgap, but my Zope stopped restarting every 2 minutes...

Hoop we can get this fixed before the final is released..
:-)

So what you REALLY want is the ZOracleDA to promote the weakling 
dco2.DateTime objects to full Zope DateTime objects.

Aha!

I think that's doable in short order.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Problems with Oracle DA and Dates

2001-10-02 Thread Matthew T. Kromer

Matthew T. Kromer wrote:

 Geir Bækholt wrote:

 Hello Dario,
 Just noticed behaviour similar to this a couple of days ago , but
 haven't had the time to file a report on it.. - We managed to narrow
 it down a bit , tho' :

 In our installation ; Zope silently restarted , quite quickly , and
 almost unnoticeable to our editors.
 This was triggered either when we passed one of the
 oracle-date-objects returned by DCO2 to DateTime(), or when we
 believed them to be DateTime objects and tried to run DateTime methods
 on them...

 So what you REALLY want is the ZOracleDA to promote the weakling 
 dco2.DateTime objects to full Zope DateTime objects.

 Aha!

 I think that's doable in short order. 

In fact, here's the simple patch (only the last bit is really necessary, 
but as long as I was looking at it I tweaked the listtype to be a 
pseudo-constant as an argument.  I've checked it in to CVS too :)

Index: db.py
===
RCS file: /cvs-repository/Products/DCOracle2/db.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -r1.8 -r1.9
--- db.py   14 Sep 2001 15:11:19 -  1.8
+++ db.py   2 Oct 2001 21:13:58 -   1.9
@@ -83,8 +83,8 @@
 #
 ##
 
-'''$Id: db.py,v 1.8 2001/09/14 15:11:19 matt Exp $'''
-__version__='$Revision: 1.8 $'[11:-2]
+'''$Id: db.py,v 1.9 2001/10/02 21:13:58 matt Exp $'''
+__version__='$Revision: 1.9 $'[11:-2]
 
 import DCOracle2, DateTime
 #DCOracle.dbi.dbiDate=DateTime.DateTime
@@ -230,12 +230,14 @@
 #
 # Do we get tuples back in results? should just be lists
 #
-def _lobConvert(self, result):
+def _lobConvert(self, result, listtype=type([])):
 for i in xrange(len(result)):
 t = type(result[i])
-if t == type([]): self._lobConvert(result[i])
+if t == listtype: self._lobConvert(result[i])
 elif t == DCOracle2.dco2.LobLocatorType:
 result[i] = LobLocator(result[i])
+elif t == DCOracle2.dco2.OracleDateType:
+result[i] = DateTime.DateTime(float(result[i]))
 
 # Added for ChrisM (11/13/2000 MTK)
 def commit_sub(self, *arg, **kw):



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] compiling Zope 2.4.1 on Mac OS 10.1

2001-10-01 Thread Matthew T. Kromer

Mitchell L Model wrote:

 Trouble compiling Zope 2.4.1 on Mac OS 10.1:

 tried 'python wo_pcgi' with both a Python 2.2a4 I just made and
 with my previous Python 2.1, both with and without sudo

 cc  -bundle -undefined suppress  ./ExtensionClass.o   -o
 ./ExtensionClass.so

 /usr/bin/ld: -undefined error must be used when
 -twolevel_namespace is in effect

 [...]


 The 'two-level namespace' business is a change from the OX X developer 
 tools version 10.0 to 10.1.


 Anyone know what's going on here and how to fix it?

Well, besides the obvious that MacOS 10.1 is only hot-off-the-shelves 
for a few hours, the problem is that the dynamic loader under Darwin 
used a flat namespace to load ALL shared libraries into (does Windows do 
the same? I dont know) whereas most Unixes load each shared library with 
its own namespace [which is a bit of a technical abstraction and hand 
waving].  It sure looks to me like Apple is trying to fix the namespace 
collision problems with dyld, with that -twolevel_namespace (which 
sounds like a funky namespace munger).  My best guess at noodling the 
error message is that it means your library must use the symbol 
'undefined error' when the loader goes to load a name from the library 
and it is not present.  It may be that there is something like a 
interrogation call to the library -- do you have member X? -- rather 
than what I'm used to, which is the loader looking in the library's 
table of contents.  I suspect, but dont know, that it has to do with how 
Objective C methods get bound dynamically.


Thats not a very helpful response, but until I go to an Apple store to 
get OS 10.1, I can't see it in action or read the docs.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] somehow OT: How do I get a tar file containing the .dtml files only?

2001-10-01 Thread Matthew T. Kromer

Joachim Werner wrote:

Hi!

I am working on internationalizing a Zope 2.4.1 instance with ZBabel and
providing a set of patches for Zope 2.4.1. My idea was that things would
become much easier if I had a tarball that just contains all *.dtml files
and the folder structure for them, so I can distribute the tarball as a
patch that can be applied to a plain 2.4.1 instance.

Do we have any Unix/Linux/GNU experts on the list who know how to write that
as a shell command? I am sure that with tar and grep and some pipes or so it
is possible, I am just too dumb to find out the right syntax  ...

Cheers

Joachim


So you want something like a find . -name '*.dtml' which doesn't show 
directories?  Well, what you could do to create the tarball is something 
like:

find . -name '*.dtml' | tar -T - -cvf dtml.tar

which uses -T on gnu tar to read the filenames from stdin.





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Zope 2.5: User limit for Local Security Role assignment list

2001-09-28 Thread Matthew T. Kromer

One of the things Zope 2.5 is going to do is limit the answers that are 
available from a user folder when presenting the local role dialog; this 
will prevent humongous lists from displaying and instead show a simple 
text box where a user ID can be typed.

However, for existing user folders, this poses a question of:  what's 
the reasonable number of users to list without having to go adjust a 
property on the folder?

The revised implementation will let you tune how many users to display 
before putting up an input box; but currently I've just defaulted it to 
'old behavior' -- ie show all the possible users.  Maybe a better thing 
to do would be to define a  default of 250 or so.  This would probably 
be large enough to not impact most sites, yet small enough to make it 
meaningful for those sites that do have enormous user folders.

It is also worth pointing out that local role assignment code doesn't 
enforce that the users for whom it has local roles exist in any current 
database; it is possible to grant rights to nonexistent users.  Before 
the ability to enter a name in directly was available, you would have 
had to explicitly change the form values manually -- but this exposes 
the problem.  I dont think it poses a problem (ie I dont believe Zope 
has ever tried to define a mechanism whereby objects are notified that a 
user for whom they have permissions has been deleted.)



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ZODBCA for Python 2.1

2001-07-11 Thread Matthew T. Kromer

Andy McKay wrote:
 
 Since the source of ZODBCA is not available could someone in DC compile this
 for Python 2.1 / Zope 2.4? Or make the source available? Pretty please :)
 
 http://www.zope.org/Products/DA/ZODBCDA
 
 Thanks.
 --
   Andy McKay.

Hey, AFAIK the src directory inside the ZODBCDA tarball is the
source; I spent a few moments and rebuilt the SQL.PYD file for
Python2.1.  The replacement SQL.PYD file is at

   http://www.zope.org/Members/matt/ZODBCDA/sql.pyd

Just copy that over the top of the existing sql.pyd file in
lib/python/Products/ZODBCDA and you should be back.  I make zero
guarantees how well it works -- I only recompiled it, and I
don't use it myself.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] DCOracle2 Beta 3

2001-06-18 Thread Matthew T . Kromer


On Saturday, June 16, 2001, at 08:41 PM, Andreas Repp wrote:
 (quoting himself)
 ... To solve the performance problem I´d suggest to cache the user
 connections for a certain period of time. ...

 Of course I haven't meant to 'cache' the connections but to keep em 
 alive
 while the respective user is active and shut it down after a timeout
 period has passed.

I was reading the Oracle9i OCI documentation (Oracle 9i was just 
released) and one of the interesting things it has is connection 
pooling, with some kind of proxy authorization.  As I get a chance I may 
investigate this further; first to see if it works, then to get a feel 
for for how to integrate it into Zope.

It looks like an extension of something you could already do, but the 
documentation is frustratingly light on examples.  In any case, the 
promising part was that you could leave the proxied user's password 
blank, as long as session was authorized for proxies.  That would 
eliminate the whole necessity of having to track the database level user 
passwords.


 ##default_connection_id=(changed via ZSQL manage_main Interface)
 ##connection_type=[ standard | custom | optional ]
 ##connection_options=(individual Oracle Connection String)
 sql yada yada

 'standard' = obvious
 'custum'   = forced custom connection - will raise error if 
 connection_options
  dont' have a valid connection string
 'optional' = try 'custom' and fallback to 'standard' if it fails

 btw: would be nice to have a drop box in the manage_main screen for this
 stuff in a _far future_ release of ZSQL-Methods ;-)

Well there are still userid mapping problems present.  It's not 
necessarily reasonable to assume there is a 1:1 mapping between Zope 
userids and the database userids.  What I'd probably be inclined to do 
is change the *connection* object to look for some other acquired 
authenticator, which it could use to start a session.  That way you can 
write your own authentication conversions and plug them in.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] DCOracle2 Beta 3

2001-06-12 Thread Matthew T. Kromer

For all of you (or at least, BOTH of yousmirk) Oracle users out there, 
I packed up DCOracle2 Beta 3 this morning, including Z Oracle Stored 
Procedures as part of ZOracleDA.  

It can be found in the usual place,

http://www.zope.org/Members/matt/dco2

This isn't much different from Beta 2 except for some SP tweaks and the 
Z Oracle Stored Procedures, except that the NT builds are release 
versions with Oracle 8i (e.g. SP discovery code) features enabled -- 
when I built Beta 2 I goofed the build and packaged a debug release 
without the 8i specific features on.

I'll confess that I'm not a heavy user of Stored Procedures, so your 
mileage may vary; particulary when using IN/OUT binds where type 
conversions need to take place from unweildy Oracle native types (like 
SQLT_NUM.)

If you play with the stored procedure objects in Zope, it's probably 
worth pointing out that the default permissions won't include execute 
permission -- you have to specifically enable that for non-manager roles.

On the far-out front, a few people have asked how you might do 
user-specific connections to the database (ie instead of pooling by a 
common connection, you individually authenticate to Oracle with your own 
password).  Clearly, cloning the base Connection object would be fairly 
straighforward, but authenticating it to Oracle would not be, as the 
user password wouldn't be available.  If anyone feels strongly about 
that (ie you have a notion on how you would make it work and/or you need 
it), feel free to write me.  From a performance standpoint it would be 
terrible, but security auditors would love it.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] DCOracle2 Beta 1 Announcement

2001-06-05 Thread Matthew T. Kromer

Description
DCOracle2 is a replacement for DCOracle, written primarily in C. DCOracle 1
uses OCI 7 bindings for most Oracle calls, with OCI 8 mixed in for LOB
support. Oracle 8i disallows mixing of calls within a statement, and so
breaks LOB support. DCO2 uses entirely OCI 8 calls, and thus can use LOBs.

New in this Release
Beta 1
Stored procedure input works properly, cycles in stored procedures removed.
Stored procedures now have meaningful docstrings (describing their
parameters). Type coercion change from a tuple kludge to a TypeCoercion
object. Set ability (and default) to do static binding for BindingArrays,
working around dynamic fetch occasional NULL bug on Linux. Batch
executemany(). Add backward compatable dbiRaw and execute modes.

Alpha 6
Nested Cursors, e.g. SELECT ENAME, CURSOR(SELECT ENAME FROM EMP WHERE
MGR=7908) FROM EMP WHERE EMPNO=7908). API 2.0 type objects. Add trim() to
LobLocators. Wrap LOB permissions for Zope in ZOracleDA. Change result on
execute w/o results to None, not []. Return statement type code after
execute().

Alpha 5
Stored procedure fixes, and debugging enhancements.

Alpha 4
Stored procedure IN/OUT variables, changes to executemany()

Alpha 3
Changed ZOracleDA to not use method call to connect(). Partial stored
procedure work

Alpha 2
Bug fixes, largely packaging, from Alpha 1. Added SQLT_AFC handler, SPARC
alignment fixes.

Alpha 1
First www.zope.org release

Contents
This release contains both DCOracle2 and a slightly modified ZOracleDA; it
will register as ZOracleDA would (to silently upgrade Oracle connections)
and thus cannot be run concurrently with ZOracleDA/DCOracle.

Installation
To replace ZOracleDA, untar into lib/python/Products and make, move
ZOracleDA out of lib/python/Products, and rename lib/python/Products/DCO2 to
lib/python/Products/ZOracleDA.

Usage
This release is intended for testing with ZOracleDA feature compatibility
(including LOB support) and is also intended for general use.

Platforms
NT support has been tested, Microsoft Visual Studio project files are
included; this has only received testing with Oracle 8.0 and Oracle 8.1 on
Linux, Solaris, and Windows NT; a wider variety of platform experience is
welcomed.

Download
The product is available at http://www.zope.org/Members/matt/dco2.




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] DogFishbowl?

2001-05-31 Thread Matthew T. Kromer

trumpets blare

Ta da, I grant  you *official* permission to develop and contribute such a
product.

;)

That and $7CDN will get you lunch at McZargalds.

Yes, Wikis are are too sticky, and that's not cricket.

- Original Message -
From: Andy McKay [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, May 30, 2001 7:10 PM
Subject: [Zope-dev] DogFishbowl?


 How about combining a fishbowl and a dogbowl by using Shanes Workflow tool
 on the fishbowl? It seems a little wierd that is all a bunch of Wiki's,
 great for making comments and development but rather hard to track
changes,
 manage (I assume) and enforce guidelines.

 Wouldn't it make more sense to have a product that did this? Doesnt have
to
 be anything fancy. It would be nice if for example I could an xml-rpc call
 to Proposals to get a list of new proposals, rather than now which is read
 through it by hand.

 Time for the fishbowl to mature now its used a lot? Perhaps start a
proposal
 on the fishbowl?

 Cheers.
 --
   Andy McKay.




 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ZODBCA questions...

2001-04-09 Thread Matthew T. Kromer

ZODBCA is on my list of things to evaluate freshing...  there is a good
chunk of code out there for a CLI interface.  One of the project wikis
mentions it somewhat, which I'll update on dev.zope.org as I get better
details.

The downside is I am not an NT evangelist; I'm more like a penguin in a
demilitarized zone.  Now, if we were talking SQL/DS on VM; I'd happily crank
out 370 asm bindings for REXX :)  I even used to have one of those cool
orange plasma tubes (but what you didn't probably know is that they were
SLOW to refresh.)


- Original Message -
From: "Andy McKay" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, April 09, 2001 1:45 PM
Subject: [Zope-dev] ZODBCA questions...


 Im getting some performance issues with my sites, im connecting Zope to a
 SQL Server (not using ZEO). I've got suspicion, but no proof, that its
 ZODBCA that is the problem. Does anyone have experiences with this on
large
 sites and / or a suggestion to replace it?

 Cheers.
 --
   Andy McKay.




 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] DCOracle2 Alpha 5

2001-03-21 Thread Matthew T. Kromer

I just packaged up DCOracle2 Alpha 5 on Zope.org.  Given that
there are no glitches or
geez-I-shouldnta-changed-that-at-the-last-minute fixes present,
I'm ready to have a wider set of people eyballing it.  A few
have, largely just from seeing it show up on the News column of
Zope.org.

If you use DCOracle, either with Zope/ZOracleDA or directly, and
are interested in evaluating it, I am currently welcoming all
feedback, from packaging, documentation, functionality, etc.

While I am nominally subscribed to zope-dev, I route all its
mail into a separate folder that -- surprise, surprise -- I
don't read very frequently.  I strongly encourage people to
contact me directly with their comments or questions.

I have NT builds and Unix/NT source available at 

http://www.zope.org/Members/matt/dco2

The NT build is against Oracle 8.1.7; I am not a real NT person
so I don't know how portable that binary is.

I am *NOT* interested in answering general Oracle questions; I'm
trusting those who mail me will take this into consideration
(i.e. if it isn't about DCOracle or DCOracle2...)

This should build on Solaris with gcc and on Linux systems; I
have no access to other platforms to try them.  The 'make'
command at the top directory should be sufficient to build from
source on Solaris or Linux.  Obviously, you need Oracle
installed -- I get flaky behavior with Oracle 8.0 on Linux, but
much better behavior with Oracle 8.1.  

Stupid database inserts (e.g. repeat 'INSERT INTO...' 100 times)
seem to be about 25% faster than DCOracle.  LOBS seem to work. 
Stored procedures (at least my simple ones) work.  Complex stuff
may not.  I still have not done any REF CURSOR stored procedure
testing, so that's probably quite buggy.  Ditto automatic
conversion for ROWIDs (see the documentation at how to get at
the raw unconverted data).  I get bad Oracle behavior (NULL
data) trying to insert more than 8-10 rows with executemany()
but it's very random and not visible where the problem is (it
just *CANT* be an Oracle bug!)

Anyone who wants to do platform migration to other 'nixes is
encouraged to submit patches.

If you rate the docs, please don't complain about formatting in
the HTML version, as it's just a "save as..." HTML doc, and
looks like it.  The dco2documentation.aw file is the source
document, but I've also provided postscript, rtf, and html
versions.

And, while there is a setup.py in the top level (for distutils)
it isn't platform aware; Andreas Jung kindly provided it to me
and I haven't spiffed it up, since I (like Majikthies and
Vroomfondle) demand rigidly defined areas of doubt and
uncertainty (read: I know how to do it better with shell scripts
 makefiles).

Thanks,


Matt Kromer
[EMAIL PROTECTED]

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] ZOracleDA not working with 2.2.1?

2000-09-15 Thread Matthew T. Kromer

[EMAIL PROTECTED] wrote:
 
 Hi *,
 
 after upgrading to latest Zope we are having problems with the
 ZOracleDA product. The first SQLMethod **after** a call to an external
 method using DCOracle to update the db, fails with an error saying that
 the connection is closed. (connection can't be re-opened, the only way
 out is to restart zope, sigh.)
 
 anybody experienced the same problem? is ZOracleDA non-compatible
 with Zope 2.2.1?
 
 ciao and tahnk you very much (as always),
 federico
 
 --
 Federico Di Gregorio
 MIXAD LIVE System Programmer   [EMAIL PROTECTED]
 Debian GNU/Linux Developer  Italian Press Contact[EMAIL PROTECTED]
  A short story: I want you. I love you. I'll miss you. -- Me

Please try using DCOracle 1.3b1 to see if that fixes things;
there are some bugs with connection handling in versions of
DCOracle where if the connection dies it will not recover.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] DCOracleStorage: problems running Full.sql

2000-09-07 Thread Matthew T. Kromer

Roman Milner wrote:
 
 I'm trying to get DCOracleStorage up and running, but I'm getting
 errors trying to run Full.sql.  I know didly about plsql, so I
 apalogize if this is obvious.
 
 When I run Full.sql I get the following errros.  Does this mean my
 oracle installation is missing something?
 
 Thanks,
 ^Roman
 
 ...
 
 Table created.
 
 Table created.
 
 Index created.
 
 Commit complete.
 
 Package created.
 
 No errors.
 
 Warning: Package Body created with compilation errors.
 
 create package body zodb_methods as
 *
 ERROR at line 1:
 ORA-01041: internal error. hostdef extension doesn't exist
 
 No errors.
 ORA-24323: value not allowed
 Error accessing package DBMS_APPLICATION_INFO
 ERROR:
 ORA-03114: not connected to ORACLE

Offhand, I'd say that represents a goofed oracle install. 
Sadly, goofing the oracle install is easy to do.  I've tried
DCORacleStorage against 8.1 and had no problems getting Full.SQL
to run, but I have not tried it against 8.0.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] ZOracle LOB Questions

2000-08-01 Thread Matthew T. Kromer

Monty Taylor wrote:
 
 Is anyone working on fixing DCOracle? I'll have someone look into if no
 one else is, but I'd prefer to spend the energy elsewhere if it's
 already being addressed. (The LOB support would be nice for me as well.)

I've got a partial rewrite, but the LOB support hasn't gone in
yet.  Most of the functionality other than LOB support has gone
in.  When I tested retrieving a LOB this morning I got a nice
core dump in Python after the fetch completed, and I haven't
figured it out quite yet.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




  1   2   >