Re: [Zope] URL0 returns index_html not index.html

2005-08-18 Thread John Eikenberry
Mark Barratt wrote:

 Zope 2.7.4 on Debian
 
 DTML method index_html in the root says dtml-var _['index.html']
 
 A link in a page template to
 
 tal:attributes=href string:${context/REQUEST/URL0}/source.html

URL0 always(?) includes the ending published object (eg. index_html).
Normally if you don't want this you just use URL1.

 where the page is addressed by [path]/ and is actually at [path]/index.html
 
 returns [path]/index_html/source.html

Using URL1 you should get [path]/source.html which would seem to me to be
what you want.
 
 I can vaguely see why this is happening. My question: is there a 
 straightforward way of making the links (and error reports) return the 
 actual page address?

By actual page address you want the URL with index.html instead of
index_html? Given your current setup as described, redirecting would work.

dtml-call RESPONSE.redirect(URL1+'/index.html')

Though this would cause every hit on a page's index to tranverse twice. If
the site isn't super high traffic it wouldn't be noticable.

If you were working with zope projects there are other tricks you could
pull, but it doesn't sound like you are doing this.

 (I think I understand that 'actual' 'page' and 'address' are all 
 metaphors which may be leading me into error... )

In zope there is the URL which represents the object traversal and there is
the returned content (html, images, etc). The returned content (maybe what
you mean by actual page) can feasibly come from some other part of the
site, a database, or just about anything else. The URL or address is just
what method/object gets run to get said content.

 Thanks for any help/enlightenment.
 
Not sure if I'm making this much clearer. But there it is anyways.
 

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: FIXED: Re: [Zope] URL0 returns index_html not index.html

2005-08-24 Thread John Eikenberry
Mark Barratt wrote:

 Well, no, because some of the objects I want to append /source.html to 
 are not called index.html (but that *is* how we did it for another site 
 and it works as you say).
 
 A text substitution covers both cases:
 
 tal:attributes=href 
 python:context.REQUEST['URL0'].replace('index_html','index.html')+'/source.html'

Wouldn't this just result in [path]/index.html/source.html? Do you want
the index.html in the URL to source.html. This will work with Zope but
looks strange. Using URL1 instead of URL0 would leave the index[._]html
off the generated URL. 
 
 If you were working with zope projects there are other tricks you could
 pull, but it doesn't sound like you are doing this.
 
 I don't understand this: is a 'zope project' different from my project 
 using zope?

Opps. I meant a 'zope product', ie. a python based product.
 

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] manage_afterAdd quirks

2005-08-24 Thread John Eikenberry
Philip J?genstedt wrote:

 It seems I need one of three things:
 
 1. A better way to add images and a template altogether.
 2. A hook which is only called once, when the object is first created.
 3. A method to check if the objects exist in the newly added item
 directly, not aquired.

As Peter mentioned, use aq_base. There are 2 ways to use it. One as an
object attribute (self.aq_base) or using the aq_base() function as imported
from Acquisition. The latter method won't cause an attribute error if you
already are working with the non-acquisition object (in that case it will
simply return the object. For what you are doing I recommend one of these
methods.

To get more control over when manage_afterAdd runs some code, you can set
some flags using the copy hooks. Before running manage_afterAdd when
copying/moving/renaming ObjectManager calls the method _notifyOfCopyTo() on
the object being copied/moved/renamed. It passes one argument (op) giving
some context to the call. It is set to 1 for moves and renames, and set to
0 for copies. 

So you can use this to set a flag on your object to modify
manage_afterAdd's behaviour in these circumstances. Eg.

class Container(ObjectManager):

def _notifyOfCopyTo(self,op):
# use a _v_ volitile attribute here to avoid persistence
self._v_copy_flag = op

def manage_afterAdd(self, item, container):
if hasattr(self,'_v_copy_flag'):
if self._v_copy_flag == 0:
# stuff you want done when copying
...
if self._v_copy_flag == 1:
# stuff you want done when moving/renaming
...
# clear the flag
delattr(self,'_v_copy_flag')
else:
# stuff you want done only on initial install
...
# stuff you want done no matter what.
.,.

Of course you can simplify this if you don't care if its been moved,
renamed or copied. Just wanted to show the different possibilities.

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] passing a parameter - namespace and dtml-with

2005-08-29 Thread John Eikenberry
Sean Kelley wrote:

 I am trying to pass the title of a page to a dtml method which is in another 
 folder named links. If I pass the actual title like below everything works. 
 However, when I try to pass the value of the current title to 
 category_results using various other methods it does not.
 What is the syntax so that I can pass the current value of dtml-var title 
 to the method? Would the dtml-with block change the namespace it pulls the 
 title variable from to the context of the method category_results in the 
 links folder?

Yes.

Here's a quick hack that will work.

dtml-call REQUEST.set('orig_title',this().title)
dtml-with links
dtml-var catagory_results(_.None,REQUEST.get('orig_title'))
/dtml-with

  dtml-with links
 dtml-commentThis line works/dtml-comment
 dtml-var category_results(_.None, _, title1='Hannoverian Stallions Lauries 
 Crusador ')
 /dtml-with

 ___
 Zope maillist  -  Zope@zope.org
 http://mail.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists - 
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope-dev )


-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] passing a parameter - namespace and dtml-with

2005-08-30 Thread John Eikenberry
Peter Bengtsson wrote:

 Better
 
 dtml-let orig_title='Hannoverian Stallions Lauries Crusador ' 
 dtml-with links
 dtml-var catagory_results(_, _.None, title1=orig_title)
 /dtml-with
 /dml-let
 
 Use dtml-let and notice the extra _ in the parameters. I doubt that
 DTML methods accept plain arguments except self, REQUEST and RESPONSE.
 All other things must be passed with keyword arguments.
 
Opps. Yes. dtml-let is what you should use. Been awhile since I've done to
much with dtml.

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: passing a parameter - namespace and dtml-with

2005-08-30 Thread John Eikenberry
Alexander Limi wrote:

 comment mode=off-topic
 Wow, I had forgotten how incredibly ugly DTML can be. That just looks  
 wrong, like you stumbled on your keyboard. Maybe I should go hang out on  
 the perl lists and build up some tolerance for interesting syntax. :^)
 /comment
 
Yes. DTML is ugly, which is why its bee superseded by page templates. ZPTs
are much nicer and just about the best way to do html-embedded display
code that I've seen. At least compared to the alternatives.

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


[Zope-dev] ZPatterns design questions

2000-10-03 Thread John Eikenberry


First, some context... I'm working on a new data storage system for a
related set of (primarily) Zope sites. The data will be kept in ZODB and
mirrored out to a RDB (MySQL) to provide read-only access to some non-zope
stuff. One other thing to note, all of this will be done via python code
(no zclasses).

Now say that I want to keep data about a company. I have a specialist, rack
and dataskin already created. I currently can store data on the skin via
AttributeProviders, but this was mainly done to get something working to
mess around with. 

What I think I want is several SheetProviders. Two for each block of data
(a block basically reflect the breakdown of tables in the MySQL DB). Why 2
for each block, one for the ZODB store and one for mirroring the data out
to the RDB. So, the breakdown for the company would look like this:

Company Rack
  - Company Information (name, url, etc). 
. SheetProvider (ZODB) - read/write
. SheetProvider (SQL) - write only
  - Address Information (city, state, zip, etc)
. SheetProvider (ZODB) - read/write
. SheetProvider (SQL) - write only

Does this sound reasonable? I thought about writing a mix-in class to add
the SQL stuff to the SheetProvider, eliminating the need for 2 entries for
each block of data. But I liked the idea of each being its own plugin,
seemed cleaner.

The specific question I have about this design is where does the
_properties() attribute go? They need to be associated with their
appropriate SheetProviders, so that seems the most likely place, but that
would require a SheetProvider subclass for each data block which bothers
me. It also just doesn't seem to fit in with the design pattern, but my
understanding of it is still fairly limited.


Next Topic... 

Adding to the above design, we add a second entity; People. I need to
express relationships between the people and the companies. These
relationships can be of various sorts; eg. technical contact, marketing
contact, company admin, etc.

My idea is to create another specialist/rack/dataskin to contain the
information about these relationships and associated data (type of
relationship, corresponding roles, references to the related entities, etc). 

I don't have any code for this aspect of the design yet, so I'm only
fishing for general criticisms/thoughts at this point. Is there a better
way to do this? BTW, there are going to be many more types of entities and
relationships that these examples, so it needs to scale. It also needs to
perform well, as the authentication mechanism will be fetching the roles
off the relationships and, possibly, their related objects. 

Thanks in advance for any advice.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] ZPatterns design questions

2000-10-03 Thread John Eikenberry

Phillip J. Eby wrote:

 If I can offer a suggestion...  

Please...

 It sounds to me like you don't need SheetProviders at all, if you
 effectively define the property sheets as part of your class, and make
 the attributes direct attributes on the DataSkin.  You then need only set

Ok, I'm feeling pretty stupid right now but I have to ask what you mean by
"define the property sheets as part of your class"? Do you mean using
something like this in the DataSkin subclass:

def __init__(self,id):
self.DataSkin.__init__(self,id)
ps = self.propertysheets
ps.manage_addPropertySheet('CompanyData','CompanyData')
ps.get('CompanyData')._properties = (
{'id':'name', 'type':'string',  'mode':'w'},
...
)

 up a single trigger that checks whether any of the attributes you want to
 mirror have changed, and then fires that off to the SQL db.  It would

By 'trigger' you are referring to a RuleAgent plugin? Hmm... I had briefly
thought about this, but most of the discussions relating to interfacing to
an external RDB seemed to indicate subclassing a SheetProvider was the best
course of action. I'll have look into this (hadn't spent much time figuring
out RuleAgents yet).

 actually be a bit easier to set up if you were using a ZClass, since you
 could create the property sheets there by just adding "DataSkin Property
 Sheet" objects to the ZClass.  But the basic principle would still apply.

We've come to the conclusion that ZClasses really are more a hinderance
than a help, trading functionality for shorter learning curve. With
straight python code you get much more control and the ability to use
conventional editors and tools (cvs), without losing anything besides a bit
of time figuring things out (which is better in the long run anyways).

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] AttributeProvider and type information

2000-10-11 Thread John Eikenberry


I just want to ask a couple quick questions...

First, when using the AttributeProvider class you are pretty much required
to subclass it to add methods for altering the values of the attributes.
Basically I want to be sure I'm not missing something criticle as there
don't seem to be any methods on the default dataskin class for this (as
there are for propertysheets)... and it seems like there should be.

Related to this is dealing with typing information like PropertySheets used
(via the _properties attribute). Wouldn't make sense to keep the
AttributeProvider consistent with this... instead of the simple Attributes
list? Seems like the type information and other meta information would as 
useful for handling attributes on the dataskin as for propertysheets.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] DataSkins containing DataSkins

2000-11-06 Thread John Eikenberry


I have an idea for getting around my ConflictError problem without a major
redesign. But it involves DataSkins containing DataSkins. I have a few
questions about this...

The base DataSkin will be in a Rack. Would there be any issues in making
the embedded DataSkins (contained on the base DataSkins) use this Rack as
well?  Seems like it might result in the confusion about which DataSkin a
PlugIn on the Rack worked with.

Would it be better to use Customizers in some way... either have the base
Dataskin or perhaps the Specialist also inherit from the Customizer?

Phillip, I think you've considered this (didn't you once mention converting
PropertySheets to DataSkins). What issues do you think I should be aware of
in planning for this.

I tried to dig around in the list archives, but I couldn't come up with a
search pattern with any applicable hits. So I'm sorry if I'm reiterating
something already discussed.

Thanks in advance for any help...

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] Custom Unauthorized errors...

2000-11-30 Thread John Eikenberry


With all the work going on related to Zope's authentication system has
there been any progress on allowing custom unauthorized errors? I'd refer
to the bug id in the collector, but it seems to be down at the moment.

I'm planning on using LoginManager which fixes this to a certain extent
last I heard... that is it will let you use a customized unauthorized error
when you know who the user is (ie. they've logged in), but not for
anonymous users.

Just out of curiousity (I'm looking at the source now to figure this out,
but I thought it couldn't hurt to ask), why is this? The anonymous user has
a user object, and so would seem to be a real user as far as the system
goes.

I'm tired of hacking custom unauthorized errors into zope all the time, so
I'd really like to get this fixed at some point. Any thoughts as to
directions to go, or dead ends to be avoided. I'll be looking into this as
I find time so any help would be appreciated.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] ZPatterns scalability...

2000-12-05 Thread John Eikenberry


Just to get more opinions... we have a service coming up where we'd like to
use a Zope/Zpatters solution. Its a feedback tool, with the potential of up
to 50,000 entries. These would be stored in a Specialist/Rack/DataSkin
combo, stored in a standard zodb/filestorage system. Using a ZCatalog for
listings and the Specialist.getItem() for individual record retrieval. Any
thoughts on scalability issues.

Thanks in advance for any insights.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] partial search performance problems...

2000-12-20 Thread John Eikenberry


I just did some tests on the performance of the partial matching features
of ZCatalog. They weren't as good as I hoped.

A Catalog of 2000 objects, text index on a 10 char string (person's last
name), with a partial search that will match only 200 items... it takes 10
seconds. At 1000 matches it takes over a minute. And Zope is inaccessible
this entire time. 

This is after I upgraded to Python-2.0 for the new re module (which helped)
and messing with interpreter check interval (which didn't really help).

Are there any other tricks to get this to perform better? Aside from
switching to a relational database.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




[Zope-dev] LoginManger: multiple LoginMethods

2001-01-07 Thread John Eikenberry


Just got a basic loginmanager setup working with an SQL backend based off
the howto. Very simple once I figured out a couple things (thanks dlpierson
whoever you are).

Anyways... I want to be able to support both cookie based auth and basic
auth. With cookie based auth being the default unless they don't have
cookies (either because they have them turned off, behind a proxy filter,
etc), in which case they should use basic auth. The problem is I'm not sure
how to get it to use something other than cookie based auth. The default
setup tries to use cookie auth whether or not cookies are in use.

Any tips on how to get this working? I know there is no great way of
detecting cookies, besides setting one and testing for it. But even if I
did this, how do I say I want basic auth if the test cookie isn't found.

Thanks in advance for any tips/advice.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Massive scalability

2001-01-16 Thread John Eikenberry

Michael Bernstein wrote:

 So, again: Has anyone run up against any performance or
 other limitations regarding large numbers (hundreds of
 thousands or more) of objects stored within the ZODB either
 in a BTree Folder or a Rack?

I was looking into the same issues recently, but for a much smaller set of
data (5ish). In my tests ZPatterns/binary-trees scaled well for storage
and retrieval. But ZCatalog did not. It was basically useless for partial
matching searches (taking many minutes for searches that retrieved more
than 100 matches). I was also concerned about the indexing overhead. It
doesn't scale well when changing/adding many things at a time (we might
have bulk adds/changes).

I ended up deciding to go with a RDBMS backend for data storage with a
ZPatterns interface. SkinScripts work so well for this that I'm actually
glad I switched. It simplified my design and implementation immensely. 

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Massive scalability

2001-01-16 Thread John Eikenberry

Andy McKay wrote:

 Does ZPatterns provide a nice interface / way for storing classes in a
 RDBMS? I have to say using an RDBMS is not as transparent as I would like,
 this may may improve it. Finally a reason for me to ZPatterns...
 
The best way to get a taste is to try it out. The easiest way to do this is
to install LoginManager plus the DB/DA. Then follow the instructions in...

LoginManager with SQL and Skinscript
http://www.zope.org/Members/dlpierson/sqlLogin

I had it up and running pretty quickly following these instructions.

More generally the combo of sql-methods and ZPatterns (ie. skinscripts)
seems pretty compelling. I've used SQLMethods pretty extensively for the
past 18 months.  They have definate limitations which is why I'm working on
the object based system. The ZPatterns abstraction seems to provide the
best of both worlds.  You get the nice parts of SQLMethods; timed cache,
dtml query syntax and web viewable sql querries, plus you get a nice object
abstractaction (much better than plugable brains). SkinScripts allow for
easy attribute and trigger handling, basically like a simple object
description language.

If you can't tell, I'm pretty sold on ZPatterns. And once deciding that an
RDBMS was the best way to go for data storage, it fit into the 'pattern'
very nicely. I haven't deployed it yet, but its pretty fun to work on. :)

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Massive scalability

2001-01-16 Thread John Eikenberry

Michael Bernstein wrote:

 John Eikenberry wrote:
  
  I was looking into the same issues recently, but for a much smaller set of
  data (5ish). In my tests ZPatterns/binary-trees scaled well for storage
  and retrieval. But ZCatalog did not. It was basically useless for partial
  matching searches (taking many minutes for searches that retrieved more
  than 100 matches)
 
 Was this true even for cases where the batch size was
 smaller than 100? For example, if a search returns over 100
 results but the batch size is only 20 (so that only 20
 results at a time are displayed), do you still get the
 performance hit?

Short answer: yes 

Long answer: If you check out the source and/or hit it with the profiler
you'll see that the way the partial search works is to first do a more
general search then to limit the hits as much as possible via regex's.
Both these steps have to happen no matter the batch size, and this is where
you take the performance hit.  

  [snip]
  I ended up deciding to go with a RDBMS backend for data storage with a
  ZPatterns interface. SkinScripts work so well for this that I'm actually
  glad I switched. It simplified my design and implementation immensely.
 
 So you're saying that you are doing all searching using SQL
 statements, and not just object retreival and storage,
 correct? How are you handling full text searches?

Yes. I'll use MySQL's built in pattern matching facilities. It can do full
text searches with partial matching, and it can do this fast.  I'm working
on a system that will return the DataSkin's in responce to the query.
Allowing me to deal with just the objects yet use all of MySQL's
facilities. 

I'v just started to work on this as part of a larger project, but I'm doing
it full time and should have something fairly soon. My company is very free
software friendly, so I'll be able to share it once its ready. If you
happen to be interested.

-- 

John Eikenberry
[[EMAIL PROTECTED] - http://zhar.net]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] FastCGI question

2001-03-25 Thread John Eikenberry


I believe you can stop this just by changing the PCGI_PUBLISHER variable in
your Zope.cgi to a directory, /dev/null or something.

We do this for all our site, using the mod_pcgi apache module. It works
fairly well... at least well enough for us not to consider changing yet.

Michael Olivier wrote:

 At 07:33 PM 3/23/2001 -0800, sam gendler wrote:
 I haven't used Zope to any significant degree in over a year.  When I
 was last using Zope regularly, FastCGI still had some stability issues,
 which made it difficult to use in a production environment. However, I
 have always felt that FastCGI was the better solution when choosing
 between proxy, pcgi and fastcgi, from a theoretical standpoint.  Can
 someone comment as to the efficacy of using FastCGI to implement an
 interface between my HTTP enabled application and current version of
 Zope, or should I use one of the other solutions.
 
 Hi Sam and others --
 
 I was about to ask the same question myself.  I'm also wondering about 
 switching solutions to FastCGI or proxy.  I read on the zope web site that 
 zope.org is using the proxy approach.  Can someone in-the-know please shed 
 some light?
 
 I'm using PCGI (2.0a4) with ZPublisher and while it's worked reasonably 
 well for very small traffic levels, I have recently run into an issue where 
 lots of extra pcgi_publisher.py processes get started until the system 
 grinds to a halt.  I found two old postings from others (John Eikenberry, 
 Craig Allen) about this problem, including this one:
 
 http://classic.zope.org/pipermail/zope/1999-April/003926.html
 
 ... and am wondering if it's been solved and if there's a later release 
 (Zope 2.3 stable has pcgi 2.0a4 still).
 
 Thanks!
 Michael
 

-- 

John Eikenberry [[EMAIL PROTECTED]]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] (ZPatterns) Speeding up Skinscripts

2001-03-29 Thread John Eikenberry


We have a fairly large and complex app framework built on ZPatterns. It
uses MySQL for storage and the standard Specialist/Rack/DataSkin setup with
skinscripts for attributes and triggers.

We've found that the speed of getItem is a bit slower than we need. For
instance retrieving 200 dataskins takes about 8 seconds on a P2-300. After
profiling and digging around I think I've found the primary bottleneck. Its the
running of eval() on the skinscript's python expression (stored in the Compute
class as _fromex and Triggers as callexpr).

Note that this becomes the bottleneck after the SQL Method gets cached. The
query to the DB takes the most time on the first hit, but after its been cached
it takes very little time.

The optimization I've been looking at is changing the code from storing a
string and eval()ing the string to using compile() at save time and exec() when
evaluated.

Profiling these 2 ways in little test programs seems to indicate about a 2.5x
speedup. Not huge, but combined with better hardware should be enough.

But I'm curious why this avenue wasn't taken to begin with. Seems like the way
to do it to me. Am I missing something? 

-- 

John Eikenberry [[EMAIL PROTECTED]]
__
"A society that will trade a little liberty for a little order
 will deserve neither and lose both."
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] default roles in ac_permissions

2001-06-21 Thread John Eikenberry


Note: I'm still using Zope 2.2.5, which is why I'm forced to use
__ac_permissions__ and not the new security code in 2.3.x.

In the documentation I've found on how __ac_permissions__ works you are
supposed to be able to set default roles for a permission in a 3rd tuple.
I've tried various things and can never seem to get them to take effect.
The acquired permissions always take precedence.

The only thing I can think of to do is create a method that I can call to
set them using the builtin methods (eg. manage_changePermissions).

Has anyone had any sucess with default roles in the __ac_permissions__? If
so how, if not what did you do to automatically set default roles for a
product?

Thanks in advance for any info.

-- 

John Eikenberry [[EMAIL PROTECTED]]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Session Errors

2003-03-13 Thread John Eikenberry

Sorry, its Zope 2.6.1.

Chris McDonough wrote:

 John,
 
 Which Zope 2.6?  Zope 2.6.1?  Here's what line 807 of the current
 Transience.py looks like:
 
 v = self._data[b].get(k, notfound)
 
 Does yours look like that?

Yes.

 What is the value of the __version__ variable at the top of the
 Transience.py file?

__version__='$Revision: 1.28.6.4 $'[11:-2]

 
 On Thu, 2003-03-13 at 07:11, John Eikenberry wrote:
  Since upgrading to Zope-2.6 we've been getting KeyErrors when using
  Sessions. They seem to happen more now that we've started using
  hasSessionData(), but I'm pretty sure they happened prior to that.
  
  Anyways, here are the 2 related tracebacks. Has anyone else seen these?
  
  Traceback #1 occurs most frequently. The KeyError's value is an unprintable
  string of non-ascii characters.
  
  * Module ZPublisher.Publish, line 150, in publish_module
  * Module ZPublisher.Publish, line 114, in publish
  * Module The application server.App.startup, line 182, in
zpublisher_exception_hook
  * Module ZPublisher.Publish, line 98, in publish
  * Module ZPublisher.mapply, line 88, in mapply
  * Module ZPublisher.Publish, line 39, in call_object
  * Module App.special_dtml, line 61, in __call__
  * Module DocumentTemplate.DT_String, line 474, in __call__
  * Module Products.Transience.Transience, line 342, in nudge
  * Module Products.Transience.Transience, line 467, in _getCurrentBucket
  * Module Products.TemporaryFolder.LowConflictConnection, line 34, in
setstate
  * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load
  KeyError:
  
  Traceback #2 happens less frequently, though today it seemed like it was
  trying to catch up (3 of these today).
  
  * Module ZPublisher.Publish, line 98, in publish
  * Module ZPublisher.mapply, line 88, in mapply
  * Module ZPublisher.Publish, line 39, in call_object
  * Module OFS.DTMLMethod, line 126, in __call__
  * Module DocumentTemplate.DT_String, line 474, in __call__
  * Module Products.DotOrg.Pages.KContent, line 149, in __call__
  * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo
  * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData
  * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData
  * Module Products.Sessions.SessionDataManager, line 175, in
_hasSessionDataObject
  * Module Products.Transience.Transience, line 838, in has_key
  * Module Products.Transience.Transience, line 807, in get
  
  KeyError: 1047409860 
  
  
  -- 
  
  John Eikenberry [EMAIL PROTECTED]
  __
  A society that will trade a little liberty for a little order
   will deserve neither and lose both.
--B. Franklin
  
  ___
  Zope-Dev maillist  -  [EMAIL PROTECTED]
  http://mail.zope.org/mailman/listinfo/zope-dev
  **  No cross posts or HTML encoding!  **
  (Related lists - 
   http://mail.zope.org/mailman/listinfo/zope-announce
   http://mail.zope.org/mailman/listinfo/zope )
 
 

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors

2003-03-13 Thread John Eikenberry

Patch applied and the first results are in... so far its a lot of these:


2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get,
checking _data BTree
--
2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded


Chris McDonough wrote:

 Hi John,
 
 Can you apply the attached diff to your Transience.py file and run with
 it in place for a couple of days?  It will not fix the problem (the
 symptoms will remain) but it should print some diagnostic information to
 the Zope event log (the STUPID_LOG_FILE, hopefully you've got that
 going) that will help us track down what this might be.
 
 Once you notice it happen, send the relevant parts of your logfile to me
 and I will see if I can analyze it.
 
 - C
 
 
 
 
 On Thu, 2003-03-13 at 15:19, John Eikenberry wrote:
  
  Sorry, its Zope 2.6.1.
  
  Chris McDonough wrote:
  
   John,
   
   Which Zope 2.6?  Zope 2.6.1?  Here's what line 807 of the current
   Transience.py looks like:
   
   v = self._data[b].get(k, notfound)
   
   Does yours look like that?
  
  Yes.
  
   What is the value of the __version__ variable at the top of the
   Transience.py file?
  
  __version__='$Revision: 1.28.6.4 $'[11:-2]
  
   
   On Thu, 2003-03-13 at 07:11, John Eikenberry wrote:
Since upgrading to Zope-2.6 we've been getting KeyErrors when using
Sessions. They seem to happen more now that we've started using
hasSessionData(), but I'm pretty sure they happened prior to that.

Anyways, here are the 2 related tracebacks. Has anyone else seen these?

Traceback #1 occurs most frequently. The KeyError's value is an unprintable
string of non-ascii characters.

* Module ZPublisher.Publish, line 150, in publish_module
* Module ZPublisher.Publish, line 114, in publish
* Module The application server.App.startup, line 182, in
  zpublisher_exception_hook
* Module ZPublisher.Publish, line 98, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module App.special_dtml, line 61, in __call__
* Module DocumentTemplate.DT_String, line 474, in __call__
* Module Products.Transience.Transience, line 342, in nudge
* Module Products.Transience.Transience, line 467, in _getCurrentBucket
* Module Products.TemporaryFolder.LowConflictConnection, line 34, in
  setstate
* Module Products.TemporaryFolder.TemporaryStorage, line 94, in load
KeyError:

Traceback #2 happens less frequently, though today it seemed like it was
trying to catch up (3 of these today).

* Module ZPublisher.Publish, line 98, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module OFS.DTMLMethod, line 126, in __call__
* Module DocumentTemplate.DT_String, line 474, in __call__
* Module Products.DotOrg.Pages.KContent, line 149, in __call__
* Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo
* Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData
* Module Products.Sessions.SessionDataManager, line 101, in hasSessionData
* Module Products.Sessions.SessionDataManager, line 175, in
  _hasSessionDataObject
* Module Products.Transience.Transience, line 838, in has_key
* Module Products.Transience.Transience, line 807, in get

KeyError: 1047409860 


-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )
   
   
  
  -- 
  
  John Eikenberry [EMAIL PROTECTED]
  __
  A society that will trade a little liberty for a little order
   will deserve neither and lose both.
--B. Franklin
  
  ___
  Zope-Dev maillist  -  [EMAIL PROTECTED]
  http://mail.zope.org/mailman/listinfo/zope-dev
  **  No cross posts or HTML encoding!  **
  (Related lists - 
   http://mail.zope.org/mailman/listinfo/zope-announce
   http://mail.zope.org/mailman/listinfo/zope )
 

 ? btreecheck.diff
 ? kedaipatch
 Index: Transience.py
 ===
 RCS file: /cvs-repository/Zope/lib/python/Products/Transience/Transience.py,v
 retrieving revision 1.28.6.4
 diff -r1.28.6.4 Transience.py
 34a35
  from BTrees.check import check, display
 45a47
  from cStringIO import

Re: [Zope-dev] Session Errors

2003-03-14 Thread John Eikenberry
Chris McDonough wrote:

 OK, thanks John.

Thank you for helping.
 
 I hate to ask this (I should have done this to start with), but would
 you be willing to use the following patch --against the original file,
 not your recently patched version-- and try again?  I only checked one
 of the two BTrees that might be at the heart of the problem with the
 first patch, this patch checks the second as well.

Put the patch in place and have a couple errors already. Doesn't look like
its going to be much help though. Just a couple repetitions of this:

--
2003-03-14T03:35:53 PROBLEM(100) Transience KeyError raised in get,
checking BTrees
--
2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for data succeeded
--
2003-03-14T03:35:53 PROBLEM(100) Transience BTree check for index succeeded



 - C
 
 
 On Thu, 2003-03-13 at 18:18, John Eikenberry wrote:
  
  Patch applied and the first results are in... so far its a lot of these:
  
  
  2003-03-13T15:18:07 PROBLEM(100) Transience KeyError raised in get,
  checking _data BTree
  --
  2003-03-13T15:18:07 PROBLEM(100) Transience BTree check succeeded
  
  
  Chris McDonough wrote:
  
   Hi John,
   
   Can you apply the attached diff to your Transience.py file and run with
   it in place for a couple of days?  It will not fix the problem (the
   symptoms will remain) but it should print some diagnostic information to
   the Zope event log (the STUPID_LOG_FILE, hopefully you've got that
   going) that will help us track down what this might be.
   
   Once you notice it happen, send the relevant parts of your logfile to me
   and I will see if I can analyze it.
   
   - C
   
   
   
   
   On Thu, 2003-03-13 at 15:19, John Eikenberry wrote:

Sorry, its Zope 2.6.1.

Chris McDonough wrote:

 John,
 
 Which Zope 2.6?  Zope 2.6.1?  Here's what line 807 of the current
 Transience.py looks like:
 
 v = self._data[b].get(k, notfound)
 
 Does yours look like that?

Yes.

 What is the value of the __version__ variable at the top of the
 Transience.py file?

__version__='$Revision: 1.28.6.4 $'[11:-2]

 
 On Thu, 2003-03-13 at 07:11, John Eikenberry wrote:
  Since upgrading to Zope-2.6 we've been getting KeyErrors when using
  Sessions. They seem to happen more now that we've started using
  hasSessionData(), but I'm pretty sure they happened prior to that.
  
  Anyways, here are the 2 related tracebacks. Has anyone else seen these?
  
  Traceback #1 occurs most frequently. The KeyError's value is an unprintable
  string of non-ascii characters.
  
  * Module ZPublisher.Publish, line 150, in publish_module
  * Module ZPublisher.Publish, line 114, in publish
  * Module The application server.App.startup, line 182, in
zpublisher_exception_hook
  * Module ZPublisher.Publish, line 98, in publish
  * Module ZPublisher.mapply, line 88, in mapply
  * Module ZPublisher.Publish, line 39, in call_object
  * Module App.special_dtml, line 61, in __call__
  * Module DocumentTemplate.DT_String, line 474, in __call__
  * Module Products.Transience.Transience, line 342, in nudge
  * Module Products.Transience.Transience, line 467, in _getCurrentBucket
  * Module Products.TemporaryFolder.LowConflictConnection, line 34, in
setstate
  * Module Products.TemporaryFolder.TemporaryStorage, line 94, in load
  KeyError:
  
  Traceback #2 happens less frequently, though today it seemed like it was
  trying to catch up (3 of these today).
  
  * Module ZPublisher.Publish, line 98, in publish
  * Module ZPublisher.mapply, line 88, in mapply
  * Module ZPublisher.Publish, line 39, in call_object
  * Module OFS.DTMLMethod, line 126, in __call__
  * Module DocumentTemplate.DT_String, line 474, in __call__
  * Module Products.DotOrg.Pages.KContent, line 149, in __call__
  * Module Products.DotOrg.Pages.KContent, line 194, in getEditInfo
  * Module Products.DotOrg.Pages.KContent, line 506, in hasSessionData
  * Module Products.Sessions.SessionDataManager, line 101, in hasSessionData
  * Module Products.Sessions.SessionDataManager, line 175, in
_hasSessionDataObject
  * Module Products.Transience.Transience, line 838, in has_key
  * Module Products.Transience.Transience, line 807, in get
  
  KeyError: 1047409860 
  
  
  -- 
  
  John Eikenberry [EMAIL PROTECTED]
  __
  A society that will trade a little liberty for a little order
   will deserve neither and lose both.
--B. Franklin
  
  ___
  Zope-Dev maillist  -  [EMAIL PROTECTED]
  http://mail.zope.org/mailman/listinfo/zope-dev
  **  No cross posts

Re: [Zope-dev] Session Errors

2003-03-14 Thread John Eikenberry

Sorry for the length of this one... but I'm trying to braindump to give you
as much info about the problem as possible. 

To be sure it doesn't get lost in my below ramblings, there is probably
important peice of information I haven't mentioned yet... that is that
these errors seem to coincide with the session data timeout setting [1]. I
don't get the errors at all until the timeout is reached or has passed.

[1] The timeout setting I'm refering to is denoted by the label: Data
object timeout value in minutes on the /temp_folder/session_data object.


Chris McDonough wrote:

 OK, thanks John.  Let's try one more thing... currently the mounted
 database used to store the session data uses a connection that ignores
 read conflicts.  This is known to be bad because the machinery which
 deals with keeping the sessioning index data will also ignore read
 conflicts, which may create inconcstencies between two data structures
 (BTrees) that need to be kept in sync.

I tried this and it seemed to help some. I haven't seen the get() error
we've been dicussing yet, but a the load() error just occurred (line 94 in
TemporaryStorage - this was error #1 in my original email). Though the
traceback is a bit different from my original email, as the
LowConflictConnection isn't being used. Here's the new Traceback:

Error Type: KeyError
Error Value: [non-ascii chars]

Traceback (innermost last):

* Module ZPublisher.Publish, line 98, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module Products.DotOrg.Pages.KPage, line 110, in testSession
* Module Products.DotOrg.Utils.Spawn, line 42, in launchProcess
* Module Products.DotOrg.Utils.Spawn, line 73, in storeArgs
* Module Products.Sessions.SessionDataManager, line 180, in
* _getSessionDataObject
* Module Products.Transience.Transience, line 175, in new_or_existing
* Module Products.Transience.Transience, line 797, in get
* Module Products.Transience.Transience, line 546, in _getCurrentBucket
* Module ZODB.Connection, line 509, in setstate
* Module Products.TemporaryFolder.TemporaryStorage, line 94, in load


 Here's a patch to lib/python/Products/TemporaryFolder/TemporaryFolder.py
 that reenables read conflict generation on the database.
 
 Index: TemporaryFolder.py
 ===
 RCS file:
 /cvs-repository/Zope/lib/python/Products/TemporaryFolder/TemporaryFolder.py,v
 retrieving revision 1.7
 diff -r1.7 TemporaryFolder.py
 72c72
  db.klass = LowConflictConnection
 ---
  #db.klass = LowConflictConnection
 
 You may see many more conflicts with this running.  But maybe the data
 structures will not become desynchronized.

You weren't kidding about the increase in conflict errors.
 
 Another problem, still unexplained, experienced by Andrew Athan, is that
 if a reference is made to a session data object from within the standard
 error message, somehow things get screwy under high load.  If you're
 doing the same, please let me know.

Before this started happening there was a hasSessionData check getting
called during standard error publishing, though we removed that early this
week when this started happening.

---

It might help you to better understand what might be causing the problem if
you know where we're using sessions and how we can force this problem to
occur. Not sure if this willl be of much help, but thought it couldn't
hurt.

We use sessions primarily as a sort of authenticated user marker. It just
stored their username and a state field that get used in non-authenticated
sections of our site to detect the user as having logged into the site (we
can then raise an unautorized error to get the basic auth info for that
user). Anyways, these calls happen on our basic Content class (subclassed
from DTMLMethod) in its __call__() method. We use it a couple other places
for small things, but this one sees the most use.

I've figured out how to force these errors to happen to some extent. I've
written a method that starts up a thread, which uses Client.call to call
another method, which then basically just loops endlessly calling
hasSessionData and getSessionData, incrementing a number in the session
data and sleeping for a N number of seconds between loops. One of these
guys will run forever without a problem.

Once you start a second thread ReadConflictErrors start getting raised.
Which thread gets the conflict and which one keeps working seems variable
(probably just a timing thing). If I start enough of these threads I can
cause the error to happen. But only once the session timeout is reached.

Note that to help speed up getting the errors I either set the session time
to 1 minute via _setTimeout() call or even manually tweak the appropriate
session data managers attributes (_timeout_secs, _period and
_timeout_slices) to very small values (ie. a few seconds).


-- 

John Eikenberry

Re: [Zope-dev] Session Errors

2003-03-14 Thread John Eikenberry
John Eikenberry wrote:

 Once you start a second thread ReadConflictErrors start getting raised.
 Which thread gets the conflict and which one keeps working seems variable
 (probably just a timing thing). If I start enough of these threads I can
 cause the error to happen. But only once the session timeout is reached.

Of course, once I finally post this I notice that the behaviour is not
really this way. If the sleep delay is set higher and in a way such that
the 2 threads won't run at the same time very often, they will both run
fine. Its only if they try to access the sessions at the same time that the
ReadConflictError gets raised.

Oh, and in my threading code I catch/print/ignore these errors so my
threads will keep going. The threads also do a full commit each time
through (to help better simulate actual usage).
 

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors

2003-03-17 Thread John Eikenberry
Toby Dickenson wrote:

 Read conflicts occur if a change is committed in between the start of a 
 transaction, and the transaction needing to load the object. A workaround to 
 reduce the number of read conflicts is to touch the objects that are likely 
 to change early in the transaction.

Thanks for the tip. My sessions test script had a sleep() call right after
the transaction commit, thus making it right at the beginning of the next
transaction. Moving it to right before the commit (after all the session
tests) pretty much eliminated all the read conflict errors. Just get a
standard ConflictError occasionally now (not nearly as often).

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors

2003-03-17 Thread John Eikenberry
John Eikenberry wrote:

 Toby Dickenson wrote:
 
  Read conflicts occur if a change is committed in between the start of a 
  transaction, and the transaction needing to load the object. A workaround to 
  reduce the number of read conflicts is to touch the objects that are likely 
  to change early in the transaction.
 
 Thanks for the tip. My sessions test script had a sleep() call right after
 the transaction commit, thus making it right at the beginning of the next
 transaction. Moving it to right before the commit (after all the session
 tests) pretty much eliminated all the read conflict errors. Just get a
 standard ConflictError occasionally now (not nearly as often).

This turned out to be better than I thought. After nearly 3 hours of
constant abuse I have yet to see a ReadConflictError and have yet to get
either the load() or get() KeyErrors, previously I could force one of these
in a 2-3 minutes of abuse.

Our live sessions code uses the sessions about half to two-thirds of the
way through the transaction. Given what can happen in that first half,
there is easily plenty of time for read conflicts. I think I might be able
to move our session use to the beginning of the transaction (just storing
stuff in the REQUEST object for later usage) and hopefully fix our issue.

The KeyErrors happen under similar circumstances to the ReadConflictErrors.
The significant difference being that the KeyErrors happen after the
transience timeout has occured. When I am running with the
LowConflictConnection disabled the KeyErrors occur in the
Connection.setstate method at line 509, before the ReadConflictError check.

So say you have 2 threads; A and B.  If A starts first, hits the session
code and triggers a _housekeep() call during which time B has started but
has not reached the Sessions code until after _housekeep() has finished.
When it does reach the sessions code, you get the KeyErrors. 

Sound reasonable?

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors (read conflicts)

2003-03-18 Thread John Eikenberry
Chris McDonough wrote:

 On Mon, 2003-03-17 at 20:34, John Eikenberry wrote:
  The KeyErrors happen under similar circumstances to the ReadConflictErrors.
  The significant difference being that the KeyErrors happen after the
  transience timeout has occured. When I am running with the
  LowConflictConnection disabled the KeyErrors occur in the
  Connection.setstate method at line 509, before the ReadConflictError check.
 
 Is this the same KeyError you reported as coming out of
 TemporaryStorage.load?

Yes. I'm referring to the traceback I reported in a previous mail:

http://mail.zope.org/pipermail/zope-dev/2003-March/019118.html

The error occurs on the same line in TemporaryStorage.load(), whether it gets
called from ZODB/Connection.py or TemporaryFolder/LowConflictConnection.py.
 
  So say you have 2 threads; A and B.  If A starts first, hits the session
  code and triggers a _housekeep() call during which time B has started but
  has not reached the Sessions code until after _housekeep() has finished.
  When it does reach the sessions code, you get the KeyErrors. 
 
 Would you mind restating that?  I think this is important, and I'm not
 sure I understand the second sentence above.

I'm working off the idea that the load() KeyErrors went away when the
ReadConflictErrors did. So it seemed like they were probably triggered by
similar scenarios. The difference being that for the former, the timeout had
been reached and the _housekeep() code had been run (and possibly some
additional code in _getCurrentBucket).

Now given Toby's description of how ReadConflicts occur [1], it seems that
instead of a change being committed to the object it is instead deleted (in
_housekeep). Thus instead of getting the object and seeing it is marked as
invalid, it cannot get the object at all when it expects to be able to...
resulting in the KeyError in load().



[1] Read conflicts occur if a change is committed in between the start of a
transaction, and the transaction needing to load the object. A workaround to
reduce the number of read conflicts is to touch the objects that are likely
to change early in the transaction.

-- 

John Eikenberry [EMAIL PROTECTED]
__
A society that will trade a little liberty for a little order
 will deserve neither and lose both.
  --B. Franklin

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )