On 5/28/13 7:53 PM, Dylan Jay wrote:

My colleague is having the issue outlined below. I thought this list might be better to give an insight as to what is going on?

Begin forwarded message:

*From: *Adam Terrey <a...@pretaweb.com <mailto:a...@pretaweb.com>>
*Subject: **Fwd: Zope/Plone/Zeo Concurrency Issue*
*Date: *28 May 2013 5:23:33 PM AEST
*To: *Dylan Jay <d...@pretaweb.com <mailto:d...@pretaweb.com>>

-------- Original Message --------
Subject:        Zope/Plone/Zeo Concurrency Issue
Date:   Tue, 28 May 2013 16:04:21 +1000
From:   Adam Terrey <a...@pretaweb.com>
To:     z...@zope.org


Can someone offer some insight into what might be going on here and perhaps how I can debug the following issue?

In Plone there is a request patten used to create content which looks like...

  * (A) GET request to /MySite/createObject?type_name=Document
    Responds with a redirect to a tempory document at a location such
  * (B) GET
    Responds with from to set fields in the Document
  * (C) POST request to
    Responds with a redirect to the final location of the document
    such as /Mysite/my-page
  * (D) GET /Mysite/my-page
    Responds with the newly created page

In one of our production systems we are running multiple Zope/Plone instances connecting to a Zeo server. And the above patten works about 95% of the time. However, sometimes request (D) will respond with 404 Page Not Found. My assumptions is that request (C) and (D) are going to different instances and somehow the instance handling request (D) does not yet see the transaction with completed the page creation.

I've checked that the transaction commit happens before response headers are emitted for request (C) - suspecting the case that the browser handles the redirect before the transaction is completed - but this is clearly not the case from what i can see. The transaction is well committed before response headers are returned.

Versions are as follows:

  * Plone 4.1.3 (4112)
  * CMF 2.2.4
  * Zope 2.13.10
  * Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1]

I have been using the following script to recreate the issue. I seem to be able to produce the error quicker if I put server2 under apache-bench load, and I have been able to recreate the issue on a local copy of our production system but it is a far more rare case.

auth = ('admin', 'admin')
server1 = 'http://localhost:46101'
server2 = 'http://localhost:46102'
site = '/Plone'

import requests

ses = requests.Session()

for i in range(10):

    # Create tempory Object
    create_url = server1 + site + '/createObject?type_name=Document'
    print "GET", create_url
    res = ses.get (create_url)
    tempory_obj = res.url.rsplit("/", 1)[0]
    print "tempory object:", tempory_obj

    # Submit data to create object (response will
    # be the redirect ot the ojbect's location)

    data = {
                 'id': '',
                 'title': 'page' + str(i)   ,
                 'form.button.save': 'Save',
                 'form.submitted': '1',
                 "description": "some secriot",
                 "text": "some text"

    post_url = tempory_obj + "/atct_edit"
    print "POST", post_url
    res = ses.post(post_url, data=data, allow_redirects=False)

    redirect_location = "/" + res.headers['location'][7:].split('/',1)[1]
    print 'redirect_location', redirect_location

    # Request object from the redirect from server2
    redirect_url = server2 + redirect_location
    print "GET", redirect_url
    res = ses.get(redirect_url)

    if res.status_code != 200:
        last404 = res
        print res;

    print "---"

Any help would be appreciated.

I've encountered a problem like this once or twice and had some trouble reproducing it consistently, but assume it's due to ZEO invalidation messages not making it to the other instance before it serves request D.

You can work around the issue using sticky sessions in your load balancer. If you don't want users to be stuck to a particular backend indefinitely you can set a cookie with a brief expiration in an ObjectAddedEvent handler and use it as the basis for the stickiness.

For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to