Re: [Zope] Zope vs Plone: performance issues!

2006-01-25 Thread Dario Lopez-Kästen

Hugo Ramos wrote:

Well... They were already using Zope's cache and Squid to proxy some static HTML
I guess the problem is Plone itself... Skins etc...



eh, sorry for probably stating the obvious. you menationed in an earlier 
mail that you have good hardware. If this hardware is multi-cpu then I 
suggest that you use some tool (i.e. taskset on linux) to bind each zope 
python process to a single cpu.


Threaded python applications does not work well in multi-cpu settings 
where there is a chance of a threaded python process being spread out 
over more than one CPU. I causes a, sometimes signinficant, performance 
loss.


Just by binding the zope python processes to a single CPU (not 
necessarily the same, just make sure that any single python process is 
not spread over more than one CPU) you will notice a preformance increase.


And like all the others said, Caching is you friend. Start by checking 
the ZODB cache numbers in the zope-conf file (too start simple). IIRC, 
the default is 5000 objects. Increase that to, say 50.000 or so (you 
mention that you have a busy site).


Good luck.

/dario

--
-- ---
Dario Lopez-Kästen, IT Systems  Services Chalmers University of Tech.
Lyrics applied to programming  application design:
emancipate yourself from mental slavery - redemption song, b. marley

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread bruno desthuilliers
Hugo Ramos wrote:
 Hi all,
 
 I'm responsible for a big corporate intranet and corporate portal.
 They were using Zope/Plone before I got here and the problems are too many.
 
 Let's talk about performance issues... Plone is VERY SLOW responding
 requests. All content is stored in ZODB objects and zcatalog is used
 to search. But even if you request the intranet home only, it takes
 about 10 seconds to load. The hardware is very good and lots of ram is
 used in the servers.
 
 They also have zeo + 2 zope instances running in the same machine.
 I've never seen this in my life... only 3 machines running 2 zope
 instances + zeo instance, separately.
 
 Do you think Zope(logic)+MySQL(content)+Apache(HTTP server) is better
 solution than Zope+Plone ???

I think the problem has more to do with Plone than with Zope or the
ZODB. We have a couple 'barebones' zope applications here that serve
public website. Hosting is on a shared server (behind apache) with no
zeo, no cache, no tuning, no fancy configuration, and performances are
pretty good - even with search. For the record, we've tried Plone on the
same server, and even with almost no content, it *is* *very* slow.

-- 
bruno desthuilliers
développeur
[EMAIL PROTECTED]
http://www.modulix.com
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Chris Withers

bruno desthuilliers wrote:

I think the problem has more to do with Plone than with Zope or the
ZODB. We have a couple 'barebones' zope applications here that serve
public website. Hosting is on a shared server (behind apache) with no
zeo, no cache, no tuning, no fancy configuration, and performances are
pretty good - even with search. For the record, we've tried Plone on the
same server, and even with almost no content, it *is* *very* slow.


Me? I couldn't _possibly_ comment ;-)

Chris . o O ( me? say Plohn is slow? no... never)

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Lennart Regebro
On 1/24/06, Hugo Ramos [EMAIL PROTECTED] wrote:
 Let's talk about performance issues... Plone is VERY SLOW responding
 requests. All content is stored in ZODB objects and zcatalog is used
 to search. But even if you request the intranet home only, it takes
 about 10 seconds to load. The hardware is very good and lots of ram is
 used in the servers.

 They also have zeo + 2 zope instances running in the same machine.
 I've never seen this in my life... only 3 machines running 2 zope
 instances + zeo instance, separately.

 Do you think Zope(logic)+MySQL(content)+Apache(HTTP server) is better
 solution than Zope+Plone ???

No. That problem is Plone/CMF, and most of the problem is in the
skins. It's a combination of how portal_skins work, and the fact
that Plones skins are not very optimized. CPS, who also uses
portal_skins, are faster than an empty Plone site, for example, but
still not as fast as a pure Zope-site can be.

Zope 3 and Five both promises to solve that by it's View paradigm,
but neither Plone nor CMF uses that fully yet, so if you use a lot of
standard products that probably will not help you. I'm not a speed
guru, but load balancing with ZEO would probably be the easiest way of
improving the performance in this case. That's my guess.

--
Lennart Regebro, Nuxeo http://www.nuxeo.com/
CPS Content Management http://www.cps-project.org/
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Peter Bengtsson
On 1/24/06, Hugo Ramos [EMAIL PROTECTED] wrote:
 Hi all,

 I'm responsible for a big corporate intranet and corporate portal.
 They were using Zope/Plone before I got here and the problems are too many.

 Let's talk about performance issues... Plone is VERY SLOW responding
 requests. All content is stored in ZODB objects and zcatalog is used
 to search. But even if you request the intranet home only, it takes
 about 10 seconds to load. The hardware is very good and lots of ram is
 used in the servers.

 They also have zeo + 2 zope instances running in the same machine.
 I've never seen this in my life... only 3 machines running 2 zope
 instances + zeo instance, separately.

 Do you think Zope(logic)+MySQL(content)+Apache(HTTP server) is better
 solution than Zope+Plone ???

Yes!
But not good enough.
Most performance issues with Plone can be solved by caching. Don't
underestimate it.
I know some companies that use Plone for content management but push
out the content in a very non-Plone way and cache that heavily and you
get a great result.
But, you're not going to get away from the 10 sec catalog search
problem. I doubt a ZCatalog search takes this long. Once in there it
should be possible to do a search on millons of bytes of text under 1
sec.

I have a ZCatalog instance here with roughly 55,000 cataloged
objectindices which corresponds to about 1.3Gb of text data. When the
IssueTrackerProduct does a search on this, the function
_searchCatalog() does 6 different searches on that catalog each!! and
here's a copy of some debug output
Search took 0.149714946747
Search took 0.271862030029
Search took 0.0381798744202
Search took 0.0339410305023
Search took 0.0346419811249
Search took 0.0137779712677
Search took 0.0435471534729

It's a 1Gb RAMed, Intel Mobile 1.6Mhz thinkpad laptop!!

Conclusion, ZCatalog is very fast and so is a correctly cache-headed
website with Squid.


 TIA

 --
 Hugo Ramos - [EMAIL PROTECTED]
 Senior Zope consultant
 ___
 Zope maillist  -  Zope@zope.org
 http://mail.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists -
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope-dev )



--
Peter Bengtsson,
work www.fry-it.com
home www.peterbe.com
hobby www.issuetrackerproduct.com
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Andrew Sawyers

 standard products that probably will not help you. I'm not a speed
 guru, but load balancing with ZEO would probably be the easiest way of
 improving the performance in this case. That's my guess.
 

If he's got a dual cpu'd machine and is running two zeo instances, he's
fine unless he's got a ram problem.  To be running 2 zeo instances on
the same machine, they have to be doing some type of load balancing or
faking load balancing so that ea. zeo client can respond to requests.
One thing that might help is CRANK up your on disk zeo cache as high as
you can afford - it's just disk space.  If this hasn't been done, the
zeo clients might be spending a lot of time 'requesting' objects from
the zeo storage server when you can be caching them on the zeo clients.

Some substantial effort would likely be required to optimize the Plone
installation.  If you choose this route, look at the skins - there's
lots, lots of layers, and lots of searches going on you could eliminate
or rewrite to be more efficient.  If you go down the Zope/MySQL path,
this would require some substantial effort also.  I would recommend, if
you're going to be putting substantial effort into the application, look
at a CMF/Five implementation, relying heavily on using the View paradigm
Lennart references.  Another option, not knowing your technical level,
is to implement in Zope 3.  I personally am liking Zope 3 alot.
IIRC, there is a Zope 3 CMS project out there.

Andrew Sawyers



___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Lennart Regebro
On 1/24/06, Andrew Sawyers [EMAIL PROTECTED] wrote:
 If he's got a dual cpu'd machine and is running two zeo instances, he's
 fine unless he's got a ram problem.  To be running 2 zeo instances on
 the same machine, they have to be doing some type of load balancing or
 faking load balancing so that ea. zeo client can respond to requests.

I totally forgot that he was running ZEO already. That actually makes
it even easier. since I was thinking of buying more machines. ;)
Processing power is cheap today. Buying another machine (or two) would
not cost much, and probably half the processing time (unless there is
something actually wrong).

As always, profiling is probably a good idea *before* trying to fix
performnce problems. ;)

--
Lennart Regebro, Nuxeo http://www.nuxeo.com/
CPS Content Management http://www.cps-project.org/
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Sasha Vincic
On 1/24/06, Peter Bengtsson [EMAIL PROTECTED] wrote:
 On 1/24/06, Hugo Ramos [EMAIL PROTECTED] wrote:
  Hi all,
 
  I'm responsible for a big corporate intranet and corporate portal.
  They were using Zope/Plone before I got here and the problems are too many.
 
  Let's talk about performance issues... Plone is VERY SLOW responding
 

snip

 Yes!
 But not good enough.
 Most performance issues with Plone can be solved by caching. Don't
 underestimate it.

Yes Plone is slow but with caching you get very good performance. I
have boosted performance on plone sites from default 1-3req/s to
100req/s and then it was the bandwith that was the bottleneck. The
sites where serving both anonymous and authenticated content. I
recommend to check out the CacheFu product and documentation in the
collective.

/Sasha
Lovely Systems - www.lovelysystems.com
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Dieter Maurer
Hugo Ramos wrote at 2006-1-24 11:08 -0300:
 ...
Let's talk about performance issues... Plone is VERY SLOW responding
requests. All content is stored in ZODB objects and zcatalog is used
to search. But even if you request the intranet home only, it takes
about 10 seconds to load. The hardware is very good and lots of ram is
used in the servers.

Use a profiler to find out where the time is spent (e.g. ZopeProfiler).

Maybe, your site it badly configured (running in debug mode,
ZODB cache too small, missing skin cache, ...)

-- 
Dieter
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Sasha Vincic
On 1/24/06, Hugo Ramos [EMAIL PROTECTED] wrote:
 Well... They were already using Zope's cache and Squid to proxy some static 
 HTML
 I guess the problem is Plone itself... Skins etc...

As I said and Limi, check out CacheFu. With it you can cache most of
your site not just some static HTML and you cache it with Squid so
Zope will have more time for the none cachable stuff.

Doing profile on your site is something you should always do when
optimizing. Follow Dieters recommendation and then CacheFu.
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Michael Vartanyan
How did you go about caching authenticated content?  Any simple 
personalization element like the user name in the corner disqualifies 
the whole page from effective participation in downstream 
proxying/caching. The only thing you can do with such page is to cache 
it for one particular user hoping that s/he may need the same page again 
(which is not likely).


I have an extreme situation when 90% of visitors are authenticated. 
Moreover, they participate in dozens of groups that define their access 
to individual content elements. Users are being granted and revoked 
participation in groups very actively, it is a normal situation that a 
few such changes occur within one session. And to finish the picture - 
publication and access times are important for records purposes so the 
current server time should be displayed on all pages.


I use RAM caches for parts/macros of pages that may be identical for all 
users or some of the groups. They help, but the performance is still far 
from being perfect. Any success stories/best practices for such cases?


Thanks.

zope 2.7.6, plone 2.0.5 (going to upgrade soon), freebsd 4.10


Sasha Vincic wrote:


Yes Plone is slow but with caching you get very good performance. I
have boosted performance on plone sites from default 1-3req/s to
100req/s and then it was the bandwith that was the bottleneck. The
sites where serving both anonymous and authenticated content. I
recommend to check out the CacheFu product and documentation in the
collective.

/Sasha
Lovely Systems - www.lovelysystems.com
 


___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Paul Winkler
I'm no expert in this realm, but:

Find Chris M's presentation on scaling zope, somewhere on
plope.com.  It doesn't go into great detail on this topic,
but basically, IIRC the strategy is to push personalization
(and final page composition) downstream. This generally means either:

- Squid and ESI

- client-side (javascript)

Then the parts of the page that are *not* personalized can still
be cached very effectively by squid, and you greatly reduce the load
on zope.

The JS approach may allow for browser-side caching of 
(some of) the personalization data, if it can be re-used across pages.

-PW

On Tue, Jan 24, 2006 at 11:27:39PM +0100, Michael Vartanyan wrote:
 How did you go about caching authenticated content?  Any simple 
 personalization element like the user name in the corner disqualifies 
 the whole page from effective participation in downstream 
 proxying/caching. The only thing you can do with such page is to cache 
 it for one particular user hoping that s/he may need the same page again 
 (which is not likely).
 
 I have an extreme situation when 90% of visitors are authenticated. 
 Moreover, they participate in dozens of groups that define their access 
 to individual content elements. Users are being granted and revoked 
 participation in groups very actively, it is a normal situation that a 
 few such changes occur within one session. And to finish the picture - 
 publication and access times are important for records purposes so the 
 current server time should be displayed on all pages.
 
 I use RAM caches for parts/macros of pages that may be identical for all 
 users or some of the groups. They help, but the performance is still far 
 from being perfect. Any success stories/best practices for such cases?
 
 Thanks.
 
 zope 2.7.6, plone 2.0.5 (going to upgrade soon), freebsd 4.10
 
 
 Sasha Vincic wrote:
 
 Yes Plone is slow but with caching you get very good performance. I
 have boosted performance on plone sites from default 1-3req/s to
 100req/s and then it was the bandwith that was the bottleneck. The
 sites where serving both anonymous and authenticated content. I
 recommend to check out the CacheFu product and documentation in the
 collective.
 
 /Sasha
 Lovely Systems - www.lovelysystems.com
  
 
 ___
 Zope maillist  -  Zope@zope.org
 http://mail.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )

-- 

Paul Winkler
http://www.slinkp.com
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Sasha Vincic
On 1/24/06, Michael Vartanyan [EMAIL PROTECTED] wrote:
 How did you go about caching authenticated content?  Any simple
 personalization element like the user name in the corner disqualifies
 the whole page from effective participation in downstream
 proxying/caching. The only thing you can do with such page is to cache
 it for one particular user hoping that s/he may need the same page again
 (which is not likely).
Well I did cache the page for each user, because most of the content
on a site isn't that dynamic that has to be reloaded every second.
Should content on a page change you invalid them for everyone. Even if
the cache is for short period it's worth it.

How: I used CacheFu and the description there.

 And to finish the picture -
 publication and access times are important for records purposes so the
 current server time should be displayed on all pages.
Well this is a problem, here you have something that invalidates the
page every second or at least minut, but still caching one minute is
worth alot on an active site. There are probably ways to do this with
JS so it's on the client side instead but then you can't be 100% that
it's the same time as on server but it shouldn't diff that much if you
initiate the JS with right time.

If your dynamic parts are not to complicated I would consider Squid3 and ESI.

/Sasha
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Zope vs Plone: performance issues!

2006-01-24 Thread Hugo Ramos
Well... They were already using Zope's cache and Squid to proxy some static HTML
I guess the problem is Plone itself... Skins etc...




On 1/24/06, Sasha Vincic [EMAIL PROTECTED] wrote:
 On 1/24/06, Peter Bengtsson [EMAIL PROTECTED] wrote:
  On 1/24/06, Hugo Ramos [EMAIL PROTECTED] wrote:
   Hi all,
  
   I'm responsible for a big corporate intranet and corporate portal.
   They were using Zope/Plone before I got here and the problems are too 
   many.
  
   Let's talk about performance issues... Plone is VERY SLOW responding
  

 snip

  Yes!
  But not good enough.
  Most performance issues with Plone can be solved by caching. Don't
  underestimate it.

 Yes Plone is slow but with caching you get very good performance. I
 have boosted performance on plone sites from default 1-3req/s to
 100req/s and then it was the bandwith that was the bottleneck. The
 sites where serving both anonymous and authenticated content. I
 recommend to check out the CacheFu product and documentation in the
 collective.

 /Sasha
 Lovely Systems - www.lovelysystems.com



--
Hugo Ramos - [EMAIL PROTECTED]
http://otugga.blogspot.com/
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )