Re: [Zope-dev] Zope server speed
what i have done before to get around the speed penalty is to create an external method. the external method calls the ZSQL method directly and iterates over the result objects to pull out all interesting attributes and stick them into simple python objects, in this case i returned a list of dictionaries from the external method. in DTML the iteration over it still looks the same but it was much faster because there is no security enforced on the simple python objects returned from the external method. jens On Thursday, December 6, 2001, at 09:33 , MANOTTI Alessandro wrote: > Thank you for your suggestions (thanks to Matt). > > I will try to make smaller but more requests (1000 requests of 1000 > records). > > I will even try to "bypass" Zope and write an "external method" in Python > (I > know Pytho very well). > About this solution: can you tell me what I have to do to use ZOracle > directly in Python (import ... ? ) ? > > Thanks to everybody! > > > > > -Original Message- > From: Matthew T. Kromer [mailto:[EMAIL PROTECTED]] > Sent: Thursday, December 06, 2001 15:27 > To: MANOTTI Alessandro; [EMAIL PROTECTED] > Subject: Re: [Zope-dev] Zope server speed > > > Hi Alessandro, > > Zope and Python combined aren't too bad if you write some wrappers that > help > you manipulate data; the problem comes about when you try to do large > naive > queries -- there is a LOT of overhead in taking Oracle data, encapsulating > it in Python objects, and then wrapping that in the Zope machinery. > > I have a sample database in Oracle of US Zip codes -- it has about 45,000 > records in it. I can pull this into DCOracle2 on the C adapter side > (dco2.so) in less than a second. However, wrapping each result cell in a > python object blows that up into about 9 seconds. If I were to iterate > over > the whole thing in Zope (which I haven't timed) it would get a LOT longer, > because Zope is enforcing policy on every object accessed. > > So, one of the ways to make that go faster is to use python modules > 'outside' of Zope (ie external methods) to perform large calculations > which > takes a ZOracleDA connection object to get a cursor object on the > database, > does the query directly, and formats and returns the results. > > I realize this isn't ideal, but Zope is not optimized for the case of > "extract thousands of records from the database and format them in a huge > table." Zope likes "extract hundreds of records from the database and > format them in a big table" instead. > > With each release, Zope has more features that enable you to program > through > the web, but when you're doing performance intensive calculations, it's > usually time to write a python module and plug it in to zope (via an > external method, or by writing a product). If you have the time to learn, > I > suggest the product route as being more satisfying, but it can be more > frustrating as well. > > > - Original Message - > From: "MANOTTI Alessandro" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Thursday, December 06, 2001 8:52 AM > Subject: [Zope-dev] Zope server speed > > >> >>> HI all, >>> >>> I have a problem: >>> >>> my company is looking for an application server. I know Zope (I > presented >>> it), and a friend of mine presented Macromedia ColdFusion. >>> >>> We made some tests about the speed of the systems, arghh!!! Zope is VERY >>> slow compared to ColdFusion! >>> >>> Some examples: >>> >>> I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! > it >>> is a test... two hundred fiftyseven thousand records!) and Zope took > more >>> than 10 minutes, since I stopped it after that time (call SQL and print >>> data via TAG). >>> >>> The same PC, the same server, ColdFusion took about 2 minutes to > complete >>> the operation!!! >>> >>> Coldfusion is more than 5 times faster than Zope!!! >>> >>> PHASE 2: >>> >>> I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... > and >>> I tried to get the records and perform a calculation, without no > output... >>> >>> Sigh! the results are the same! >>> >>> I am depressed, since I talk about Zope very well in my company, but >>> now... >>> >>> Coldfusion and Zope are similar, so we shall select ColdFusion to > develop >>> web applications. >>> >>&
Re: [Zope-dev] Zope server speed
On Thu, 6 Dec 2001 14:52:06 +0100, MANOTTI Alessandro <[EMAIL PROTECTED]> wrote: >> I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it >> is a test... two hundred fiftyseven thousand records!) and Zope took more >> than 10 minutes, since I stopped it after that time (call SQL and print >> data via TAG). Yes, the SQL driver abstraction layer is slower than it could be. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Zope server speed
Thank you for your suggestions (thanks to Matt). I will try to make smaller but more requests (1000 requests of 1000 records). I will even try to "bypass" Zope and write an "external method" in Python (I know Pytho very well). About this solution: can you tell me what I have to do to use ZOracle directly in Python (import ... ? ) ? Thanks to everybody! -Original Message- From: Matthew T. Kromer [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 06, 2001 15:27 To: MANOTTI Alessandro; [EMAIL PROTECTED] Subject: Re: [Zope-dev] Zope server speed Hi Alessandro, Zope and Python combined aren't too bad if you write some wrappers that help you manipulate data; the problem comes about when you try to do large naive queries -- there is a LOT of overhead in taking Oracle data, encapsulating it in Python objects, and then wrapping that in the Zope machinery. I have a sample database in Oracle of US Zip codes -- it has about 45,000 records in it. I can pull this into DCOracle2 on the C adapter side (dco2.so) in less than a second. However, wrapping each result cell in a python object blows that up into about 9 seconds. If I were to iterate over the whole thing in Zope (which I haven't timed) it would get a LOT longer, because Zope is enforcing policy on every object accessed. So, one of the ways to make that go faster is to use python modules 'outside' of Zope (ie external methods) to perform large calculations which takes a ZOracleDA connection object to get a cursor object on the database, does the query directly, and formats and returns the results. I realize this isn't ideal, but Zope is not optimized for the case of "extract thousands of records from the database and format them in a huge table." Zope likes "extract hundreds of records from the database and format them in a big table" instead. With each release, Zope has more features that enable you to program through the web, but when you're doing performance intensive calculations, it's usually time to write a python module and plug it in to zope (via an external method, or by writing a product). If you have the time to learn, I suggest the product route as being more satisfying, but it can be more frustrating as well. - Original Message - From: "MANOTTI Alessandro" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, December 06, 2001 8:52 AM Subject: [Zope-dev] Zope server speed > > > HI all, > > > > I have a problem: > > > > my company is looking for an application server. I know Zope (I presented > > it), and a friend of mine presented Macromedia ColdFusion. > > > > We made some tests about the speed of the systems, arghh!!! Zope is VERY > > slow compared to ColdFusion! > > > > Some examples: > > > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > > is a test... two hundred fiftyseven thousand records!) and Zope took more > > than 10 minutes, since I stopped it after that time (call SQL and print > > data via TAG). > > > > The same PC, the same server, ColdFusion took about 2 minutes to complete > > the operation!!! > > > > Coldfusion is more than 5 times faster than Zope!!! > > > > PHASE 2: > > > > I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... and > > I tried to get the records and perform a calculation, without no output... > > > > Sigh! the results are the same! > > > > I am depressed, since I talk about Zope very well in my company, but > > now... > > > > Coldfusion and Zope are similar, so we shall select ColdFusion to develop > > web applications. > > > > SIGH !!! > > > > Can I try something to increase Zope power?! > > > > > > > > > > Alessandro Manotti > > Presidente dell'Associazione "RIUSA" > > > > Sito web: http://riusa.apritisesamo.net > > email:[EMAIL PROTECTED] > > mailing-list: [EMAIL PROTECTED] > > > > > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope server speed
Hi Alessandro, Zope and Python combined aren't too bad if you write some wrappers that help you manipulate data; the problem comes about when you try to do large naive queries -- there is a LOT of overhead in taking Oracle data, encapsulating it in Python objects, and then wrapping that in the Zope machinery. I have a sample database in Oracle of US Zip codes -- it has about 45,000 records in it. I can pull this into DCOracle2 on the C adapter side (dco2.so) in less than a second. However, wrapping each result cell in a python object blows that up into about 9 seconds. If I were to iterate over the whole thing in Zope (which I haven't timed) it would get a LOT longer, because Zope is enforcing policy on every object accessed. So, one of the ways to make that go faster is to use python modules 'outside' of Zope (ie external methods) to perform large calculations which takes a ZOracleDA connection object to get a cursor object on the database, does the query directly, and formats and returns the results. I realize this isn't ideal, but Zope is not optimized for the case of "extract thousands of records from the database and format them in a huge table." Zope likes "extract hundreds of records from the database and format them in a big table" instead. With each release, Zope has more features that enable you to program through the web, but when you're doing performance intensive calculations, it's usually time to write a python module and plug it in to zope (via an external method, or by writing a product). If you have the time to learn, I suggest the product route as being more satisfying, but it can be more frustrating as well. - Original Message - From: "MANOTTI Alessandro" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, December 06, 2001 8:52 AM Subject: [Zope-dev] Zope server speed > > > HI all, > > > > I have a problem: > > > > my company is looking for an application server. I know Zope (I presented > > it), and a friend of mine presented Macromedia ColdFusion. > > > > We made some tests about the speed of the systems, arghh!!! Zope is VERY > > slow compared to ColdFusion! > > > > Some examples: > > > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > > is a test... two hundred fiftyseven thousand records!) and Zope took more > > than 10 minutes, since I stopped it after that time (call SQL and print > > data via TAG). > > > > The same PC, the same server, ColdFusion took about 2 minutes to complete > > the operation!!! > > > > Coldfusion is more than 5 times faster than Zope!!! > > > > PHASE 2: > > > > I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... and > > I tried to get the records and perform a calculation, without no output... > > > > Sigh! the results are the same! > > > > I am depressed, since I talk about Zope very well in my company, but > > now... > > > > Coldfusion and Zope are similar, so we shall select ColdFusion to develop > > web applications. > > > > SIGH !!! > > > > Can I try something to increase Zope power?! > > > > > > > > > > Alessandro Manotti > > Presidente dell'Associazione "RIUSA" > > > > Sito web: http://riusa.apritisesamo.net > > email:[EMAIL PROTECTED] > > mailing-list: [EMAIL PROTECTED] > > > > > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope server speed
On Thu, 6 Dec 2001, MANOTTI Alessandro wrote: > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > > is a test... two hundred fiftyseven thousand records!) and Zope took more > > than 10 minutes, since I stopped it after that time (call SQL and print > > data via TAG). That is quite a lot of records :) Out of interest, why test pulling out that many records? Does your application need to return that many records to the user? Can calculations be done in a stored procedure or something. I've never tried doing anything like this, so I don't really know what I'm talking about :) however I would have thought that it doesn't sound like a particularly realistic test (not that I know what your application does). I wonder how fast Zope is compared to CF in doing, say, 1000 repeat requests each of, say, 1000 records. -Matt -- Matt Hamilton [EMAIL PROTECTED] Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope server speed
> HI all, > > I have a problem: > > my company is looking for an application server. I know Zope (I presented > it), and a friend of mine presented Macromedia ColdFusion. > > We made some tests about the speed of the systems, arghh!!! Zope is VERY > slow compared to ColdFusion! > > Some examples: > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > is a test... two hundred fiftyseven thousand records!) and Zope took more > than 10 minutes, since I stopped it after that time (call SQL and print > data via TAG). > > The same PC, the same server, ColdFusion took about 2 minutes to complete > the operation!!! > > Coldfusion is more than 5 times faster than Zope!!! > > PHASE 2: > > I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... and > I tried to get the records and perform a calculation, without no output... > > Sigh! the results are the same! > > I am depressed, since I talk about Zope very well in my company, but > now... > > Coldfusion and Zope are similar, so we shall select ColdFusion to develop > web applications. > > SIGH !!! > > Can I try something to increase Zope power?! > > > > > Alessandro Manotti > Presidente dell'Associazione "RIUSA" > > Sito web: http://riusa.apritisesamo.net > email:[EMAIL PROTECTED] > mailing-list: [EMAIL PROTECTED] > > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
