Re: [Zope] Zope threads and the GIL
Dieter Maurer wrote: > Manuel Vazquez Acosta wrote at 2008-5-4 11:12 -0400: >> ... >> That would mean that the only place where real multitasking occurs is >> when fetching objects from ZEO and other I/O bounded tasks, isn't? > > I depends what you mean with "real multitasking". > > If you mean by this that several CPUs are concurrently active for your > Zope process, then you are near. Exactly. Several CPUs, I would like to use. > You can, however, also write code > for expensive time consuming operations in a language different from > Python and then release the GIL before you active such code. Then, > this code, too, can occupy a CPU in addition to the Python activity. > I'm very new to Python, and don't grasp yet the C API. Also I would like to implement almost everything in Python: I need this, b/c my of my team's skills in C are None. Best regards, Manuel. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
Manuel Vazquez Acosta wrote at 2008-5-4 11:12 -0400: > ... >That would mean that the only place where real multitasking occurs is >when fetching objects from ZEO and other I/O bounded tasks, isn't? I depends what you mean with "real multitasking". If you mean by this that several CPUs are concurrently active for your Zope process, then you are near. You can, however, also write code for expensive time consuming operations in a language different from Python and then release the GIL before you active such code. Then, this code, too, can occupy a CPU in addition to the Python activity. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
Andreas Jung wrote: > > > --On 5. Mai 2008 10:11:32 -0400 Manuel Vazquez Acosta > <[EMAIL PROTECTED]> wrote: > >> Andreas Jung wrote: >>> Then it should play nicely with Python. But keep in mind that thread >>> switches can also be blocked by some thread without having the GIL >>> locked. We've seen such situations were a thread had to perform complex >>> regular expression matching operations. Obviously code in Python C >>> extension is also able to lock up the Python interpreter until the >>> result is being passed back to the Python interpreter. >>> >> >> Another Q: Is AccessControl GIL aware? I mean, this is probably the most >> used component in Zope. Maybe a performance boost is experienced if >> those checks can be made simultaneously. > > Better tell us what your _real_ question/problem is. I am not getting > the point what you are really interested in. > > -aj I'm thinking in CPU-bounded parts of a request: Reindexing, Clustering, Automatic Content Extraction, and the like... We have implemented k-means clustering algorithm on top ZCTextIndex (now plan to port to zc.catalog), although this is batch operation, some other are per-query: Summarizing is one of those: Per cluster, and per document. (Although this is maybe a question for the OTS list). So I'm a little bit worried about how the resource utilization. Best regards, Manuel. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
--On 5. Mai 2008 10:11:32 -0400 Manuel Vazquez Acosta <[EMAIL PROTECTED]> wrote: Andreas Jung wrote: Then it should play nicely with Python. But keep in mind that thread switches can also be blocked by some thread without having the GIL locked. We've seen such situations were a thread had to perform complex regular expression matching operations. Obviously code in Python C extension is also able to lock up the Python interpreter until the result is being passed back to the Python interpreter. Another Q: Is AccessControl GIL aware? I mean, this is probably the most used component in Zope. Maybe a performance boost is experienced if those checks can be made simultaneously. Better tell us what your _real_ question/problem is. I am not getting the point what you are really interested in. -aj pgpenBunKtLxQ.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
Andreas Jung wrote: > Then it should play nicely with Python. But keep in mind that thread > switches can also be blocked by some thread without having the GIL locked. > We've seen such situations were a thread had to perform complex regular > expression matching operations. Obviously code in Python C extension is > also able to lock up the Python interpreter until the result is being > passed back to the Python interpreter. > Another Q: Is AccessControl GIL aware? I mean, this is probably the most used component in Zope. Maybe a performance boost is experienced if those checks can be made simultaneously. Thinking harder, I realize that a tradeoff is in place: Objects can change its security status (the security policy -- i.e, the security object -- can be changed also, but is a weird use case, isn't it?). So maybe write-lock is needed. On the other hand, for objects with a high demand rate, which are kept in memory, good results are to be expected. CacheFu deals OK with Anonymous users, but a boost is to be expected for authenticated users. Best regards, Manuel. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
--On 4. Mai 2008 21:05:55 -0500 sam <[EMAIL PROTECTED]> wrote: Basically mxODBC is a third party extension and is GIL "aware" as it yields the GIL before accessing the DB Then it should play nicely with Python. But keep in mind that thread switches can also be blocked by some thread without having the GIL locked. We've seen such situations were a thread had to perform complex regular expression matching operations. Obviously code in Python C extension is also able to lock up the Python interpreter until the result is being passed back to the Python interpreter. Andreas pgpdlbM51S6fb.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
Basically mxODBC is a third party extension and is GIL "aware" as it yields the GIL before accessing the DB - Original Message - From: "Andreas Jung" <[EMAIL PROTECTED]> To: "sam" <[EMAIL PROTECTED]>, "Manuel Vazquez Acosta" <[EMAIL PROTECTED]> Cc: "Zope user list" Sent: Sunday, May 4, 2008 7:36:08 PM (GMT-0600) America/Guatemala Subject: Re: [Zope] Zope threads and the GIL --On 4. Mai 2008 19:52:38 -0500 sam <[EMAIL PROTECTED]> wrote: > another scenario is if you are using GIL aware Database adapters like > mxODBC to access an RDBMS > What means GIL-aware in this context? -aj ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
--On 4. Mai 2008 19:52:38 -0500 sam <[EMAIL PROTECTED]> wrote: another scenario is if you are using GIL aware Database adapters like mxODBC to access an RDBMS What means GIL-aware in this context? -aj pgpUCTUVaiWM3.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
another scenario is if you are using GIL aware Database adapters like mxODBC to access an RDBMS sam - Original Message - From: "Manuel Vazquez Acosta" <[EMAIL PROTECTED]> To: "Andreas Jung" <[EMAIL PROTECTED]> Cc: "Zope user list" Sent: Sunday, May 4, 2008 9:12:16 AM (GMT-0600) America/Guatemala Subject: Re: [Zope] Zope threads and the GIL Andreas Jung wrote: > > > --On 4. Mai 2008 09:44:17 -0400 Manuel Vazquez Acosta > <[EMAIL PROTECTED]> wrote: > >> Hi all, How does the GIL affect Zope threads? > > Zope threads are Python threads and therefore a thread switch is not > possible is some thread is holding the GIL. > > -aj > That would mean that the only place where real multitasking occurs is when fetching objects from ZEO and other I/O bounded tasks, isn't? Best regards, Manuel ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
Andreas Jung wrote: > > > --On 4. Mai 2008 09:44:17 -0400 Manuel Vazquez Acosta > <[EMAIL PROTECTED]> wrote: > >> Hi all, How does the GIL affect Zope threads? > > Zope threads are Python threads and therefore a thread switch is not > possible is some thread is holding the GIL. > > -aj > That would mean that the only place where real multitasking occurs is when fetching objects from ZEO and other I/O bounded tasks, isn't? Best regards, Manuel ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope threads and the GIL
--On 4. Mai 2008 09:44:17 -0400 Manuel Vazquez Acosta <[EMAIL PROTECTED]> wrote: Hi all, How does the GIL affect Zope threads? Zope threads are Python threads and therefore a thread switch is not possible is some thread is holding the GIL. -aj pgpE59zdNIMkB.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope threads and the GIL
Hi all, How does the GIL affect Zope threads? Best regards, Manuel. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
