Re: [Python-Dev] Desired changes to Hg emails to python-checkins

2010-03-13 Thread Georg Brandl 
Am 04.03.2010 21:20, schrieb Brett Cannon:
> 1) I miss not having the affected files listed in the subject line.
> 
> 2) The To field is set to [email protected]  which
> gets rejected as an invalid email address if you reply. Would be better
> to set it to python-checkins so that the habitual reply to a checkin
> won't get rejected.

Before I forget:  for CPython, if all branches are to be in one hg repo
with named branches, the branch name also needs to be in the subject line.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] interesting article on regex performance

2010-03-13 Thread Nick Coghlan 
Collin Winter wrote:
> On Fri, Mar 12, 2010 at 8:12 AM, Nick Coghlan  wrote:
>> There are major practical problems associated with making such a leap
>> directly (Google's re2 engine is in C++ rather than C and we'd have to
>> keep the existing implementation around regardless to handle the
>> features that re2 doesn't support).
> 
> I don't see why C++ would be a deal-breaker in this case, since it
> would be restricted to an extension module.

It wouldn't be a dealbreaker for an optional regex acceleration module,
I agree. It's still a practical problem in the sense that C++ code is
very much in the minority in the CPython code base.

>> I would say it is better to let re2 bake for a while and see if anyone
>> is motivated to come up with Python bindings for it and release them on
>> PyPI.
> 
> FWIW, re2 is heavily, heavily used in production at Google.
> Stabilizing any proposed Python bindings would be a good idea, but I'm
> not sure how much more "baking" re2's core functionality needs.

It was more the bindings side of things I was referring to - the
algorithm's performance is no doubt well established, but how easy it is
for non-Google users to pick up and integrate with other software is
still to be determined.

> We considered such a hybrid approach for Unladen Swallow, but rejected
> it on the advice of the V8 team [1]: you end up maintaining two
> totally separate, incompatible regex engines; the hybrid system comes
> with interesting, possibly unintuitive performance/correctness issues
> when bailing from one implementation to another; performance is
> unstable as small changes are made to the regexes; and it's not
> obvious that enough Python regexes would benefit from re2's
> performance/restrictions tradeoff to make such a hybrid system
> worthwhile in the long term. (There is no representative corpus of
> real-world Python regexes weighted for dynamic execution frequency to
> use in assessing such tradeoffs empirically like there is for
> JavaScript.)
> 
> re2 is very useful when you want to run user-provided regexes and want
> to protect your backends against pathological/malicious regex input,
> but I'm not sure how applicable it is to Python. I think there are
> more promising strategies to improve regex performance, such as
> reusing the new JIT infrastructure to JIT-compile regular expressions
> to machine code (along the lines of V8's irregexp). Some work has
> already been done in this direction, and I'd be thrilled to mentor any
> GSoC students interested in working on such a project this summer.
> 
> Lastly, anyone interested in working on Python regex performance
> should take a look at the regex benchmarks in the standard benchmark
> suite [2].

This is the kind of baking that I was really talking about - there's an
awful lot of investigation to be done before moving Python in the
direction of an NFA based regex engine could be seriously considered.

Good to hear the perspective of someone that has already looked into
this in some detail though.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C++

2010-03-13 Thread Neal Becker 
Neil Hodgson wrote:

> Antoine Pitrou:
> 
>> Is this concern still valid? We are in the 2010s now.
>> I'm not saying I want us to put some C++ in the core interpreter, but
>> the portability argument sounds a little old...
> 
>There are still viable platforms which only support subsets of C++.
> IIRC, Android does not support exceptions in C++.
> 
>Neil

Does re2 require exceptions?

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __file__ and bytecode-only

2010-03-13 Thread Nick Coghlan 
Barry Warsaw wrote:
> *creation* is the key here.  As per BDFL pronouncement, we'll support reading
> pyc-only modules just like we do today.  This is in PEP 3147.  We won't
> support creating them though.

The creation side could be made a little more explicit in the PEP. We
could also do something via the compileall module.

(Pause while Nick goes and reads the source code for compileall for the
first time ever...)

Hmm - methinks the PEP actually needs to talk explicitly about the
py_compile and compileall modules. These compile the files directly
rather than using the import system's side-effect, so they'll need to
understand the intricacies of the new system.

While it's probably OK if the import side-effects only create files
using the new scheme, the standard library modules will likely need to
support both schemes (although I'm not sure if "same as import system"
or "same as Python 3.1" make more sense as the default semantics -
probably the former).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C++

2010-03-13 Thread skip 

Jeffrey> If you know of platforms that don't support particular features
Jeffrey> of C++, please link to documentation of that like Neil Hodgson
Jeffrey> did. If not, please stop spreading FUD.

No need to get your knickers in a twist.  Every couple years someone asks,
"why isn't Python written in C++?"  The lack of good C++ support for many
(minority) platforms is often the primary explanation.  If that's changing,
I have no problem with it.

I spent a fair amount of time with Google trying to find representative
threads here or in comp.lang.python but failed to find any needles in the
haystack.  Perhaps you will have better luck.

Skip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Patch to telnetlib.py

2010-03-13 Thread gregory dudek 
The Telnet module telnetlib.py can be
very slow -- unusably slow -- for large automated data transfers.  There are 
typically done in raw mode.

The attached patch greatly increased the speed of telnet interactions in raw 
mode.  I submitted this a couple of year ago, but it was for an older branch of 
python.

There are 2 key things being done:
 1) concatenations string with string.join instead of '+'  (which is probably a 
minor issue)
 2) wholesale appending the raw and processed buffers when the IAC character is 
not found.  The should be examined
 carefully since I am not an expert in the Telnet protocol, but it seems to 
work very well giving me a 5x speedup.


--- installed/telnetlib.py  2010-02-02 22:57:58.0 -0500
+++ telnetlib.py2010-03-13 12:17:02.0 -0500
@@ -30,6 +30,7 @@
 - timeout should be intrinsic to the connection object instead of an
   option on one of the read calls only
 
+Modified by G. Dudek for greater efficiency.
 """
 
 
@@ -420,6 +421,14 @@
 """
 buf = ['', '']
 try:
+if self.rawq:
+if not IAC in self.rawq:
+# speed hack, no IAC just grab whole queue. --Dudek
+buf[self.sb] = "".join((buf[self.sb] , self.rawq))
+self.cookedq = "".join((self.cookedq , buf[0] ))
+self.sbdataq = "".join((self.sbdataq , buf[1] ))
+self.rawq_flush()
+return
 while self.rawq:
 c = self.rawq_getchar()
 if not self.iacseq:
@@ -428,7 +437,7 @@
 if c == "\021":
 continue
 if c != IAC:
-buf[self.sb] = buf[self.sb] + c
+buf[self.sb] = "".join((buf[self.sb] , c))
 continue
 else:
 self.iacseq += c
@@ -480,8 +489,14 @@
 self.iacseq = '' # Reset on EOF
 self.sb = 0
 pass
-self.cookedq = self.cookedq + buf[0]
-self.sbdataq = self.sbdataq + buf[1]
+self.cookedq = "".join((self.cookedq , buf[0] ))
+self.sbdataq = "".join((self.sbdataq , buf[1] ))
+
+
+def rawq_flush(self):
+""" Set the queue to empty status """
+self.rawq = ''
+self.irawq = 0
 
 def rawq_getchar(self):
 """Get next char from raw queue.
@@ -516,7 +531,7 @@
 buf = self.sock.recv(50)
 self.msg("recv %r", buf)
 self.eof = (not buf)
-self.rawq = self.rawq + buf
+self.rawq = "".join((self.rawq,buf))
 
 def sock_avail(self):
 """Test whether data is available on the socket."""

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch to telnetlib.py

2010-03-13 Thread Terry Reedy 

On 3/13/2010 12:24 PM, gregory dudek wrote:

The Telnet module telnetlib.py can be
very slow -- unusably slow -- for large automated data transfers.  There are 
typically done in raw mode.

The attached patch


Please submit to the tracker. If there is an existing issue, attach it 
to that.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch to telnetlib.py

2010-03-13 Thread Eric Smith 

Can you create an issue on the bug tracker? Otherwise this will get lost.

Eric.

On 3/13/2010 12:24 PM, gregory dudek wrote:

The Telnet module telnetlib.py can be
very slow -- unusably slow -- for large automated data transfers.  There are 
typically done in raw mode.

The attached patch greatly increased the speed of telnet interactions in raw 
mode.  I submitted this a couple of year ago, but it was for an older branch of 
python.

There are 2 key things being done:
  1) concatenations string with string.join instead of '+'  (which is probably 
a minor issue)
  2) wholesale appending the raw and processed buffers when the IAC character 
is not found.  The should be examined
  carefully since I am not an expert in the Telnet protocol, but it seems 
to work very well giving me a 5x speedup.


--- installed/telnetlib.py  2010-02-02 22:57:58.0 -0500
+++ telnetlib.py2010-03-13 12:17:02.0 -0500
@@ -30,6 +30,7 @@
  - timeout should be intrinsic to the connection object instead of an
option on one of the read calls only

+Modified by G. Dudek for greater efficiency.
  """


@@ -420,6 +421,14 @@
  """
  buf = ['', '']
  try:
+if self.rawq:
+if not IAC in self.rawq:
+# speed hack, no IAC just grab whole queue. --Dudek
+buf[self.sb] = "".join((buf[self.sb] , self.rawq))
+self.cookedq = "".join((self.cookedq , buf[0] ))
+self.sbdataq = "".join((self.sbdataq , buf[1] ))
+self.rawq_flush()
+return
  while self.rawq:
  c = self.rawq_getchar()
  if not self.iacseq:
@@ -428,7 +437,7 @@
  if c == "\021":
  continue
  if c != IAC:
-buf[self.sb] = buf[self.sb] + c
+buf[self.sb] = "".join((buf[self.sb] , c))
  continue
  else:
  self.iacseq += c
@@ -480,8 +489,14 @@
  self.iacseq = '' # Reset on EOF
  self.sb = 0
  pass
-self.cookedq = self.cookedq + buf[0]
-self.sbdataq = self.sbdataq + buf[1]
+self.cookedq = "".join((self.cookedq , buf[0] ))
+self.sbdataq = "".join((self.sbdataq , buf[1] ))
+
+
+def rawq_flush(self):
+""" Set the queue to empty status """
+self.rawq = ''
+self.irawq = 0

  def rawq_getchar(self):
  """Get next char from raw queue.
@@ -516,7 +531,7 @@
  buf = self.sock.recv(50)
  self.msg("recv %r", buf)
  self.eof = (not buf)
-self.rawq = self.rawq + buf
+self.rawq = "".join((self.rawq,buf))

  def sock_avail(self):
  """Test whether data is available on the socket."""

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch to telnetlib.py

2010-03-13 Thread Guido van Rossum 
On Sat, Mar 13, 2010 at 9:24 AM, gregory dudek  wrote:
> The Telnet module telnetlib.py can be
> very slow -- unusably slow -- for large automated data transfers.  There are 
> typically done in raw mode.
>
> The attached patch greatly increased the speed of telnet interactions in raw 
> mode.  I submitted this a couple of year ago, but it was for an older branch 
> of python.

For which Python version is your new patch?

> There are 2 key things being done:
>  1) concatenations string with string.join instead of '+'  (which is probably 
> a minor issue)

This looks like superstition in the patch below -- there is no reason
why "".join((a, b)) should be any faster than a+b. (The join trick is
important when joining *many* strings, but for two it doesn't make a
difference, both strings have to be copied at least once no matter
what you try.)

>  2) wholesale appending the raw and processed buffers when the IAC character 
> is not found.  The should be examined
>     carefully since I am not an expert in the Telnet protocol, but it seems 
> to work very well giving me a 5x speedup.

This makes sense and is likely the reason your patch is faster.

You should really submit this to the bug tracker at bugs.python.org.

> --- installed/telnetlib.py      2010-02-02 22:57:58.0 -0500
> +++ telnetlib.py        2010-03-13 12:17:02.0 -0500
> @@ -30,6 +30,7 @@
>  - timeout should be intrinsic to the connection object instead of an
>   option on one of the read calls only
>
> +Modified by G. Dudek for greater efficiency.

You're gonna need to submit a Python contributor agreement
(http://www.python.org/psf/contrib/contrib-form/) if/when your patch
gets accepted. Welcome to the club!

>  """
>
>
> @@ -420,6 +421,14 @@
>         """
>         buf = ['', '']
>         try:
> +            if self.rawq:
> +                if not IAC in self.rawq:
> +                    # speed hack, no IAC just grab whole queue. --Dudek
> +                    buf[self.sb] = "".join((buf[self.sb] , self.rawq))
> +                    self.cookedq = "".join((self.cookedq , buf[0] ))
> +                    self.sbdataq = "".join((self.sbdataq , buf[1] ))
> +                    self.rawq_flush()
> +                    return
>             while self.rawq:
>                 c = self.rawq_getchar()
>                 if not self.iacseq:
> @@ -428,7 +437,7 @@
>                     if c == "\021":
>                         continue
>                     if c != IAC:
> -                        buf[self.sb] = buf[self.sb] + c
> +                        buf[self.sb] = "".join((buf[self.sb] , c))
>                         continue
>                     else:
>                         self.iacseq += c
> @@ -480,8 +489,14 @@
>             self.iacseq = '' # Reset on EOF
>             self.sb = 0
>             pass
> -        self.cookedq = self.cookedq + buf[0]
> -        self.sbdataq = self.sbdataq + buf[1]
> +        self.cookedq = "".join((self.cookedq , buf[0] ))
> +        self.sbdataq = "".join((self.sbdataq , buf[1] ))
> +
> +
> +    def rawq_flush(self):
> +        """ Set the queue to empty status """
> +        self.rawq = ''
> +        self.irawq = 0
>
>     def rawq_getchar(self):
>         """Get next char from raw queue.
> @@ -516,7 +531,7 @@
>         buf = self.sock.recv(50)
>         self.msg("recv %r", buf)
>         self.eof = (not buf)
> -        self.rawq = self.rawq + buf
> +        self.rawq = "".join((self.rawq,buf))
>
>     def sock_avail(self):
>         """Test whether data is available on the socket."""
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C++

2010-03-13 Thread Gregory P. Smith 
On Sat, Mar 13, 2010 at 6:00 AM, Neal Becker  wrote:
> Neil Hodgson wrote:
>
>> Antoine Pitrou:
>>
>>> Is this concern still valid? We are in the 2010s now.
>>> I'm not saying I want us to put some C++ in the core interpreter, but
>>> the portability argument sounds a little old...
>>
>>    There are still viable platforms which only support subsets of C++.
>> IIRC, Android does not support exceptions in C++.
>>
>>    Neil
>
> Does re2 require exceptions?

Nope.  No exceptions.

It was written following this style guide:
http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Exceptions
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch to telnetlib.py

2010-03-13 Thread Jack Diederich 
On Sat, Mar 13, 2010 at 12:24 PM, gregory dudek  wrote:
> The Telnet module telnetlib.py can be
> very slow -- unusably slow -- for large automated data transfers.  There are 
> typically done in raw mode.
>
> The attached patch greatly increased the speed of telnet interactions in raw 
> mode.  I submitted this a couple of year ago, but it was for an older branch 
> of python.
>
> There are 2 key things being done:
>  1) concatenations string with string.join instead of '+'  (which is probably 
> a minor issue)
>  2) wholesale appending the raw and processed buffers when the IAC character 
> is not found.  The should be examined
>     carefully since I am not an expert in the Telnet protocol, but it seems 
> to work very well giving me a 5x speedup.

As others mentioned, please post the bug to the tracker.  Also, please
assign the patch to me ("jackdied") and mention the previous bug
number - I thought I had reviewed all the telnetlib bugs including
those that were closed WONTFIX.

Thanks for the kick in the pants, I have a whole new inner loop for
data processing but I haven't applied it.  I've been adding unit tests
to the module so I could be sure I wouldn't break anything but never
finished the job.

-Jack
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Antoine Pitrou 

Hello,

As some of you may know, Dave Beazley recently exhibited a situation
where the new GIL shows quite a poor behaviour (the old GIL isn't very
good either, but still a little better). This issue is followed in
http://bugs.python.org/issue7946

This situation is when an IO-bound thread wants to process a lot of
incoming packets, while one (or several) CPU-bound thread is also
running. Each time the IO-bound thread releases the GIL, the CPU-bound
thread gets it and keeps holding it for at least 5 milliseconds
(default setting), which limits the number of individual packets which
can be recv()'ed and processed per second.

I have proposed two mechanisms, based on the same idea: IO-bound
threads should be able to steal the GIL very quickly, rather than
having to wait for the whole "thread switching interval" (again, 5 ms
by default). They differ in how they detect an "IO-bound threads":

- the first mechanism is actually the same mechanism which was
  embodied in the original new GIL patch before being removed. In this
  approach, IO methods (such as socket.read() in socketmodule.c)
  releasing the GIL must use a separate C macro when trying to get the
  GIL back again.

- the second mechanism dynamically computes the "interactiveness" of a
  thread and allows interactive threads to steal the GIL quickly. In
  this approach, IO methods don't have to be modified at all.

Both approaches show similar benchmark results (for the benchmarks
that I know of) and basically fix the issue put forward by Dave Beazley.

Any thoughts?

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Benjamin Peterson 
2010/3/13 Antoine Pitrou :
> Any thoughts?
\
The latter solution seems best to me as it would help any 3rd party IO
libraries and require less code modification.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Martin v. Löwis 
> Any thoughts?

My initial reaction to this report, and still my current opinion is:

This issue is not a problem.

It's a boundary case, so it's not clear whether Python should be able to
deal with it gracefully. I doubt it's a realistic case.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Daniel Stutzbach 
On Sat, Mar 13, 2010 at 3:46 PM, Antoine Pitrou  wrote:

> - the second mechanism dynamically computes the "interactiveness" of a
>  thread and allows interactive threads to steal the GIL quickly. In
>  this approach, IO methods don't have to be modified at all.
>

I like the second approach as well, assuming "interactiveness" can be
computed cheaply.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread skip 

If the 5ms interval is too long would it be possible to adaptively reduce it
in this situation?  How would you detect it?  I assume this would be akin to
your interactiveness computation.

Skip

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Antoine Pitrou 
Le Sat, 13 Mar 2010 17:11:41 -0600,
[email protected] a écrit :
> 
> If the 5ms interval is too long would it be possible to adaptively
> reduce it in this situation?  How would you detect it?  I assume this
> would be akin to your interactiveness computation.

I think modulating the interval is what Dave Beazley's own patch
proposal does. It has a full-blown priority system. It is more
complicated than either of the two approaches I am more proposing,
though.
(also, per Dave's own admission, his patch is a proof-of-concept rather
than a finished piece of work)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Brett Cannon 
On Sat, Mar 13, 2010 at 13:52, Benjamin Peterson wrote:

> 2010/3/13 Antoine Pitrou :
> > Any thoughts?
> \
> The latter solution seems best to me as it would help any 3rd party IO
> libraries and require less code modification.
>

Plus the interactiveness approach has been tested by OS thread schedulers
for years and is shown to work. So I vote for the latter approach as well.

-Brett



>
>
>
> --
> Regards,
> Benjamin
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Nick Coghlan 
Brett Cannon wrote:
> 
> 
> On Sat, Mar 13, 2010 at 13:52, Benjamin Peterson  > wrote:
> 
> 2010/3/13 Antoine Pitrou  >:
> > Any thoughts?
> \
> The latter solution seems best to me as it would help any 3rd party IO
> libraries and require less code modification.
> 
> 
> Plus the interactiveness approach has been tested by OS thread
> schedulers for years and is shown to work. So I vote for the latter
> approach as well.

+1

Cheap to calculate and means developer don't need to guess if they
should be using the "I am CPU bound" or the "I am IO bound" GIL
acquisition macro.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

2010-03-13 Thread Cameron Simpson 
On 13Mar2010 23:03, Martin v. L�wis  wrote:
| > Any thoughts?
| 
| My initial reaction to this report, and still my current opinion is:
| This issue is not a problem.
| It's a boundary case, so it's not clear whether Python should be able to
| deal with it gracefully. I doubt it's a realistic case.

How was it tripped over?

Speaking for myself, I have an app with a daemon mode which I expect
will sometimes behave as described; it answers requests and thus has I/O
bound threads waiting for requests and dispatching replies, and threads
doing data handling, which make constant use of the zlib library. On the
client side the same app is often throughput bound by a data examination
process that is compute bound; I can easily see it having compute bound
threads and I/O bound threads talking to daemon instances.

Currently all the above is somewhat "batchy"; the client end looks like
an archiving command line tool, but it's intended to have a FUSE mode to
present the archive as a filesystem; that I can imagine tripping over
this issue as a user uses the filesystem for "stuff".

Still, it is unusual and I suspect it will self limit to an extent
anyway; I can't fairly claim to have had it cause me trouble yet.
(Indeed, I've not used the new GIL at all - it's mostly using python
2.6).

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

I very strongly suggest that you periodically place ice packs over the abused
areas.  - Steve Garnier
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com