Ivan Krstic <[EMAIL PROTECTED]> wrote:
> * Bite the bullet; write and support a stdlib SHM primitive that works
> wherever possible, and simply doesn't work on completely broken
> platforms (I understand Windows falls into this category). Utilize it in
> a lightweight fork-and-coordinate wrapper p
Bob Ippolito wrote:
> Candygram is heavyweight by trade-off, not because it has to be.
> Candygram could absolutely be implemented efficiently in current
> Python if a Twisted-like style was used.
Specifically?
>> * Bite the bullet; write and support a stdlib SHM primitive that works [..]
>> a l
On 9/18/06, Ivan Krstić <[EMAIL PROTECTED]> wrote:
> > What use case *requires* sharing?
>
> Strictly speaking, it's always avoidable. But in setup-heavy systems,
> avoiding SHM is a massive and costly pain. Consider web applications;
> ideally, you can preload one copy of all of your translations,
On Sun, 17 Sep 2006 18:18:32 -0700, Josiah Carlson <[EMAIL PROTECTED]> wrote:
>import autorpc
>caller = autorpc.init_processes(autorpc.num_processors())
>
>import callables
>caller.register_module(callables)
>
>result = caller.fcn1(arg1, arg2, arg3)
>
>The point is not to compar
Calvin Spealman wrote:
> So reload the information you want available to worker tasks and pass
> that information along to them, or provide a mechanism for them to
> request it from its preloaded housing.
With large sets, you can't afford duplicate copies in memory, so there's
nothing to reload.
On 9/17/06, Ivan Krstić <[EMAIL PROTECTED]> wrote:
At present, the Python approach to multi-processing sounds a bit like"let's stick our collective hands in the sand and pretend there's no
problem". In particular, one oft-parroted argument says that it's notworth changing or optimizing the language
Paul Prescod wrote:
> I think that there would be value in saying: "There's a
> big problem here and we intend to fix it in Python 3000."
I'm not at all convinced that this is something to be addressed in 3.0.
Py3k is about removing cruft, not adding features; a proper MP system
represents a featu
Brett Cannon wrote:
> On 9/17/06, *Nick Coghlan* <[EMAIL PROTECTED]
>- use threads to perform blocking I/O in parallel
>- use multiple interpreters to perform Python execution in parallel
>
>
> Possibly, but as it stands now interpreters just execute in their own
> Python thread
Antoine Pitrou wrote:
> Le dimanche 17 septembre 2006 à 23:16 +1000, Nick Coghlan a écrit :
>> Brett Cannon's sandboxing work (which aims to provide first-class support
>> for
>> multiple interpreters in the same process for security reasons) also seems
>> like a potentially fruitful approach to
"Ruminating" is the best word I can think of here - I've been slowly
digesting the ideas and discussions over the last couple months. Part of
the reason why all this is relevant to me is that I'm working on a
couple of side projects, some of which involve "mini-languages" that
have similar issu
I think all the attempts to expose GIL-less semantics to python code
miss the point. Reference counting turns all references into
modifications. You can't avoid the GIL without first changing
reference counting.
There's a few ways to approach this:
* atomic INCREF/DECREF using cpu instructions.
Guido -- If I'm not mis-stating, this might be a candidate for PEP 3099.
On 9/18/06, Ivan Krstić <[EMAIL PROTECTED]> wrote:
> Paul Prescod wrote:
> > Ivan: why don't you write a PEP about this?
> I'd like to hear Guido's overarching thoughts on the matter, if any, and
> would afterwards be happy
On Mon, Sep 18, 2006 at 11:56:51AM -0400, Jim Jewett wrote:
> IIRC, his most recent statements boiled down to:
>
> (1) The GIL works well enough, most of the time.
1a. On multiprocessor/multicore systems use processes, not threads.
> (2) Taking it out is harder than people realize.
> (3) Ther
"Adam Olsen" <[EMAIL PROTECTED]> writes:
> * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally unsupported
> by the C standards and changes cache characteristics that CPython has
> been designed with for years, likely with a very large performance
> penalty.
Last time I did some GC benchma
The thread subject notwithstanding, the majority of the discussion was about ways to work around the GIL, not remove it. Therefore the thing you might put in PEP 3099 is not the thing under active discussion.
On 9/18/06, Jim Jewett <[EMAIL PROTECTED]> wrote:
Guido -- If I'm not mis-stating, this mi
On 9/18/06, Marcin 'Qrczak' Kowalczyk <[EMAIL PROTECTED]> wrote:
> "Adam Olsen" <[EMAIL PROTECTED]> writes:
>
> > * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally unsupported
> > by the C standards and changes cache characteristics that CPython has
> > been designed with for years, likely
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Sep 18, 2006, at 12:27 PM, Marcin 'Qrczak' Kowalczyk wrote:
> "Adam Olsen" <[EMAIL PROTECTED]> writes:
>
>> * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally
>> unsupported
>> by the C standards and changes cache characteristics that CPy
Le lundi 18 septembre 2006 à 09:48 -0600, Adam Olsen a écrit :
> * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally unsupported
> by the C standards and changes cache characteristics that CPython has
> been designed with for years, likely with a very large performance
> penalty.
Has it be
I would love to contribute code for this problem. Unfortunately, I am not able to do so, but see the problem for myself and others. Therefore, I wanted to raise the question in time for Py3k. The number of responses indicates that it is not just me who struggles and there are people who know how to
On 9/18/06, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
>
> Le lundi 18 septembre 2006 à 09:48 -0600, Adam Olsen a écrit :
> > * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally unsupported
> > by the C standards and changes cache characteristics that CPython has
> > been designed with for ye
[Adam Olsen]
> I don't like the idea of a conservative GC at all in general, but
> Boehm GC seems to have very good quality, and it's easy to use from
> the point of view of a C API.
Several thoughts:
* An easier C API would significantly benefit the language in terms of
more extensions being ava
On 9/18/06, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
>
> Le lundi 18 septembre 2006 à 09:48 -0600, Adam Olsen a écrit :
> > * Bolt-on tracing GC such as Boehm-Demers-Weiser. Totally unsupported
> > by the C standards and changes cache characteristics that CPython has
> > been designed with for ye
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Sep 18, 2006, at 3:56 PM, Raymond Hettinger wrote:
> * An easier C API would significantly benefit the language in terms of
> more extensions being available and in terms of increased reliability
> for those extensions. The current refcount scheme
On 9/18/06, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> ... I agree with Raymond that it can be quite difficult to get
> C code to be refcount-correct, ...
How much of this (particularly for beginners) is remembering the
refcount affects of standard functions? Could this be avoided by just
always
On 9/18/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Adam Olsen]
> > I don't like the idea of a conservative GC at all in general, but
> > Boehm GC seems to have very good quality, and it's easy to use from
> > the point of view of a C API.
This was Marcin, not me ;)
> Several thoughts:
>
On Sep 18, 2006, at 9:56 PM, Raymond Hettinger wrote:
* I doubt the anecdotal comments about Boehm GC with respect to
performance. It may be better or it may be worse. While I think the
latter is more likely, only an implementation patch will tell the
tale.
hear, hear ;-). Other anecdoti
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Sep 18, 2006, at 4:33 PM, Jim Jewett wrote:
> On 9/18/06, Barry Warsaw <[EMAIL PROTECTED]> wrote:
>
>> ... I agree with Raymond that it can be quite difficult to get
>> C code to be refcount-correct, ...
>
> How much of this (particularly for begi
Antoine Pitrou schrieb:
> Has it been measured what cache effects reference counting entails ?
I don't think so.
> With reference counting, each object is mutable from the point of view
> of the CPU cache (refcnt is always incremented and later decremented).
> This means almost every cache line c
Raymond Hettinger schrieb:
> * An easier C API would significantly benefit the language in terms of
> more extensions being available and in terms of increased reliability
> for those extensions. The current refcount scheme results in pervasive
> refleak bugs and subsequent, interminable bughunts.
Ronald Oussoren schrieb:
> hear, hear ;-). Other anecdotical evidence says that a GC can be
> significantly faster than manual allocation, especially a copying
> collector where allocation can be really, really cheap.
OTOH, it isn't typically faster than obmalloc (which also allocates
in constant
[Raymond Hettinger]
>> * At my company, we write real-time apps that benefit from the
current
>> refcounting scheme. We would have to stick with Py2.x unless Boehm
GC
>> can be implemented without periodically killing responsiveness.
[Jim Jewett]
>Do you effectively turn off cyclic collections, (
Nick Coghlan wrote:
> I was thinking it would be easier to split out the Global Interpreter Lock
> and
> a per-interpreter Local Interpreter Lock, rather than trying to go to a full
> free-threading model. Anyone sharing other objects between interpreters would
> still need their own synchroni
Adam Olsen wrote:
> Reference counting turns all references into
> modifications.
>
> There's a few ways to approach this:
I've just thought of another one: Instead of a single
refcount per object, each thread has its own set of
refcounts. Then the object has a count of the number
of threads that
Raymond Hettinger wrote:
> * An easier C API would significantly benefit the language in terms of
> more extensions being available and in terms of increased reliability
> for those extensions. The current refcount scheme results in pervasive
> refleak bugs and subsequent, interminable bughunts.
Jim Jewett wrote:
> Would it solve this problem if there were a PyTEMPREF that magically
> treated the refcount as an automatic variable? (It increfed
> immediately, and decrefed whenever the function exited, without the
> user having to track this manually.)
This would be wrong, because most fu
We seem to have a situation where we have refcounting,
which incurs a small penalty many times, but which we're
willing to pay for the benefits it brings, and locking,
which in theory should also only have a small penalty
most of the time, not much bigger than refcounting,
but it seems we're not wi
Ronald Oussoren wrote:
> I'd be more worried about changes in semantics, it's pretty convenient
> to write 'open(somefile, 'r').read()' to read a file in bulk, currently
> this will immediately close the file but with a GC system it may be a
> long time before the file is actually closed.
A
On Sep 18, 2006, at 10:56 PM, Barry Warsaw wrote:
I don't know, I hate macros. :)
It's been a long while since I programmed on the NeXT, so Mac folks
here please chime in, but isn't there some Foundation idiom where
temporary Objective-C objects didn't need to be explicitly released
if their
Barry Warsaw wrote:
> It's been a long while since I programmed on the NeXT, so Mac folks
> here please chime in, but isn't there some Foundation idiom where
> temporary Objective-C objects didn't need to be explicitly released
> if their lifetime was exactly the duration of the function in
39 matches
Mail list logo
