Re: Threads and run time/compile time
I wish I knew why you are discussing in -internals issue on this list. You should be specifying behaviour not how it is implemented. A mention of implementation is reasonable but _don't_ spend too much time. If Larry wants it. -internals will give it to him. Anyway, please recall that because of threading concerns, the final internal form of any compiled piece of code will be as immutable as possible. So that if another thread needs to reslurp a module, the compiled form will be available. Of course, some initializations would have to be rerun, but that is minor compared to the other costs. Remember specify _as if_ it would do X. -internals will make it so. As fast as possible. (Of course some requests will not be doable and some revisitin will have to be performed but the first cut should not be too concerned.) > "SWM" == Steven W McDougall <[EMAIL PROTECTED]> writes: SWM> Based on your examples, I have to assume that you are serious about SWM> RFC1v3 item 6: SWM> 6. Modules should be loaded into the thread-global space when used SWM>[...] SWM>Subsequent threads should then reslurp these modules back in on SWM>their start up. SWM>[...] SWM>each thread needs to reuse the original modules upon creation. SWM>[...] SWM>This, of course, could lead to massive program bloat SWM> This is a non-starter for me. Right now, I am working on apps that may SWM> create 10 threads per *second*. I cannot possibly get the performance SWM> I need if every thread has to recompile all its own modules. SWM> We could either discuss alternate approaches for RFC1, or I could SWM> submit a new RFC for a thread architecture that gives me the SWM> performance I want. -- Chaim FrenkelNonlinear Knowledge, Inc. [EMAIL PROTECTED] +1-718-236-0183
Re: Threads and run time/compile time
> > We could either discuss alternate approaches for RFC1, or I could > > submit a new RFC for a thread architecture that gives me the > > performance I want. > Both are more than welcome. (If you want alternate approaches or > counter-arguments to be fully documented, then doing both in an RFC > would probably be best. Most commentaries that I'll include in later > versions of RFC 1 will be, at best, rough summaries.) I'm going to try to put together an RFC. If I can make my ideas coherent, then we'll have two proposals to compare. If I can't make my ideas coherent, then at least I'll understand why they don't work. - SWM
Re: Threads and run time/compile time
On Sun, 27 Aug 2000, Steven W McDougall wrote: > Based on your examples, I have to assume that you are serious about > RFC1v3 item 6: No offense, but I wouldn't have suggested it if I weren't serious. Misguided, perhaps. Joking, no. > This is a non-starter for me. Right now, I am working on apps that may > create 10 threads per *second*. I cannot possibly get the performance > I need if every thread has to recompile all its own modules. I certainly think that we want to get the best performance out of Perl and with threads that we can, but that performance often comes with a price. In my case, I value the overall interface more than power programming. (Mainly, because my performance requirements are often at the upper-edge of C and assembler anyways, so I don't ever expect Perl to be able to handle it beyond the prototype stage.) But let's see if we can help you out anyway. 1) How about something like a base-level thread-space cache? Something that would create a blank thread-space that would never be modified. Thread creation would involve only copying the thread-space, vice a module-list reslurp each time. 2) In addition to Threads->new(), how about Threads->clone(), which would make a copy of the parent thread, vice starting fresh? > > We could either discuss alternate approaches for RFC1, or I could > submit a new RFC for a thread architecture that gives me the > performance I want. Both are more than welcome. (If you want alternate approaches or counter-arguments to be fully documented, then doing both in an RFC would probably be best. Most commentaries that I'll include in later versions of RFC 1 will be, at best, rough summaries.) -- Bryan C. Warnock ([EMAIL PROTECTED])
Re: Threads and run time/compile time
> > as in the non-threaded case, or do we get
> >
> > $global::{foo} -> *global::foo -> &global::foo -> { print 1 }
> > $thread::{foo} -> *thread::foo -> &thread::foo -> { print 2 }
> Okay, I understand. Here's how I perceive it
>
> There is no global::foo, just two thread-specific foos. In which case,
> the eval'd foo would create (or replace) that thread's previous foo.
[examples snipped]
Based on your examples, I have to assume that you are serious about
RFC1v3 item 6:
6. Modules should be loaded into the thread-global space when used
[...]
Subsequent threads should then reslurp these modules back in on
their start up.
[...]
each thread needs to reuse the original modules upon creation.
[...]
This, of course, could lead to massive program bloat
This is a non-starter for me. Right now, I am working on apps that may
create 10 threads per *second*. I cannot possibly get the performance
I need if every thread has to recompile all its own modules.
We could either discuss alternate approaches for RFC1, or I could
submit a new RFC for a thread architecture that gives me the
performance I want.
- SWM
Re: Threads and run time/compile time
On Sat, 26 Aug 2000, Steven W McDougall wrote:
> as in the non-threaded case, or do we get
>
> $global::{foo} -> *global::foo -> &global::foo -> { print 1 }
> $thread::{foo} -> *thread::foo -> &thread::foo -> { print 2 }
>
> Does this program output
>
> 12
>
> or
>
> 11
Okay, I understand. Here's how I perceive it
There is no global::foo, just two thread-specific foos. In which case,
the eval'd foo would create (or replace) that thread's previous foo.
Examples:
package Foo::Server;
...
sub foo { print 1 }
sub hack_foo { eval 'sub foo { print 2 }' }
1;
# Example 1
#!/your/path/to/perl
use Threads;
use Foo::Server;
my $t2 = Threads->new(\&Foo::Server::hack_foo);
Foo::Server::foo();
$t2->join();
# Both the main thread and the created thread put Foo:Server
# into their thread-space. The second thread replaces its initial
# thread-space foo() with the eval'd foo(). Output = '12' (or '21').
#Example 2
#!/your/path/to/perl
use Threads;
Threads->new( sub { require Foo::Server;
Foo::Server::hack_foo(); Foo::Server::foo(); exit } );
Foo::Server::foo();
# Only the second thread has Foo::Server in its thread space.
# The main thread would get a WTFO message on the call.
# The second thread would print '2'
#Example 3
#!/your/path/to/perl
use Threads;
use Foo::Server;
eval 'sub Foo::Server::hack_foo { }' # remove hacking ability
my $t2 = Threads->new(\&Foo::Server::hack_foo);
Foo::Server::foo();
$t2->join();
# Would still print "12" or "21", as the main thread only clobbered
# its own hack_foo.
--
Bryan C. Warnock
([EMAIL PROTECTED])
Re: Threads and run time/compile time
>> Users can (and do) write open code in modules.
> I don't understand. Do you think that needs to be prevented?
No, I just want to know what happens when they do it.
Let's look at an example.
1. Non-threaded
#!/usr/local/bin/perl
sub foo { print 1 }
foo();
eval 'sub foo { print 2 }';
foo();
When the compiler sees
sub foo { print 1 }
it
1. builds an op tree for the block `{ print 1 }'
2. creates a code ref that points to the op tree
3. creates a glob that points to the code ref
4. sets $main::{foo} to the glob
Graphically:
%main::
key value
foo *main::foo
*main::foo
SCALAR -> undef
ARRAY -> undef
HASH -> undef
CODE -> &main::foo
&main::foo
OPTREE -> { print 1 }
When the interpreter sees
eval 'sub foo { print 2 }';
it calls the compiler, which
1. builds a new op tree for the block `{ print 2 }'
2. creates a new code ref that points to the new op tree
3. sets the CODE pointer in the *main::foo glob to the new code ref
4. decrements the reference count on the old code ref.
If the ref count goes to zero,
it deletes the old code ref and its op tree
Both calls to
foo();
are bound to the *main::foo glob at compile time. When the interpreter
executes a call, it follows the CODE pointer from the glob to the code
ref, follows the OPTREE pointer from the code ref to the op tree,
and finally walks the op tree.
This program outputs
12
2. Threaded
I will write `global::' vice `main::' for the global stash,
and write `thread::' for the thread local stash.
#!/usr/local/bin/perl
use Threads;
sub foo { print 1 }
foo();
Threads->new(\&hack_foo)->join;
foo();
sub hack_foo { eval 'sub foo { print 2 }' }
At (initial) compile time, the compiler sets
STASH GLOBCODE REFOP TREE
$global::{foo} -> *global::foo -> &global::foo -> { print 1 }
The question is where { print 2 } goes. Do we get
$global::{foo} -> *global::foo -> &global::foo -> { print 2 }
as in the non-threaded case, or do we get
$global::{foo} -> *global::foo -> &global::foo -> { print 1 }
$thread::{foo} -> *thread::foo -> &thread::foo -> { print 2 }
Does this program output
12
or
11
- SWM
Re: Threads and run time/compile time
On Sat, 26 Aug 2000, Steven W McDougall wrote:
> However, the distinction between compile time and run time that it
> relies on doesn't exist in Perl. For example, if we chase through
> perlfunc.pod a bit, we find
No? I'll admit that it may run through the compile and run modes
multiple times, but that is still two modes.
>
> use Module;
>
> is exactly equivalent to
>
> BEGIN { require Module; import Module; }
> and
> require Module;
>
> locates Module.pm and does a
>
> do Module.pm
>
> which is equivalent to
>
> scalar eval `cat Module.pm`;
>
> and eval is documented as
>
>eval EXPR
>
> the return value of EXPR is parsed and
Parsed = compile-time
> executed as if it were a little Perl program.
Executed = runtime.
>
> Users can (and do) write open code in modules.
> There is nothing to prevent users from writing modules like
I don't understand. Do you think that needs to be prevented?
This is perfectly acceptable, (provided its what the user wanted).
>
> # Module.pm
> use Threads;
>
> Thread->new(\&foo);
>
> sub foo { ... }
>
>
> Users can also write their own BEGIN blocks to start threads before
> the rest of the program has been compiled
>
> sub foo { ... }
>
> BEGIN { Thread->new(\&foo) }
>
>
> Going in the other direction, users can write
>
> require Foo.pm
>
> or even
>
> eval "sub foo { ... }";
>
> to compile code after the program (and other threads) have begun execution.
>
>
> Given all this, I don't think we can sequester thread creation into
> "run time". We need a model that works uniformly, no matter when
> threads are created and run.
But all the above *is* runtime.
--
Bryan C. Warnock
([EMAIL PROTECTED])
Threads and run time/compile time
RFC1v3 says
5. Threads are a runtime construct only. Lexical scoping and
compile issues are independent of any thread boundaries. The
initial interpretation is done by a single thread. use Threads may
set up the thread constructs, but threads will not be spawned
until runtime.
However, the distinction between compile time and run time that it
relies on doesn't exist in Perl. For example, if we chase through
perlfunc.pod a bit, we find
use Module;
is exactly equivalent to
BEGIN { require Module; import Module; }
and
require Module;
locates Module.pm and does a
do Module.pm
which is equivalent to
scalar eval `cat Module.pm`;
and eval is documented as
eval EXPR
the return value of EXPR is parsed and
executed as if it were a little Perl program.
Users can (and do) write open code in modules.
There is nothing to prevent users from writing modules like
# Module.pm
use Threads;
Thread->new(\&foo);
sub foo { ... }
Users can also write their own BEGIN blocks to start threads before
the rest of the program has been compiled
sub foo { ... }
BEGIN { Thread->new(\&foo) }
Going in the other direction, users can write
require Foo.pm
or even
eval "sub foo { ... }";
to compile code after the program (and other threads) have begun execution.
Given all this, I don't think we can sequester thread creation into
"run time". We need a model that works uniformly, no matter when
threads are created and run.
- SWM
