Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-15 Thread Alexander Burger 
Hi Henrik,

 The fact that Clojure enforces this way of working is seen as one of it's
 greatest virtues and strengths. If PicoLisp's boss works in the same way you
 have the same situation where you don't have to worry about inconsistent
 data.

In PicoLisp you usually don't have to worry about inconsistent data.

The threads ('task's) are strictly cooperative (not preemptive), and
database modifications by separate processes are synchronized
automatically.

However, I would not stress (or mention) 'boss' here. Typically, IPC
between PicoLisp processes, and DB modifications, are coordinated
between sister processes by the same parent (the boss) in an implicit
way. Talking to boss directly, and forcing it to do things, I regard
as quite dangerous, because the parent should be loaded as little as
possible and - more important - should not execute any application-
level code. The parent should never be allowed to crash, as this would
cut IPC between the children and give undefined results.

Instead, sister processes send messages to each other via 'tell'
(broadcasted, or to selected processes), and synchronize DB operations
via 'put!' (or 'dbSync', 'put' and 'commit' sequences), which also use
'tell' internally. 'tell' has a well-defined behavior, as it is
dispatched centrally by the parent, and the resulting events in the
children's tasks are non-preemptive again.


 (pipe (boss 'addToFoo 2))
 
 (pipe (boss 'multiplyFooWith 3))

Using pipe in that way to create a short-term child process is an
interesting idea. I would rather have done it the obvious way:

  (unless (fork)
 (boss 'addToFoo 2)
 (bye) )

But why not? 'pipe' should work here, and it is shorter. The drawback is
some system call overhead.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-15 Thread Henrik Sarvell 
Since my only experience of STM is through Clojure and threads I don't even
know if a similar locking mechanism can be implemented in PicoLisp since
here we have forked processes instead.

As usual my knowledge of Unix/Linux internals is too lacking to say anything
definite, despite having recently started reading The Art of Unix
Programming.

This section seems to imply though that it would be possible to implement
STM between forked processes
http://www.catb.org/~esr/writings/taoup/html/ch07s02.html#id2922148 (the
shared memory section) or?

Or maybe it already exists somehow implicit in PicoLisp?


On Thu, Sep 15, 2011 at 3:14 PM, Alexander Burger a...@software-lab.dewrote:

 Hi Henrik,

  The fact that Clojure enforces this way of working is seen as one of it's
  greatest virtues and strengths. If PicoLisp's boss works in the same way
 you
  have the same situation where you don't have to worry about inconsistent
  data.

 In PicoLisp you usually don't have to worry about inconsistent data.

 The threads ('task's) are strictly cooperative (not preemptive), and
 database modifications by separate processes are synchronized
 automatically.

 However, I would not stress (or mention) 'boss' here. Typically, IPC
 between PicoLisp processes, and DB modifications, are coordinated
 between sister processes by the same parent (the boss) in an implicit
 way. Talking to boss directly, and forcing it to do things, I regard
 as quite dangerous, because the parent should be loaded as little as
 possible and - more important - should not execute any application-
 level code. The parent should never be allowed to crash, as this would
 cut IPC between the children and give undefined results.

 Instead, sister processes send messages to each other via 'tell'
 (broadcasted, or to selected processes), and synchronize DB operations
 via 'put!' (or 'dbSync', 'put' and 'commit' sequences), which also use
 'tell' internally. 'tell' has a well-defined behavior, as it is
 dispatched centrally by the parent, and the resulting events in the
 children's tasks are non-preemptive again.


  (pipe (boss 'addToFoo 2))
 
  (pipe (boss 'multiplyFooWith 3))

 Using pipe in that way to create a short-term child process is an
 interesting idea. I would rather have done it the obvious way:

  (unless (fork)
 (boss 'addToFoo 2)
 (bye) )

 But why not? 'pipe' should work here, and it is shorter. The drawback is
 some system call overhead.

 Cheers,
 - Alex
 --
 UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-15 Thread Alexander Burger 
On Thu, Sep 15, 2011 at 06:22:46PM +0700, Henrik Sarvell wrote:
 This section seems to imply though that it would be possible to implement
 STM between forked processes
 http://www.catb.org/~esr/writings/taoup/html/ch07s02.html#id2922148 (the
 shared memory section) or?
 
 Or maybe it already exists somehow implicit in PicoLisp?

I would say no, though I don't fully grasp the implications and purpose
of STM. At least shared memory is definitely not supported in public
versions of PicoLisp.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-15 Thread Henrik Sarvell 
AFAIK it's really only relevant when you're dealing with computationally
heavy and long running things that need to be multi threaded/forked to be
able to utilize multiple CPU cores. Then these processes might need to
coordinate via, or put results in, a shared space.

When it comes to your usual web application I suppose it could be used to
cache things, at least that's the use case for me. However this cached data
does not need to have very high restrictions on it in my case, ie I don't
need STM features (so far).

I suppose it would be pretty easy to wrap a memory file, something like
this:

mkdir /dev/picolisp
mount -t tmpfs -o size=10M,mode=0755 tmpfs /dev/picolisp
touch /dev/picolisp/data

And then a process does touch /dev/picolisp/lock and then works on data,
after it's finished it removes the lock file.

Any other process first checks if the lock exists before working with the
data file.

Maybe slow but that's what I could come up with from my limited knowledge of
these things.



On Thu, Sep 15, 2011 at 6:44 PM, Alexander Burger a...@software-lab.dewrote:

 On Thu, Sep 15, 2011 at 06:22:46PM +0700, Henrik Sarvell wrote:
  This section seems to imply though that it would be possible to implement
  STM between forked processes
  http://www.catb.org/~esr/writings/taoup/html/ch07s02.html#id2922148 (the
  shared memory section) or?
 
  Or maybe it already exists somehow implicit in PicoLisp?

 I would say no, though I don't fully grasp the implications and purpose
 of STM. At least shared memory is definitely not supported in public
 versions of PicoLisp.

 Cheers,
 - Alex
 --
 UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-10 Thread Henrik Sarvell 
In Clojure there are a few constructs and functions to handle them that make
use of this to keep multithreaded apps sane: http://clojure.org/refs

However, in the course of developing web apps I've only used them when I
want different request to share a piece of storage that needs to be updated
very often and not written to disk all the time.

In that scenario Clojure makes everything very easy through deref and
friends.

In PicoLisp if you create *Foo and set it to say 1 child processes spawned
by http requests can read that variable just fine, if they want to update it
they can use boss, this has been discussed before:
http://www.mail-archive.com/picolisp@software-lab.de/msg01743.html

In Clojure it could work like this:
1.) Foo is 1.
2.) Thread 1 wants to change the value of Foo and this operation ends up
taking 2s and Foo will end up being 10 afterwards.
3.) While thread 1 is manipulating Foo thread 2 is spawned, now thread 2 has
to wait for 1 to finish and gets 10 as input value, thread 2 only wants to
multiply Foo with 2.
4.) Thread 2 is finished and Foo ends up being 20.

The end result is something that is expected by all users of the application
should they compare notes because user 2 who spawned thread 2 might expect
the result to be 1 * 2 = 2, however after talking with user 1 who spawned
thread 1 they realize that user 1 was in fact there first and the result is
the expected result.

The fact that Clojure enforces this way of working is seen as one of it's
greatest virtues and strengths. If PicoLisp's boss works in the same way you
have the same situation where you don't have to worry about inconsistent
data.

When I try a similar scenario in PicoLisp like this:

(load lib/boss.l)

(setq *Foo 1)

(de addToFoo (Num)
   (wait 1000)
   (setq *Foo (+ Num *Foo)) )

(de multiplyFooWith (Num)
   (setq *Foo (* Num *Foo)))

(pipe (boss 'addToFoo 2))

(pipe (boss 'multiplyFooWith 3))

(wait 1100)

(println *Foo)

(bye)

I get 5 as the output, the multiplication happens before the addition. There
is thus no inherent enforcement in boss when it comes to in-memory
variables.

However when you are manipulating database objects you can enforce by
yourself:

(de addToFoo (Num)
   (dbSync)
   (wait 1000)
   (put *Foo 'var (+ Num (; *Foo var)))
   (commit 'upd) )

(de multiplyFooWith (Num)
   (put! *Foo 'var (* Num (; *Foo var

I haven't tested the above but I'm pretty sure the end result would be 6.




On Fri, Sep 9, 2011 at 11:32 PM, Jakob Eriksson ja...@vmlinux.org wrote:

 On Fri, Sep 09, 2011 at 03:57:49PM +0200, Alexander Burger wrote:
  On Fri, Sep 09, 2011 at 03:04:21PM +0200, Jakob Eriksson wrote:
ATM I cannot see how this could be useful. PicoLisp has no threads,
 and
  
   If you defined threads less strict, such that processes could be
 threads,
   could it make sense then?
 
  Only if some shared/mapped memory is used, as far as I understand.
  What advantages would you see, and for which purpose?


 Don't know, these things are over my head, I was hoping someone here
 would know more than me and say, yes, this is good because... :-)



 
  Cheers,
  - Alex
  --
  UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
 --
 UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-10 Thread Henrik Sarvell 
End result of 9 of course.


On Sat, Sep 10, 2011 at 9:55 PM, Henrik Sarvell hsarv...@gmail.com wrote:

 In Clojure there are a few constructs and functions to handle them that
 make use of this to keep multithreaded apps sane: http://clojure.org/refs

 However, in the course of developing web apps I've only used them when I
 want different request to share a piece of storage that needs to be updated
 very often and not written to disk all the time.

 In that scenario Clojure makes everything very easy through deref and
 friends.

 In PicoLisp if you create *Foo and set it to say 1 child processes spawned
 by http requests can read that variable just fine, if they want to update it
 they can use boss, this has been discussed before:
 http://www.mail-archive.com/picolisp@software-lab.de/msg01743.html

 In Clojure it could work like this:
 1.) Foo is 1.
 2.) Thread 1 wants to change the value of Foo and this operation ends up
 taking 2s and Foo will end up being 10 afterwards.
 3.) While thread 1 is manipulating Foo thread 2 is spawned, now thread 2
 has to wait for 1 to finish and gets 10 as input value, thread 2 only wants
 to multiply Foo with 2.
 4.) Thread 2 is finished and Foo ends up being 20.

 The end result is something that is expected by all users of the
 application should they compare notes because user 2 who spawned thread 2
 might expect the result to be 1 * 2 = 2, however after talking with user 1
 who spawned thread 1 they realize that user 1 was in fact there first and
 the result is the expected result.

 The fact that Clojure enforces this way of working is seen as one of it's
 greatest virtues and strengths. If PicoLisp's boss works in the same way you
 have the same situation where you don't have to worry about inconsistent
 data.

 When I try a similar scenario in PicoLisp like this:

 (load lib/boss.l)

 (setq *Foo 1)

 (de addToFoo (Num)
(wait 1000)
(setq *Foo (+ Num *Foo)) )

 (de multiplyFooWith (Num)
(setq *Foo (* Num *Foo)))

 (pipe (boss 'addToFoo 2))

 (pipe (boss 'multiplyFooWith 3))

 (wait 1100)

 (println *Foo)

 (bye)

 I get 5 as the output, the multiplication happens before the addition.
 There is thus no inherent enforcement in boss when it comes to in-memory
 variables.

 However when you are manipulating database objects you can enforce by
 yourself:

 (de addToFoo (Num)
(dbSync)
(wait 1000)
(put *Foo 'var (+ Num (; *Foo var)))
(commit 'upd) )

 (de multiplyFooWith (Num)
(put! *Foo 'var (* Num (; *Foo var

 I haven't tested the above but I'm pretty sure the end result would be 6.





 On Fri, Sep 9, 2011 at 11:32 PM, Jakob Eriksson ja...@vmlinux.org wrote:

 On Fri, Sep 09, 2011 at 03:57:49PM +0200, Alexander Burger wrote:
  On Fri, Sep 09, 2011 at 03:04:21PM +0200, Jakob Eriksson wrote:
ATM I cannot see how this could be useful. PicoLisp has no threads,
 and
  
   If you defined threads less strict, such that processes could be
 threads,
   could it make sense then?
 
  Only if some shared/mapped memory is used, as far as I understand.
  What advantages would you see, and for which purpose?


 Don't know, these things are over my head, I was hoping someone here
 would know more than me and say, yes, this is good because... :-)



 
  Cheers,
  - Alex
  --
  UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
 --
 UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Is Transactional Memory Relevant to PicoLisp?

2011-09-09 Thread Jakob Eriksson 


As seen here. http://en.wikipedia.org/wiki/Software_transactional_memory
I noticed that Common Lisp and other Lisps have support for transactional 
memory.

I read this and became curious:  
http://arstechnica.com/hardware/news/2011/08/ibms-new-transactional-memory-make-or-break-time-for-multithreaded-revolution.ars


best regards,
Jakob


-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-09 Thread Alexander Burger 
Hi Jakob,

 As seen here. http://en.wikipedia.org/wiki/Software_transactional_memory

ATM I cannot see how this could be useful. PicoLisp has no threads, and
in the context of database transactions it seems to make no sense in
PicoLisp's object caching model.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-09 Thread Alexander Burger 
On Fri, Sep 09, 2011 at 03:04:21PM +0200, Jakob Eriksson wrote:
  ATM I cannot see how this could be useful. PicoLisp has no threads, and
 
 If you defined threads less strict, such that processes could be threads,
 could it make sense then?

Only if some shared/mapped memory is used, as far as I understand.
What advantages would you see, and for which purpose?

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Is Transactional Memory Relevant to PicoLisp?

2011-09-09 Thread Jakob Eriksson 
On Fri, Sep 09, 2011 at 03:57:49PM +0200, Alexander Burger wrote:
 On Fri, Sep 09, 2011 at 03:04:21PM +0200, Jakob Eriksson wrote:
   ATM I cannot see how this could be useful. PicoLisp has no threads, and
  
  If you defined threads less strict, such that processes could be 
  threads,
  could it make sense then?
 
 Only if some shared/mapped memory is used, as far as I understand.
 What advantages would you see, and for which purpose?


Don't know, these things are over my head, I was hoping someone here
would know more than me and say, yes, this is good because... :-)



 
 Cheers,
 - Alex
 -- 
 UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe