Re: [Pharo-users] [Pharo-dev] [Ann] Concurrent Programming in Pharo is available

[Eliot Miranda]

Hi Stef,

  here's my review/feedback

On Sun, Feb 9, 2020 at 5:05 AM Stéphane Ducasse 
> wrote:
> >
> > On http://books.pharo.org/booklet-ConcurrentProgramming/
>


In fig 1.1 there needs to be a back arrow from Executing to Runnable which
is labelled "Preempted by a higher priority process" or similar.

In the label for fig 1.2 the word "pending" should be replaced with
"runnable".

Page 8, section "Process Priorities"
"At any time only one process is executed." should read "At any time only
one process is executing."
The line starting "Next table lists..." in the last paragraph on page 8
should read "The following table lists...".
The section might want to state some thing like "there that the current
implementation of the Pharo scheduler has process priorities from 1 to
100.  Only some of these are named.  But the programmer is free to use any
priority within that range that they see fit."  It would be good to include
examples which did things like "forkAt: Processor userPriority + 1", etc.

In the label for Fig 2.1 "ressources" should read "resources".  But
throughout that section I would word it that "processes need to share
resources", but "the process is waiting to acquire the resource".  So use
the plural for the general case and the singular for the specific case.
Sentences such as this: "P0 has finished to use the resources." should read
"P0 has finished using the resource."

In Section 2.1 Conclusion
"Semaphores are the lower synchronisation mechanisms." should read
"Semaphores are the lowest level synchronisation mechanisms."

Chapter 3, "Scheduler's Principles" should state that
"The Smalltalk scheduler is a real-time, cooperative, preemptive across
priorities, non-preemptive within priorities, scheduler."

The title to Chapter 4.5 "ShareQueue: a nice semaphore example" should read
"Share*d*Queue: a nice semaphore example"

Most of the paragraphs on page 49 in section 4.5 are still in French and
need translating.


> S.
>
> 
> Stéphane Ducasse
> http://stephane.ducasse.free.fr / http://www.pharo.org
> 03 59 35 87 52
> Assistant: Julie Jonas
> FAX 03 59 57 78 50
> TEL 03 59 35 86 16
> S. Ducasse - Inria
> 40, avenue Halley,
> Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
> Villeneuve d'Ascq 59650
> France
>
>

-- 
_,,,^..^,,,_
best, Eliot

Re: [Pharo-users] P3 and concurrenty

[ducasse]

Amazing experience!
This is the power of Pharo at work.
Thanks a lot Sven.

S

> On 10 Feb 2020, at 21:24, Petter Egesund  wrote:
> 
> Perfect, this is exactly what I needed. Thank so much!!
> 
> Petter
> 
> On Mon, Feb 10, 2020 at 8:44 PM Sven Van Caekenberghe  > wrote:
> Hi Petter,
> 
> > On 10 Feb 2020, at 16:46, Petter Egesund  > > wrote:
> > 
> > Yes, I see - that sounds sensible.
> > 
> > Could it be an idea to subclass the pool-class and to prepare statements in 
> > this method, if there is a kind of init-method in the class? What do you 
> > think?
> 
> That would have indeed been one approach but for this situation I opted for a 
> configurator block.
> 
> P3ConnectionPool>>#configurator: oneArgumentBlock
>   "Set oneArgumentBlock to be my configurator.
>This is code that will be executed once on each newly created connection 
> (P3Client).
>The default configuration asserts #isWorking on the argument"
> 
> The default is
> 
>   configurator := [ :p3Client | self assert: p3Client isWorking ]
> 
> which basically does an extra validation and ensures the opening of the 
> connection.
> 
> Now, a little piece of functionality was still missing, I added it with the 
> following commit:
> 
>   
> https://github.com/svenvc/P3/commit/e3f161d9f9ae34b3b020ad924b3c0f116f68c0b0 
> 
> 
> Your usage scenario should now be like this: you create a P3ConnectionPool 
> with the proper URL, then you set your configurator to add your prepared 
> statements (you could delegate this to a helper class if you have a lot of 
> them).
> 
>   pool configurator: [ :p3Client |
> p3Client prepare: 'SELECT ...' named: 'select1'.
> p3Client prepare: 'INSET ...' named: 'insert1' ]
> 
> Later, when using a connection from the pool, you refer to your prepared 
> statements by name.
> 
>   pool withConnection: [ :p3Client |
> (p3Client preparedStatementNamed: 'select1')
>   execute: { args } ]
> 
> Sven
> 
> PS: have a look at P3PreparedStatementTest>>#testNamedPreparedStatement for 
> an operation example (minus the connection pooling).
> 
> > On Mon, Feb 10, 2020 at 4:36 PM Sven Van Caekenberghe  > > wrote:
> > 
> > 
> > > On 10 Feb 2020, at 16:30, Petter Egesund  > > > wrote:
> > > 
> > > Hi and thanks for an answer above my expectations to our problem. 
> > > 
> > > This is really awesome, I was just about to start coding a pool myself, 
> > > but getting it directly from the author of the base library is off course 
> > > way better :)
> > > 
> > > I will try out and get back as soon as we have tested.
> > > 
> > > One quick question - are the prepared statements which are connected to 
> > > the pool thread safe?
> > 
> > Prepared statements are scoped to a single connection in PSQL, AFAIK. 
> > 
> > What you will probably want to do is use a #configurator: block to set up 
> > your prepared statements once, like you do now. Then each thread/process 
> > grabs its own connection from the pool, (which will be already configured 
> > properly) and uses it, for itself, and finally returns it to the pool.
> > 
> > Interacting with the pool should be fully thread safe. Once you are using a 
> > connection, it not longer is, but is also does not have to, since one 
> > connection equals one client thread.
> > 
> > > Petter
> > > 
> > > On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe  > > > wrote:
> > > Hi Petter,
> > > 
> > > https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863
> > >  
> > > 
> > >  adds a connection pool.
> > > 
> > > Here is the class comment:
> > > 
> > > ==
> > > 
> > > I am P3ConnectionPool.
> > > 
> > > I offer a pool of shared PSQL connections (P3Client instances) as a 
> > > resource.
> > > 
> > > After configuring me with at least a url, you use me by calling 
> > > #withConnection:
> > > 
> > >   pool := P3ConnectionPool url: 'psql://sven@localhost'.
> > >   pool withConnection: [ :p3Client | 
> > >   p3Client query: 'SELECT table_name FROM 
> > > information_schema.tables' ].
> > >   pool close.
> > > 
> > > When a connection is in use, it is not part of the pool.
> > > When a connection is returned/released to the pool, it becomes available 
> > > for reuse.
> > > 
> > > The pool's size is the number of open connection ready to be reused.
> > > The pool's capacity is the maximum number of connection that will be 
> > > pooled.
> > > Excess connections will be closed when they are returned/released to the 
> > > pool.
> > > 
> > > New connections are created as needed.
> > > You can set a #configurator to further initialize new connections.
> > > You can use #warmUp or #warmUp: to precreate a number of connections.
> > > 
> > > When an error occurs, the 

Re: [Pharo-users] P3 and concurrenty

[Petter Egesund]

Perfect, this is exactly what I needed. Thank so much!!

Petter

On Mon, Feb 10, 2020 at 8:44 PM Sven Van Caekenberghe  wrote:

> Hi Petter,
>
> > On 10 Feb 2020, at 16:46, Petter Egesund 
> wrote:
> >
> > Yes, I see - that sounds sensible.
> >
> > Could it be an idea to subclass the pool-class and to prepare statements
> in this method, if there is a kind of init-method in the class? What do you
> think?
>
> That would have indeed been one approach but for this situation I opted
> for a configurator block.
>
> P3ConnectionPool>>#configurator: oneArgumentBlock
>   "Set oneArgumentBlock to be my configurator.
>This is code that will be executed once on each newly created
> connection (P3Client).
>The default configuration asserts #isWorking on the argument"
>
> The default is
>
>   configurator := [ :p3Client | self assert: p3Client isWorking ]
>
> which basically does an extra validation and ensures the opening of the
> connection.
>
> Now, a little piece of functionality was still missing, I added it with
> the following commit:
>
>
> https://github.com/svenvc/P3/commit/e3f161d9f9ae34b3b020ad924b3c0f116f68c0b0
>
> Your usage scenario should now be like this: you create a P3ConnectionPool
> with the proper URL, then you set your configurator to add your prepared
> statements (you could delegate this to a helper class if you have a lot of
> them).
>
>   pool configurator: [ :p3Client |
> p3Client prepare: 'SELECT ...' named: 'select1'.
> p3Client prepare: 'INSET ...' named: 'insert1' ]
>
> Later, when using a connection from the pool, you refer to your prepared
> statements by name.
>
>   pool withConnection: [ :p3Client |
> (p3Client preparedStatementNamed: 'select1')
>   execute: { args } ]
>
> Sven
>
> PS: have a look at P3PreparedStatementTest>>#testNamedPreparedStatement
> for an operation example (minus the connection pooling).
>
> > On Mon, Feb 10, 2020 at 4:36 PM Sven Van Caekenberghe 
> wrote:
> >
> >
> > > On 10 Feb 2020, at 16:30, Petter Egesund 
> wrote:
> > >
> > > Hi and thanks for an answer above my expectations to our problem.
> > >
> > > This is really awesome, I was just about to start coding a pool
> myself, but getting it directly from the author of the base library is off
> course way better :)
> > >
> > > I will try out and get back as soon as we have tested.
> > >
> > > One quick question - are the prepared statements which are connected
> to the pool thread safe?
> >
> > Prepared statements are scoped to a single connection in PSQL, AFAIK.
> >
> > What you will probably want to do is use a #configurator: block to set
> up your prepared statements once, like you do now. Then each thread/process
> grabs its own connection from the pool, (which will be already configured
> properly) and uses it, for itself, and finally returns it to the pool.
> >
> > Interacting with the pool should be fully thread safe. Once you are
> using a connection, it not longer is, but is also does not have to, since
> one connection equals one client thread.
> >
> > > Petter
> > >
> > > On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe 
> wrote:
> > > Hi Petter,
> > >
> > >
> https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863
> adds a connection pool.
> > >
> > > Here is the class comment:
> > >
> > > ==
> > >
> > > I am P3ConnectionPool.
> > >
> > > I offer a pool of shared PSQL connections (P3Client instances) as a
> resource.
> > >
> > > After configuring me with at least a url, you use me by calling
> #withConnection:
> > >
> > >   pool := P3ConnectionPool url: 'psql://sven@localhost'.
> > >   pool withConnection: [ :p3Client |
> > >   p3Client query: 'SELECT table_name FROM
> information_schema.tables' ].
> > >   pool close.
> > >
> > > When a connection is in use, it is not part of the pool.
> > > When a connection is returned/released to the pool, it becomes
> available for reuse.
> > >
> > > The pool's size is the number of open connection ready to be reused.
> > > The pool's capacity is the maximum number of connection that will be
> pooled.
> > > Excess connections will be closed when they are returned/released to
> the pool.
> > >
> > > New connections are created as needed.
> > > You can set a #configurator to further initialize new connections.
> > > You can use #warmUp or #warmUp: to precreate a number of connections.
> > >
> > > When an error occurs, the connection should not be reused and be
> closed by the caller.
> > >
> > > ==
> > >
> > > There are some unit tests as well.
> > >
> > > Let me know if this works for you.
> > >
> > > Sven
> > >
> > > > On 9 Feb 2020, at 19:04, Petter Egesund 
> wrote:
> > > >
> > > > Yes, thanks for good feedback.
> > > >
> > > > I will try the pooled way, I think - not primarily because of speed,
> but due to that our library is built around prepared connections.
> > > >
> > > > Petter
> > > >
> > > > On Sun, Feb 9, 2020 at 6:01 PM Sven Van Caekenberghe 
> wrote:
> > > > Hi Petter,
> > > >

Re: [Pharo-users] [Pharo-dev] [Ann] Concurrent Programming in Pharo is available

[ducasse]

You are welcome. 
This is really one of the few resources on the topic and at this level in the 
ecosystem. 
In the future I plan to learn and explain monitors because they are better than 
semaphores or mutexes
but I have the learn about them before :).


> On 10 Feb 2020, at 20:46, Todd Blanchard  wrote:
> 
> 
> From: Todd Blanchard 
> Subject: Re: [Pharo-dev] [Ann] Concurrent Programming in Pharo is available
> Date: 10 February 2020 at 20:46:46 CET
> To: Pharo Development List 
> Cc: Any question about pharo is welcome 
> 
> 
> I have wanted a clean explanation of semaphores and process scheduling for a 
> long time.
> 
> Thanks for this.
> 
> 
>> On Feb 9, 2020, at 3:58 AM, Stéphane Ducasse > > wrote:
>> 
>> On http://books.pharo.org/booklet-ConcurrentProgramming/ 
>> 
>> 
>> 
>> S.
>> 
>> 
>> Stéphane Ducasse
>> http://stephane.ducasse.free.fr  / 
>> http://www.pharo.org  
>> 03 59 35 87 52
>> Assistant: Julie Jonas 
>> FAX 03 59 57 78 50
>> TEL 03 59 35 86 16
>> S. Ducasse - Inria
>> 40, avenue Halley, 
>> Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
>> Villeneuve d'Ascq 59650
>> France
>> 
> 
> 
> 

Re: [Pharo-users] [Pharo-dev] [Ann] Concurrent Programming in Pharo is available

[Todd Blanchard via Pharo-users]

--- Begin Message ---
I have wanted a clean explanation of semaphores and process scheduling for a 
long time.

Thanks for this.


> On Feb 9, 2020, at 3:58 AM, Stéphane Ducasse  
> wrote:
> 
> On http://books.pharo.org/booklet-ConcurrentProgramming/ 
> 
> 
> 
> S.
> 
> 
> Stéphane Ducasse
> http://stephane.ducasse.free.fr  / 
> http://www.pharo.org  
> 03 59 35 87 52
> Assistant: Julie Jonas 
> FAX 03 59 57 78 50
> TEL 03 59 35 86 16
> S. Ducasse - Inria
> 40, avenue Halley, 
> Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
> Villeneuve d'Ascq 59650
> France
> 

--- End Message ---

Re: [Pharo-users] P3 and concurrenty

[Sven Van Caekenberghe]

Hi Petter,

> On 10 Feb 2020, at 16:46, Petter Egesund  wrote:
> 
> Yes, I see - that sounds sensible.
> 
> Could it be an idea to subclass the pool-class and to prepare statements in 
> this method, if there is a kind of init-method in the class? What do you 
> think?

That would have indeed been one approach but for this situation I opted for a 
configurator block.

P3ConnectionPool>>#configurator: oneArgumentBlock
  "Set oneArgumentBlock to be my configurator.
   This is code that will be executed once on each newly created connection 
(P3Client).
   The default configuration asserts #isWorking on the argument"

The default is

  configurator := [ :p3Client | self assert: p3Client isWorking ]

which basically does an extra validation and ensures the opening of the 
connection.

Now, a little piece of functionality was still missing, I added it with the 
following commit:

  https://github.com/svenvc/P3/commit/e3f161d9f9ae34b3b020ad924b3c0f116f68c0b0

Your usage scenario should now be like this: you create a P3ConnectionPool with 
the proper URL, then you set your configurator to add your prepared statements 
(you could delegate this to a helper class if you have a lot of them).

  pool configurator: [ :p3Client |
p3Client prepare: 'SELECT ...' named: 'select1'.
p3Client prepare: 'INSET ...' named: 'insert1' ]

Later, when using a connection from the pool, you refer to your prepared 
statements by name.

  pool withConnection: [ :p3Client |
(p3Client preparedStatementNamed: 'select1')
  execute: { args } ]

Sven

PS: have a look at P3PreparedStatementTest>>#testNamedPreparedStatement for an 
operation example (minus the connection pooling).

> On Mon, Feb 10, 2020 at 4:36 PM Sven Van Caekenberghe  wrote:
> 
> 
> > On 10 Feb 2020, at 16:30, Petter Egesund  wrote:
> > 
> > Hi and thanks for an answer above my expectations to our problem. 
> > 
> > This is really awesome, I was just about to start coding a pool myself, but 
> > getting it directly from the author of the base library is off course way 
> > better :)
> > 
> > I will try out and get back as soon as we have tested.
> > 
> > One quick question - are the prepared statements which are connected to the 
> > pool thread safe?
> 
> Prepared statements are scoped to a single connection in PSQL, AFAIK. 
> 
> What you will probably want to do is use a #configurator: block to set up 
> your prepared statements once, like you do now. Then each thread/process 
> grabs its own connection from the pool, (which will be already configured 
> properly) and uses it, for itself, and finally returns it to the pool.
> 
> Interacting with the pool should be fully thread safe. Once you are using a 
> connection, it not longer is, but is also does not have to, since one 
> connection equals one client thread.
> 
> > Petter
> > 
> > On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe  wrote:
> > Hi Petter,
> > 
> > https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863
> >  adds a connection pool.
> > 
> > Here is the class comment:
> > 
> > ==
> > 
> > I am P3ConnectionPool.
> > 
> > I offer a pool of shared PSQL connections (P3Client instances) as a 
> > resource.
> > 
> > After configuring me with at least a url, you use me by calling 
> > #withConnection:
> > 
> >   pool := P3ConnectionPool url: 'psql://sven@localhost'.
> >   pool withConnection: [ :p3Client | 
> >   p3Client query: 'SELECT table_name FROM 
> > information_schema.tables' ].
> >   pool close.
> > 
> > When a connection is in use, it is not part of the pool.
> > When a connection is returned/released to the pool, it becomes available 
> > for reuse.
> > 
> > The pool's size is the number of open connection ready to be reused.
> > The pool's capacity is the maximum number of connection that will be pooled.
> > Excess connections will be closed when they are returned/released to the 
> > pool.
> > 
> > New connections are created as needed.
> > You can set a #configurator to further initialize new connections.
> > You can use #warmUp or #warmUp: to precreate a number of connections.
> > 
> > When an error occurs, the connection should not be reused and be closed by 
> > the caller.
> > 
> > ==
> > 
> > There are some unit tests as well.
> > 
> > Let me know if this works for you.
> > 
> > Sven
> > 
> > > On 9 Feb 2020, at 19:04, Petter Egesund  wrote:
> > > 
> > > Yes, thanks for good feedback.
> > > 
> > > I will try the pooled way, I think - not primarily because of speed, but 
> > > due to that our library is built around prepared connections.
> > > 
> > > Petter
> > > 
> > > On Sun, Feb 9, 2020 at 6:01 PM Sven Van Caekenberghe  wrote:
> > > Hi Petter,
> > > 
> > > > On 9 Feb 2020, at 17:27, Petter Egesund  
> > > > wrote:
> > > > 
> > > > Hi Sven and thanks for answering!
> > > > 
> > > > I use Teapot with one common sql-connecton, not one for each user 
> > > > session. At startup I create several sql statements and these does not 
> > > > 

Re: [Pharo-users] Dictionary removeKey: very low performance

[Kasper Østerbye]

Hi RIchard,

On 9 February 2020 at 17.54.30, Richard O'Keefe (rao...@gmail.com) wrote:

My library uses separate chaining
(https://en.wikipedia.org/wiki/Hash_table#Separate_chaining)
which makes deletion simple and fast and allows 'null' keys.

Does your implementation use a fixed number of buckets, or do they grow
(and shrink)?

I had been wondering about doing an open addressing with items marked as
deleted to see how it would perform.

Aka "Another technique for removal is simply to mark the slot as deleted.
However this eventually requires rebuilding the table simply to remove
deleted records.” from https://en.wikipedia.org/wiki/Open_addressing
.

Best,

Kasper

Re: [Pharo-users] About "it's not pharo but smalltalk"

[Sean P. DeNigris]

TedVanGaalen wrote
> I am only suggesting that Pharo should be downward compatible
> (that is, within Pharo's scope only). 

I agree that this is a worthy ideal, but IMHO is not realistic with the
current resources (time and staff). All that additional code would have to
be maintained. In the worst case, entire subsystems would have to remain in
the image - we changed to Calypso browser but Nautilus would still have to
lurk around in case someone hooked in. For limited cases, there could be a
compatibility layer, but again, we are struggling to implement our top
priority dreams, and don't have the luxury of extra resources available for
worthy but lower-priority things like this. Of course anyone - or group -
for which this is a higher priority, could do just that. It would be like
Grease, but for Ph2Ph I guess...



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] About "it's not pharo but smalltalk"

[Sean P. DeNigris]

Trygve Reenskaug wrote
> I have recently completed a conceptual model with tools for a new way of 
> programming for novices.

Hi trygve, what is the project and how can I install it? I try to follow all
your work...


Trygve Reenskaug wrote
> the port will probably be outdated 
> and useless by the time it's finished.

I wouldn't say that's true in my experience updating projects from Pharo 1.x
-> 8.0.1 as each version came out. While it can be difficult to do a big
jump, porting from one to the next has usually been fairly straightforward.
So if you did port it (and maybe I can help), it shouldn't be too
outdated/hard-to-maintain. That said, a Morphic replacement has been on the
burner for quite a while and I assume there's a heavy graphical component,
so that would have to be redone. However, this is a problem *everyone* will
face together so I am sure there will be plenty of support...




-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] The expanding Pharo ecosystem

[Serge Stinckwich]

And don't forget to star your favorite Pharo projects!

On Mon, Feb 10, 2020 at 5:58 PM Sean P. DeNigris 
wrote:

> Sven Van Caekenberghe-2 wrote
> > Topic tags automatically organise some projects:
> >
> >   https://github.com/topics/pharo
> >
> >   https://github.com/topics/pharo-smalltalk
>
> Yes, please everyone remember to use these tags for your Pharo projects.
> Like this we get a poor-man's catalog for free ;)
>
>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

-- 
Serge Stinckwic
​h​

Int. Research Unit
 on Modelling/Simulation of Complex Systems (UMMISCO)
​Sorbonne University
 (SU)
French National Research Institute for Sustainable Development (IRD)​
U
​niversity of Yaoundé I​, Cameroon
"Programs must be written for people to read, and only incidentally for
machines to execute."
https://twitter.com/SergeStinckwich
​

Re: [Pharo-users] The expanding Pharo ecosystem

[Sean P. DeNigris]

Sven Van Caekenberghe-2 wrote
> Topic tags automatically organise some projects:
> 
>   https://github.com/topics/pharo
> 
>   https://github.com/topics/pharo-smalltalk

Yes, please everyone remember to use these tags for your Pharo projects.
Like this we get a poor-man's catalog for free ;)



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] P3 and concurrenty

[Petter Egesund]

Yes, I see - that sounds sensible.

Could it be an idea to subclass the pool-class and to prepare statements in
this method, if there is a kind of init-method in the class? What do you
think?

Petter

On Mon, Feb 10, 2020 at 4:36 PM Sven Van Caekenberghe  wrote:

>
>
> > On 10 Feb 2020, at 16:30, Petter Egesund 
> wrote:
> >
> > Hi and thanks for an answer above my expectations to our problem.
> >
> > This is really awesome, I was just about to start coding a pool myself,
> but getting it directly from the author of the base library is off course
> way better :)
> >
> > I will try out and get back as soon as we have tested.
> >
> > One quick question - are the prepared statements which are connected to
> the pool thread safe?
>
> Prepared statements are scoped to a single connection in PSQL, AFAIK.
>
> What you will probably want to do is use a #configurator: block to set up
> your prepared statements once, like you do now. Then each thread/process
> grabs its own connection from the pool, (which will be already configured
> properly) and uses it, for itself, and finally returns it to the pool.
>
> Interacting with the pool should be fully thread safe. Once you are using
> a connection, it not longer is, but is also does not have to, since one
> connection equals one client thread.
>
> > Petter
> >
> > On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe 
> wrote:
> > Hi Petter,
> >
> >
> https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863
> adds a connection pool.
> >
> > Here is the class comment:
> >
> > ==
> >
> > I am P3ConnectionPool.
> >
> > I offer a pool of shared PSQL connections (P3Client instances) as a
> resource.
> >
> > After configuring me with at least a url, you use me by calling
> #withConnection:
> >
> >   pool := P3ConnectionPool url: 'psql://sven@localhost'.
> >   pool withConnection: [ :p3Client |
> >   p3Client query: 'SELECT table_name FROM
> information_schema.tables' ].
> >   pool close.
> >
> > When a connection is in use, it is not part of the pool.
> > When a connection is returned/released to the pool, it becomes available
> for reuse.
> >
> > The pool's size is the number of open connection ready to be reused.
> > The pool's capacity is the maximum number of connection that will be
> pooled.
> > Excess connections will be closed when they are returned/released to the
> pool.
> >
> > New connections are created as needed.
> > You can set a #configurator to further initialize new connections.
> > You can use #warmUp or #warmUp: to precreate a number of connections.
> >
> > When an error occurs, the connection should not be reused and be closed
> by the caller.
> >
> > ==
> >
> > There are some unit tests as well.
> >
> > Let me know if this works for you.
> >
> > Sven
> >
> > > On 9 Feb 2020, at 19:04, Petter Egesund 
> wrote:
> > >
> > > Yes, thanks for good feedback.
> > >
> > > I will try the pooled way, I think - not primarily because of speed,
> but due to that our library is built around prepared connections.
> > >
> > > Petter
> > >
> > > On Sun, Feb 9, 2020 at 6:01 PM Sven Van Caekenberghe 
> wrote:
> > > Hi Petter,
> > >
> > > > On 9 Feb 2020, at 17:27, Petter Egesund 
> wrote:
> > > >
> > > > Hi Sven and thanks for answering!
> > > >
> > > > I use Teapot with one common sql-connecton, not one for each user
> session. At startup I create several sql statements and these does not seem
> to be usable from different Teapot request at the same time.
> > >
> > > It is useable, but not concurrently.
> > >
> > > > I could create one connection pr. session and then close the
> database connection when the user leaves, but then I also would need to
> create all the prepared sql-statements for each session, which does not
> sound right to me?
> > >
> > > Yes and no, see further.
> > >
> > > > It seems I have will have to look into the other solution, and see
> if I can use some mutex stuff to avoid several tasks acessessing the same
> resources at the same time.
> > >
> > > Yes, follow the pointer that I gave you, it is not hard.
> > >
> > > > Creating a pool sounds like the right solution to me now - any
> meaning about this?
> > >
> > > Yes, you could create a connection pool. But that is harder than it
> sounds: what is the minimum size, the maximum size, what do you do when you
> go over it, how do you make sure that a resource (connection) is clean when
> returning it to the pool (given authentication, possible errors), ...
> > >
> > > I don't know what you are doing, but I think you focus too much on
> performance issues. I would first try to get the code correct and worry
> about performance later on.
> > >
> > > Running
> > >
> > >   P3ClientTest new runBenchmark1Bench.
> > >
> > > on my machine gives me a BenchmarkResult(67 iterations in 5 seconds 57
> milliseconds. 13.249 per second). This is a query that returns 10.000
> records with 5 columns. It is reusing the same client/connection for all
> iterations.
> > >
> > > If I 

Re: [Pharo-users] P3 and concurrenty

[Sven Van Caekenberghe]

> On 10 Feb 2020, at 16:30, Petter Egesund  wrote:
> 
> Hi and thanks for an answer above my expectations to our problem. 
> 
> This is really awesome, I was just about to start coding a pool myself, but 
> getting it directly from the author of the base library is off course way 
> better :)
> 
> I will try out and get back as soon as we have tested.
> 
> One quick question - are the prepared statements which are connected to the 
> pool thread safe?

Prepared statements are scoped to a single connection in PSQL, AFAIK. 

What you will probably want to do is use a #configurator: block to set up your 
prepared statements once, like you do now. Then each thread/process grabs its 
own connection from the pool, (which will be already configured properly) and 
uses it, for itself, and finally returns it to the pool.

Interacting with the pool should be fully thread safe. Once you are using a 
connection, it not longer is, but is also does not have to, since one 
connection equals one client thread.

> Petter
> 
> On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe  wrote:
> Hi Petter,
> 
> https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863 
> adds a connection pool.
> 
> Here is the class comment:
> 
> ==
> 
> I am P3ConnectionPool.
> 
> I offer a pool of shared PSQL connections (P3Client instances) as a resource.
> 
> After configuring me with at least a url, you use me by calling 
> #withConnection:
> 
>   pool := P3ConnectionPool url: 'psql://sven@localhost'.
>   pool withConnection: [ :p3Client | 
>   p3Client query: 'SELECT table_name FROM information_schema.tables' 
> ].
>   pool close.
> 
> When a connection is in use, it is not part of the pool.
> When a connection is returned/released to the pool, it becomes available for 
> reuse.
> 
> The pool's size is the number of open connection ready to be reused.
> The pool's capacity is the maximum number of connection that will be pooled.
> Excess connections will be closed when they are returned/released to the pool.
> 
> New connections are created as needed.
> You can set a #configurator to further initialize new connections.
> You can use #warmUp or #warmUp: to precreate a number of connections.
> 
> When an error occurs, the connection should not be reused and be closed by 
> the caller.
> 
> ==
> 
> There are some unit tests as well.
> 
> Let me know if this works for you.
> 
> Sven
> 
> > On 9 Feb 2020, at 19:04, Petter Egesund  wrote:
> > 
> > Yes, thanks for good feedback.
> > 
> > I will try the pooled way, I think - not primarily because of speed, but 
> > due to that our library is built around prepared connections.
> > 
> > Petter
> > 
> > On Sun, Feb 9, 2020 at 6:01 PM Sven Van Caekenberghe  wrote:
> > Hi Petter,
> > 
> > > On 9 Feb 2020, at 17:27, Petter Egesund  wrote:
> > > 
> > > Hi Sven and thanks for answering!
> > > 
> > > I use Teapot with one common sql-connecton, not one for each user 
> > > session. At startup I create several sql statements and these does not 
> > > seem to be usable from different Teapot request at the same time.
> > 
> > It is useable, but not concurrently.
> > 
> > > I could create one connection pr. session and then close the database 
> > > connection when the user leaves, but then I also would need to create all 
> > > the prepared sql-statements for each session, which does not sound right 
> > > to me?
> > 
> > Yes and no, see further.
> > 
> > > It seems I have will have to look into the other solution, and see if I 
> > > can use some mutex stuff to avoid several tasks acessessing the same 
> > > resources at the same time.
> > 
> > Yes, follow the pointer that I gave you, it is not hard.
> > 
> > > Creating a pool sounds like the right solution to me now - any meaning 
> > > about this?
> > 
> > Yes, you could create a connection pool. But that is harder than it sounds: 
> > what is the minimum size, the maximum size, what do you do when you go over 
> > it, how do you make sure that a resource (connection) is clean when 
> > returning it to the pool (given authentication, possible errors), ...
> > 
> > I don't know what you are doing, but I think you focus too much on 
> > performance issues. I would first try to get the code correct and worry 
> > about performance later on.
> > 
> > Running
> > 
> >   P3ClientTest new runBenchmark1Bench.
> > 
> > on my machine gives me a BenchmarkResult(67 iterations in 5 seconds 57 
> > milliseconds. 13.249 per second). This is a query that returns 10.000 
> > records with 5 columns. It is reusing the same client/connection for all 
> > iterations.
> > 
> > If I modify this slightly to use a new client/connection each time, like 
> > this
> > 
> >   [ 
> > (P3Client url: 'psql://sven@localhost')
> >   query: 'SELECT * FROM benchmark1'; 
> >   close 
> >   ] benchFor: 5 seconds
> > 
> > I get a BenchmarkResult(65 iterations in 5 seconds 34 milliseconds. 12.912 
> > per second) which almost as fast. Of course, 

Re: [Pharo-users] P3 and concurrenty

[Petter Egesund]

Hi and thanks for an answer above my expectations to our problem.

This is really awesome, I was just about to start coding a pool myself, but
getting it directly from the author of the base library is off course way
better :)

I will try out and get back as soon as we have tested.

One quick question - are the prepared statements which are connected to the
pool thread safe?

Petter

On Mon, Feb 10, 2020 at 12:13 PM Sven Van Caekenberghe  wrote:

> Hi Petter,
>
>
> https://github.com/svenvc/P3/commit/a6b409d0d92cb92bf9b44452908bb9033523b863
> adds a connection pool.
>
> Here is the class comment:
>
> ==
>
> I am P3ConnectionPool.
>
> I offer a pool of shared PSQL connections (P3Client instances) as a
> resource.
>
> After configuring me with at least a url, you use me by calling
> #withConnection:
>
>   pool := P3ConnectionPool url: 'psql://sven@localhost'.
>   pool withConnection: [ :p3Client |
>   p3Client query: 'SELECT table_name FROM
> information_schema.tables' ].
>   pool close.
>
> When a connection is in use, it is not part of the pool.
> When a connection is returned/released to the pool, it becomes available
> for reuse.
>
> The pool's size is the number of open connection ready to be reused.
> The pool's capacity is the maximum number of connection that will be
> pooled.
> Excess connections will be closed when they are returned/released to the
> pool.
>
> New connections are created as needed.
> You can set a #configurator to further initialize new connections.
> You can use #warmUp or #warmUp: to precreate a number of connections.
>
> When an error occurs, the connection should not be reused and be closed by
> the caller.
>
> ==
>
> There are some unit tests as well.
>
> Let me know if this works for you.
>
> Sven
>
> > On 9 Feb 2020, at 19:04, Petter Egesund 
> wrote:
> >
> > Yes, thanks for good feedback.
> >
> > I will try the pooled way, I think - not primarily because of speed, but
> due to that our library is built around prepared connections.
> >
> > Petter
> >
> > On Sun, Feb 9, 2020 at 6:01 PM Sven Van Caekenberghe 
> wrote:
> > Hi Petter,
> >
> > > On 9 Feb 2020, at 17:27, Petter Egesund 
> wrote:
> > >
> > > Hi Sven and thanks for answering!
> > >
> > > I use Teapot with one common sql-connecton, not one for each user
> session. At startup I create several sql statements and these does not seem
> to be usable from different Teapot request at the same time.
> >
> > It is useable, but not concurrently.
> >
> > > I could create one connection pr. session and then close the database
> connection when the user leaves, but then I also would need to create all
> the prepared sql-statements for each session, which does not sound right to
> me?
> >
> > Yes and no, see further.
> >
> > > It seems I have will have to look into the other solution, and see if
> I can use some mutex stuff to avoid several tasks acessessing the same
> resources at the same time.
> >
> > Yes, follow the pointer that I gave you, it is not hard.
> >
> > > Creating a pool sounds like the right solution to me now - any meaning
> about this?
> >
> > Yes, you could create a connection pool. But that is harder than it
> sounds: what is the minimum size, the maximum size, what do you do when you
> go over it, how do you make sure that a resource (connection) is clean when
> returning it to the pool (given authentication, possible errors), ...
> >
> > I don't know what you are doing, but I think you focus too much on
> performance issues. I would first try to get the code correct and worry
> about performance later on.
> >
> > Running
> >
> >   P3ClientTest new runBenchmark1Bench.
> >
> > on my machine gives me a BenchmarkResult(67 iterations in 5 seconds 57
> milliseconds. 13.249 per second). This is a query that returns 10.000
> records with 5 columns. It is reusing the same client/connection for all
> iterations.
> >
> > If I modify this slightly to use a new client/connection each time, like
> this
> >
> >   [
> > (P3Client url: 'psql://sven@localhost')
> >   query: 'SELECT * FROM benchmark1';
> >   close
> >   ] benchFor: 5 seconds
> >
> > I get a BenchmarkResult(65 iterations in 5 seconds 34 milliseconds.
> 12.912 per second) which almost as fast. Of course, for smaller queries,
> the connect/disconnect overhead will be more significant.
> >
> > And note that this is not using prepared statements.
> >
> > So I would start by opening/closing a connection each time you need it.
> >
> > HTH,
> >
> > Sven
> >
> > > Petter
> > >
> > >
> > > On Sun, Feb 9, 2020 at 4:52 PM Sven Van Caekenberghe 
> wrote:
> > > Hi Petter,
> > >
> > > [ CC-ing the Pharo Users list ]
> > >
> > > P3Client is not built/designed to be used by multiple processes
> concurrently. Each database connection is represented by an instance of
> P3Client and holds some state both at the client as well as at the server
> side.
> > >
> > > Typically, in a multi user server application, each connection should
> have its own 

[Pharo-users] [ANN] Pharo TechTalk Feb 20: The Colony

[Marcus Denker]

A regular chat about Pharo. Happens on Discord.

This month a bit a special TechTalk, not about Pharo but about a tool that 
might be interesting for us to use.

Topic: The Colony https://colony.io

Introduction to "The Colony", a new platform to Organize and 
Incentivize Collaborators, Contributors, and Communities.
Short demo, followed by some Q, help to get people onboarded.


https://association.pharo.org/event-3697011
 
20 Feb 2020 •  
1:30 PM - 2:30 PM (EST)

NOTE: this is EST time, for Europe, this is 19:30 (the easiest is to download 
the calendar entry)

Re: [Pharo-users] Voluntarily cancelling requests ("applying an expiration date")

[Sven Van Caekenberghe]

Hi Holger,

That is a complicated story ;-)

But, you running out of external semaphores means that you are using too many 
sockets, are not closing/releasing them (in time) and/or your GC does not run 
enough to keep up (it is easy to deplete the external semaphore table without 
the GC kicking in).

You must have a loop somewhere that goes too fast and maybe does not clean up 
properly while doing so.

YMMV, but I do similar things -- implement/offer REST services that call other 
REST/network services, all with timeouts, in several variations, for years, and 
I do not have problems like you describe.

I would suggest enabling logging so that you can see better where the 
allocations happen and if your cleanup code does its work.

Sven

PS: Zinc logging is easy, just do 

  ZnLogEvent logToTranscript

> On 9 Feb 2020, at 16:31, Holger Freyther  wrote:
> 
> tl;dr: I am searching for a pattern (later code) to apply expiration to 
> operations.
> 
> 
> 
> Introduction:
> 
> One nice aspect of Mongodb is that it has built-in data distribution[1] and 
> configurable retention[2]. The upstream project has a document called "Server 
> Discovery and Monitoring (SDAM)", defining how a client should behave. Martin 
> Dias is currently implementing SDAM in MongoTalk/Voyage and I took it on a 
> test drive.
> 
> 
> Behavior:
> 
> My software stack is using Zinc, Zinc-REST, Voyage and Mongo. When a new REST 
> requests arrives I am using Voyage (e.g. >>#selectOne:) which will use 
> MongoTalk. The MongoTalk code needs to select the right server. It's 
> currently done by waiting for a result.
> 
> Next I started to simulate database outages. The rest clients retried when 
> not receiving a result within two seconds (no back-off/jitter). What happened 
> was roughly the following:
> 
> 
> [
>   1.) ZnServer accepts a new connection
>   2.) MongoTalk waits for a server longer than 2s
>   "nothing.. the above waits..."
> ] repeat.
> 
> 
> 
> 
> Problem:
> 
> What happened next surprised me. I expected to have a bad time when the 
> database recovers and all the stale (remember the REST clients already gave 
> up and closed the socket) requests will be answered. Instead my image crashed 
> early in my test as the ExternalSemaphoreTable was full.
> 
> Let's focus on the timeout behavior and discuss the existence of the 
> ExternalSemaphoreTable and the number of entries separately at a different 
> time.
> 
> 
> 
> 
> To me the two main problems I see are:
> 
> 
> 1.) Lack of back-pressure for ZnManagingMultiThreadedServer
> 
> 2.) Disconnect of time between the Application Layer handling REST is allowed 
> to take and down the stack how long MongoTalk may sleep and wait for a server.
> 
> 
> The first item is difficult. Even answering HTTP 500 when we are out of space 
> in the ExternalSemaphore is difficult... Let's ignore this for now as well.
> 
> 
> 
> 
> 
> 
> What I look for:
> 
> 
> 1.) Voluntarily Timeout
> 
> Inside my Application code I would like to tag an operation with a timeout. 
> This means everything that is done should complete within X seconds. It can 
> be used on a voluntarily basis.
> 
> 
>>> #lookupPerson
> 
>   "We expect all database operations to complete within two seconds"
>   person := ComputeContext current withTimeout: 2 seconds during: [
>   repository selectOne: Person where: [:each name | ...],
>   ].
> 
> 
> 
> MongoTalk>>stuff
>  "See if the outer context timeout has expired and signal. E.g. before writing
>  something into the socket to keep consistency."
>  ComputeContext current checkExpired.
> 
> 
> MongoTalk>>other
>  "Sleep for up to the remaining time out
>  (someSemaphore waitTimeoutContext: ComputeContext current) ifFalse: [
> SomethingExpired signal.
>  ]
> 
> 
> 
> 2.) Cancellation
> 
> 
> More difficult to write in pseudo code (without TaskIt?). In my above case we 
> are waiting for the database to be ready while the client already closed the 
> file descriptor. Now we are not able to see this until much later.
> 
> The idea is that in addition to the timeout we can pass a block that is 
> called when an operation should be cancelled and the ComputeContext can be 
> checked if something has been cancelled?
> 
> 
> 
> 
> The above takes inspiration from Go's context package[3]. In Go the context 
> should be passed as parameter but we could make it a Process variable?
> 
> 
> 
> 
> 
> Question:
> 
> How do you handle this in your systems? Is this something we can consider for 
> Pharo9? 
> 
> 
> 
> thanks
>   holger
> 
> 
> 
> 
> 
> 
> 
> 
> [1] It has the concept of "replicationSet" and works by having a primary, 
> secondary and arbiters running.
> [2] For every write one can configure if the write should succeed immediately 
> (before it is even on disk) or when it has been written to multiple stores 
> (e.g. majority, US and EMEA)
> [3] https://golang.org/pkg/context/
> 
> 
> 

Re: [Pharo-users] The results are in!

[Guillermo Polito]

Hi Ben,

> El 9 feb 2020, a las 21:01, Ben Coman  escribió:
> 
> > This list is for people to *ask* questions about Pharo. Not to diffuse any 
> > kind of propaganda (pharo related or not).
> 
> This list is not just for asking questions.  There are plenty of [ANN] posts 
> advertising new libraries 
> and blogs articles.  Following completion of the competition, the MIT 
> licensed code repository will be announced here. 
> Hopefully it may provide an engaging exercise for Pharo teachers to their 
> with their students.

I know, and I don’t mind announcements from time to time.
But announcements are generally marked with [ANN], and are kind of self 
explanatory.
The mail Richard sent is, and I’m sorry to be harsh, spam.

> > I have the feeling this list has been lately flooded with such off-topic 
> > messages.
> 
> Unfortunately the original post was quite bare and failed to show how it was 
> Pharo-topical.
> So I can understand how it triggered you.  I hope the additional background 
> info provided here alleviates some of your concern about it.

Maybe. Maybe Richard could work his emails a bit more?

I’ll quote his original email:

> > El 8 feb 2020, a las 19:12, Richard Kenneth Eng  > > escribió:
> >
> > Round 1 — #1 Leading Team: https://youtu.be/QWHeN5WXfBQ 
> > 
> >
> > I'm actually quite amazed by their effort. They surpassed my expectations.
> >
> > At the risk of sounding immodest, I think this is a terrific way to promote 
> > Smalltalk (Pharo). I think the video is an absolute blast.
> >
> > Richard

I particularly, who hasn’t followed the competition, am completely lost with 
this email.
With the risk of sounding harsh, again, but trying to give some feedback 
Richard can work on:
- The subject does not imply it is an announcement, nor what the results are 
related to or how they got there
- For those that are not following this competition: what is this round 1? Are 
there many of them? What is this about? are there many teams?
- There is no single link to the competition website, what this is about, 
whoever contributed to it...
- Clicking on the link gives a video with a title that is as ambiguous as this 
email :(
- I check the description of the video: what is there in the video? not clear. 
How will it appear in a youtube search in the future?
   At least it seems there are links to the website...
   Yet, Really? do I have to watch the video before I can know if this is 
interesting for me or not? :P

Happily these points could be addressed in the future.

> So with this additional perspective, may one post a week be allowed to share 
> the progress of the competition?

Can the posts have better quality? :/

Still, I’d feel more confortable with an alternative, such as using discord.
What about having a dedicated #jmrpc channel on discord where people could ask 
questions too?

What about twitter? I havent seen this on twitter. Is it there?

Now, I already spent too much on this :)
Time to get back to something productive.

Cheers,
Guille

Re: [Pharo-users] The results are in!

[Ben Coman]

On Mon, 10 Feb 2020 at 08:01, horrido  wrote:

>
> > Although the "smalltalk" title of the video is not ideal, that was the
> > name of the competition when it was originally going to use VisualWorks,
> > and so it carried over.
>
> Not only that, but if you look at jrmpc.ca, it uses the word "Smalltalk"
> all
> over the place. The sponsors include LabWare (who doesn't use Pharo),
> Simberon (who doesn't use Pharo), and TSUG (Toronto *Smalltalk* User
> Group).
> It's "Smalltalk" everywhere!


That is what it is. Can't change it.  The sponsors are being very generous.


And that's fine because Pharo is a Smalltalk.
>

But pushing that line here may generate further controversy around the
competition that will not help anyones cause.
There are other forums you can do that.  Here lets focus on the "Pharo"
aspects of the competition. :)

cheers -ben