Re: [Pharo-dev] [PROVENANCE INTERNET] Re: FileSystem Permissions

[David T. Lewis]

On Tue, Jul 02, 2013 at 04:05:18PM +0200, Camillo Bruni wrote:
 
 On 2013-07-02, at 16:05, Goubier Thierry thierry.goub...@cea.fr wrote:
 
  
  
  Le 02/07/2013 15:49, Goubier Thierry a ?crit :
  Done:
  
  SLICE-Issue-11102-FileSystemError-Path--root-ThierryGoubier.1
  
  In http://ss3.gemstone.com/ss/PharoInbox
  
  (It's for 2.0 at the moment. I'll try on 3.0 to port the same code).
  
  Oh, it has a problem with utf8 :(
 
 and most probably it won't work under OSX.
 
 I think you have to implement these primitives on the different platform 
 specific stores,
 that implies a separate version for win/mac/linux

Both the concept and the implementation will be different on Windows,
but for Unix and OS X you can use this for reference:

UnixOSProcessPluginprimitiveFileStat
Call stat(2) to obtain the file protection mask for a file. Answer 
errno on failure,
or on success answer an array with: UID with: GID with: protectionMask. 
The 
protectionMask is an Array of four integers representing the protection 
mask, or
answer errno on failure. The protection mask is four Integers, each of 
which may
be considered an octal digit (0-7), with bit values 4, 2, and 1. The 
first digit selects
the set user ID (4) and set group ID (2) and save text image (1) 
attributes. The second
digit selects permissions for the user who owns the file: read (4), 
write (2), and
execute (1); the third selects permissions for other users in the 
file's group, with
the same values; and the fourth for other users not in the file's 
group, with the
same values.

For example:

  OSProcess accessor fileStat: '/etc/hosts'

Dave

Re: [Pharo-dev] [PROVENANCE INTERNET] Re: FileSystem Permissions

[David T. Lewis]

Hi Thierry,

You were right in asking for stat/fstat. The unix stat() call provides the
uid and gid of the file, so those values are provided by primitiveFileStat.
In order to determine if the file is e.g. readable, you need that information
along with the file protection mask (also provided by primitiveFileStat),
and of course you need to know the uid and gid of your VM process, see
primitiveGetUid, primitiveGetGid, primitiveGetEUid and primitiveGetEGid
in UnixOSProcessPlugin.

There are also some unit tests that show how this works, for example:

  UnixProcessAccessorTestCasetestIsReadableForUserInGroup


I'm not sure if this helps with your original question, but maybe it
will help as a reference.

Dave


On Tue, Jul 02, 2013 at 04:46:25PM +0200, Goubier Thierry wrote:
 Hi Lewis,
 
 we have already this in a way; Pharo provides the permissions as per 
 stat in fact, but probably not the uid/guid of the file which would 
 allow for a real check of the permissions.
 
 (that is, the current (wrong) situation is that isReadable returns 
 ownerCanRead, which has a strong chance of being wrong for a directory 
 owned by root, as you can guess)
 
 As you can see, I was wrong in asking for stat/fstat... But thanks for 
 the info.
 
 Thierry
 
 Le 02/07/2013 16:32, David T. Lewis a ?crit :
 On Tue, Jul 02, 2013 at 04:05:18PM +0200, Camillo Bruni wrote:
 
 On 2013-07-02, at 16:05, Goubier Thierry thierry.goub...@cea.fr wrote:
 
 
 
 Le 02/07/2013 15:49, Goubier Thierry a ?crit :
 Done:
 
 SLICE-Issue-11102-FileSystemError-Path--root-ThierryGoubier.1
 
 In http://ss3.gemstone.com/ss/PharoInbox
 
 (It's for 2.0 at the moment. I'll try on 3.0 to port the same code).
 
 Oh, it has a problem with utf8 :(
 
 and most probably it won't work under OSX.
 
 I think you have to implement these primitives on the different platform 
 specific stores,
 that implies a separate version for win/mac/linux
 
 Both the concept and the implementation will be different on Windows,
 but for Unix and OS X you can use this for reference:
 
 UnixOSProcessPluginprimitiveFileStat
  Call stat(2) to obtain the file protection mask for a file. Answer 
  errno on failure,
  or on success answer an array with: UID with: GID with: 
  protectionMask. The 
  protectionMask is an Array of four integers representing the 
  protection mask, or
  answer errno on failure. The protection mask is four Integers, each 
  of which may
  be considered an octal digit (0-7), with bit values 4, 2, and 1. The 
  first digit selects
  the set user ID (4) and set group ID (2) and save text image (1) 
  attributes. The second
  digit selects permissions for the user who owns the file: read (4), 
  write (2), and
  execute (1); the third selects permissions for other users in the 
  file's group, with
  the same values; and the fourth for other users not in the file's 
  group, with the
  same values.
 
 For example:
 
OSProcess accessor fileStat: '/etc/hosts'
 
 Dave
 
 
 
 
 
 -- 
 Thierry Goubier
 CEA list
 Laboratoire des Fondations des Syst?mes Temps R?el Embarqu?s
 91191 Gif sur Yvette Cedex
 France
 Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Tue, Sep 03, 2013 at 10:58:07AM +0200, Goubier Thierry wrote:
 
 At the same time, parts of OSProcess seems to not be working under 
 Pharo2 anyway :( I don't even think I'm able to run the tests (locked up 
 my 3.0 image it did).
 

My last set of updates to OSProcess for Pharo were done in January 2013,
and it worked at that time. Has something stopped working since then?


 Looks like a version / implementation test would be the way to go. I'll 
 write something which should work on 2.0 / 3.0, and failure protection 
 to fallback for anything else.
 
 Is there a standard way to test for implementation/version on all Squeak 
 and Pharo versions ?
 

No. And as you can see from the examples in OSProcess, it is becoming
increasingly difficult to cobble up a solution that works for an externally
maintained package. If you can find a better solution, that would be great :)

Dave

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Tue, Sep 03, 2013 at 01:15:01PM +0200, Goubier Thierry wrote:
 
 
 Le 03/09/2013 12:56, David T. Lewis a ?crit :
 On Tue, Sep 03, 2013 at 10:58:07AM +0200, Goubier Thierry wrote:
 
 At the same time, parts of OSProcess seems to not be working under
 Pharo2 anyway :( I don't even think I'm able to run the tests (locked up
 my 3.0 image it did).
 
 
 My last set of updates to OSProcess for Pharo were done in January 2013,
 and it worked at that time. Has something stopped working since then?
 
 I've been using OSProcess sucessfully on Pharo1.4 and 2.0 for quite a 
 long time now; however, when trying to test if I did everything right on 
 a 3.0 adaptation, running the tests ended up
 1- uncovering more deprecated warnings with things that do not exist 
 under Pharo2 (so that the Pharo2 solution is deprecated on Pharo3, and 
 the Pharo3 solution does not exist in Pharo2... yuck. ensureDelete it is).
 2- test failures due to missing Xcontrol plugin? (and permanent process 
 error afterwards) so I scrapped that image and my implementation, will 
 rewrite everything with a fresh new image.
 
 Can I give up on trying to run OSProcess tests?

The tests require XDisplayControlPlugin in the VM in order to do many of
the multi-process tests, because #forkSqueak is used to create cooperating
VM processes for the tests. In addition, some OSProcess function will not
work on Cog.

If you can run the tests on an interpreter VM they should pass. If you
can get XDisplayControlPlugin included in the VM, then I expect that most
of the tests would pass (we would have to try it to be sure).

Otherwise, yes, you can give up on running the tests. You may still find
them useful as a source of examples and for running specific tests that
do not require #forkSqueak.

Dave


 
 Looks like a version / implementation test would be the way to go. I'll
 write something which should work on 2.0 / 3.0, and failure protection
 to fallback for anything else.
 
 Is there a standard way to test for implementation/version on all Squeak
 and Pharo versions ?
 
 
 No. And as you can see from the examples in OSProcess, it is becoming
 increasingly difficult to cobble up a solution that works for an externally
 maintained package. If you can find a better solution, that would be great 
 :)
 
 I'm attempting something. Is is OK if I save in the OSProcess 
 squeaksource repository?
 
 Thierry

Can you post the method here first? I'd like to check it on some Squeak images
before it goes into the repository.

Thanks!
Dave

 
 Dave
 
 
 
 
 
 -- 
 Thierry Goubier
 CEA list
 Laboratoire des Fondations des Syst?mes Temps R?el Embarqu?s
 91191 Gif sur Yvette Cedex
 France
 Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Tue, Sep 03, 2013 at 01:48:46PM +0200, Goubier Thierry wrote:
 
 Le 03/09/2013 13:36, David T. Lewis a ?crit :
 On Tue, Sep 03, 2013 at 01:15:01PM +0200, Goubier Thierry wrote:
 
 I'm attempting something. Is is OK if I save in the OSProcess
 squeaksource repository?
 
 Thierry
 
 Can you post the method here first? I'd like to check it on some Squeak 
 images
 before it goes into the repository.
 
 Here it is (at least an example):
 

Thank you! I will check this when I get home this evening.

Dave

 in OSProcess class
 
 isPharo3AndLater
   Test if we are on Pharo 3.0
 
   ^ (Smalltalk classNamed: 'SystemVersion')
   ifNil: [ false ]
   ifNotNil: [ :v | v current type = 'Pharo' and: [ v current 
   major = 3 ] ]
 
 platformName
   After Squeak version 3.6, #platformName was moved to SmalltalkImage 
   Some
   versions of Pharo move this to OSPlatform and issue deprecation 
   warnings
   about the other usages. Then Pharo moved away from OSPlatform and 
 deprecated
   its use.
 
   self platformName
 
   self isPharo3AndLater
   ifTrue: [ ^ Smalltalk os name ].
   ^ (((Smalltalk hasClassNamed: #OSPlatform)
   and: [(Smalltalk at: #OSPlatform)
   respondsTo: #platformName])
   ifTrue: [Smalltalk at: #OSPlatform]
   ifFalse: [((Smalltalk classNamed: 'SmalltalkImage')
   ifNil: [^ Smalltalk osVersion]) current]) 
   platformName
 

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Tue, Sep 03, 2013 at 01:48:46PM +0200, Goubier Thierry wrote:
 
 Le 03/09/2013 13:36, David T. Lewis a ?crit :
 
 Can you post the method here first? I'd like to check it on some Squeak 
 images
 before it goes into the repository.
 
 Here it is (at least an example):
 
 in OSProcess class
 
 isPharo3AndLater
   Test if we are on Pharo 3.0
 
   ^ (Smalltalk classNamed: 'SystemVersion')
   ifNil: [ false ]
   ifNotNil: [ :v | v current type = 'Pharo' and: [ v current 
   major = 3 ] ]

The idea is right, but the details can be a PITA ;-)

- In Squeak trunk, class SystemVersion exists. But it does not understand 
#type, so this fails at runtime. 

- There are no implementors of #major in Squeak (but this can be rewritten 
using #perform:).

- In Squeak 3.8, #ifNil:ifNotNil: requires a block with no arguments.

I did not check the other Pharo versions.

Something like this might work:

isPharo3AndLater
Smalltalk
at: #SystemVersion
ifPresent: [:cls | ((cls canUnderstand: #type) and: [ cls 
canUnderstand: #major ])
ifTrue: [^ cls current type = 'Pharo' and: [ cls 
current major = 3 ]]].
^false


Dave




 
 platformName
   After Squeak version 3.6, #platformName was moved to SmalltalkImage 
   Some
   versions of Pharo move this to OSPlatform and issue deprecation 
   warnings
   about the other usages. Then Pharo moved away from OSPlatform and 
 deprecated
   its use.
 
   self platformName
 
   self isPharo3AndLater
   ifTrue: [ ^ Smalltalk os name ].
   ^ (((Smalltalk hasClassNamed: #OSPlatform)
   and: [(Smalltalk at: #OSPlatform)
   respondsTo: #platformName])
   ifTrue: [Smalltalk at: #OSPlatform]
   ifFalse: [((Smalltalk classNamed: 'SmalltalkImage')
   ifNil: [^ Smalltalk osVersion]) current]) 
   platformName
 
 Thanks!
 Dave
 
 
 Dave
 
 
 
 
 
 --
 Thierry Goubier
 CEA list
 Laboratoire des Fondations des Syst?mes Temps R?el Embarqu?s
 91191 Gif sur Yvette Cedex
 France
 Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
 
 
 
 
 -- 
 Thierry Goubier
 CEA list
 Laboratoire des Fondations des Syst?mes Temps R?el Embarqu?s
 91191 Gif sur Yvette Cedex
 France
 Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Tue, Sep 03, 2013 at 10:51:33PM -0400, David T. Lewis wrote:
 On Tue, Sep 03, 2013 at 01:48:46PM +0200, Goubier Thierry wrote:
  
  Le 03/09/2013 13:36, David T. Lewis a ?crit :
  
  Can you post the method here first? I'd like to check it on some Squeak 
  images
  before it goes into the repository.
  
  Here it is (at least an example):
  
  in OSProcess class
  
  isPharo3AndLater
  Test if we are on Pharo 3.0
  
  ^ (Smalltalk classNamed: 'SystemVersion')
  ifNil: [ false ]
  ifNotNil: [ :v | v current type = 'Pharo' and: [ v current 
  major = 3 ] ]
 
 The idea is right, but the details can be a PITA ;-)
 
 - In Squeak trunk, class SystemVersion exists. But it does not understand 
 #type, so this fails at runtime. 
 
 - There are no implementors of #major in Squeak (but this can be rewritten 
 using #perform:).
 
 - In Squeak 3.8, #ifNil:ifNotNil: requires a block with no arguments.
 
 I did not check the other Pharo versions.
 
 Something like this might work:
 
 isPharo3AndLater
   Smalltalk
   at: #SystemVersion
   ifPresent: [:cls | ((cls canUnderstand: #type) and: [ cls 
 canUnderstand: #major ])
   ifTrue: [^ cls current type = 'Pharo' and: [ cls 
 current major = 3 ]]].
   ^false
 


I am also checking the platform subtype implementation (OSProcess 
platformSubtype).

In Pharo:

Smalltalk os subtype == 'i686'

This reflects the processor type, not the os subtype (it should be 'x86_64' on 
my PC).

Is this intentional?

Dave

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Wed, Sep 04, 2013 at 02:17:29PM +0200, Marcus Denker wrote:
 
 On Sep 4, 2013, at 2:13 PM, David T. Lewis le...@mail.msen.com wrote:
 
  On Tue, Sep 03, 2013 at 10:51:33PM -0400, David T. Lewis wrote:
  On Tue, Sep 03, 2013 at 01:48:46PM +0200, Goubier Thierry wrote:
  
  Le 03/09/2013 13:36, David T. Lewis a ?crit :
  
  Can you post the method here first? I'd like to check it on some Squeak 
  images
  before it goes into the repository.
  
  Here it is (at least an example):
  
  in OSProcess class
  
  isPharo3AndLater
Test if we are on Pharo 3.0
  
^ (Smalltalk classNamed: 'SystemVersion')
ifNil: [ false ]
ifNotNil: [ :v | v current type = 'Pharo' and: [ v current 
major = 3 ] ]
  
  The idea is right, but the details can be a PITA ;-)
  
  - In Squeak trunk, class SystemVersion exists. But it does not understand 
  #type, so this fails at runtime. 
  
  - There are no implementors of #major in Squeak (but this can be rewritten 
  using #perform:).
  
  - In Squeak 3.8, #ifNil:ifNotNil: requires a block with no arguments.
  
  I did not check the other Pharo versions.
  
  Something like this might work:
  
  isPharo3AndLater
 Smalltalk
 at: #SystemVersion
 ifPresent: [:cls | ((cls canUnderstand: #type) and: [ cls 
  canUnderstand: #major ])
 ifTrue: [^ cls current type = 'Pharo' and: [ cls 
  current major = 3 ]]].
 ^false
  
  
  
  I am also checking the platform subtype implementation (OSProcess 
  platformSubtype).
  
  In Pharo:
  
  Smalltalk os subtype == 'i686'
  
  This reflects the processor type, not the os subtype (it should be 'x86_64' 
  on my PC).
  
  Is this intentional?
  
 I don't think so.
 
   Marcus
 

Thanks Goubier and Marcus,

I made the updates to OSProcess and CommandShell to handle system version and 
subtype
in Pharo 3.0.

I did not yet update ConfigurationOfOSProcess or ConfigurationOfCommandShell, 
but the
latest versions for Squeak/Pharo are in OSProcess-dtl.83 and 
CommandShell-dtl.73 in
the SqueakSource repositories.

Dave

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Wed, Sep 04, 2013 at 08:38:04AM +0200, Marcus Denker wrote:
 
 Why do you need to support Squeak 3.8? This is how many years old?
 
 I really do not understand this idea to be compatible to all old versions 
 ever.
 

I do not think that there is any right or wrong answer to this, it is
perhaps a matter of personal perspective. For me, much of my professional
experience is related to supporting manufacturing operations for a large
commercial company. In this environment, the ability to engineer solutions
to new problems without disrupting existing operations is critical. To
me, a failure related to software changes is not just a red mark on a
unit test, it is a significant emotional event if I cause a manufacturing
plant to stop production (thankfully this has rarely happened in my career
so far, knock wood).

So maybe this makes me conservative, but from my point of view the ability
to deliver future versions of software that work without breaking older
versions is a matter of good engineering discipline, and I stand by that
as a matter of principle.

Dave

Re: [Pharo-dev] OSProcess for Pharo3.0

[David T. Lewis]

On Thu, Sep 05, 2013 at 03:31:06PM +0200, Goubier Thierry wrote:
 
 
 Le 05/09/2013 14:19, David T. Lewis a ?crit :
 On Thu, Sep 05, 2013 at 11:18:27AM +0200, Goubier Thierry wrote:
 Thanks.
 
 I have merged -dtl.33 and two more change in
 OSProcess-Base-ThierryGoubier.35 on
 http://smalltalkhub.com/mc/ThierryGoubier/Alt30/main/ (osVersion and
 vmVersion), if you want to merge them as well.
 
 
 Thanks!
 
 I have a question: are you sure about the perform: #delete in OSProcess
 class#deleteFileNamed: ? ensureDelete/ensureDeleted have a different
 behavior than #delete.
 
 
 No, I am not sure about this. It is probably a bug. If you have a 
 correction, I appreciate it.
 
 OSProcess-Base-ThierryGoubier.36 in the same place: 
 http://smalltalkhub.com/mc/ThierryGoubier/Alt30/main/

I merged your fixes back into the OSProcess repository. Thank you :-)

Dave

Re: [Pharo-dev] Please help us by buying the book

[David T. Lewis]

On Mon, Sep 16, 2013 at 04:24:07PM +0200, Damien Cassou wrote:
 Hi,
 
 please support our work on Pharo by buying the new Deep into Pharo book
 
 http://rmod.lille.inria.fr/deepIntoPharo/
 
 Your money go to the Pharo Association, so it's worth it.


Thanks, I just ordered my copy of the hard copy book.

Dave
 

Re: [Pharo-dev] recovering from a deep disc crash???.

[David T. Lewis]

On Fri, Oct 18, 2013 at 01:04:25PM +0200, Torsten Bergmann wrote:
 Hardware that runs without problems must be from the good old days.
 
 If Nico is right then I guess Pharo is already running on an Ural-1? 
 Where can I download the VM? 
 
   http://en.wikipedia.org/wiki/Ural_(computer)


Thanks for this link. I had never heard of a Ural-1, and I love reading
about these early pioneers :-)

Dave
 

Re: [Pharo-dev] i wish i could live in a world...

[David T. Lewis]

On Wed, Oct 23, 2013 at 03:00:54AM +0200, Igor Stasenko wrote:
 where morphs has position relative to their owner morph.
 and laying out morphs or composing them into complex layout easy
 and event handling is easy ..
 and things are properly documented
 :)
 

I'm not sure where this project stands, but Juan is working with coordinate
systems for a morphic replacement. For clarity of code and documentation,
his work is second to none. I think that if the two of you ever did some
collaborative development, the results might be scary good :-)

http://www.jvuletich.org/Morphic3/Morphic3-200911.html

Dave

Re: [Pharo-dev] Tests with multiple processes

[David T. Lewis]

I have some experience with this, not related to Smalltalk but maybe
still relevant. I have worked with systems that were designed around
networked message passing, implemented on platforms like MS-DOS with time
slicing kernels, Series I, OS/2, etc. These systems were designed to operate
with asynchronous message passing protocols. The asynchronous designs were
intended to support primitive OS environments, including MS-DOS with
software interrupts for multitasking.

These are production-critical manufacturing applications, and in the course
of modernizing them I have reimplemented the network protocols in Java and
moved them to new computing platforms. Along the way I have found that the
asynchronous designs could always be replaced by bidirectional send-receive
transactions running in the context of a thread or process. Even the systems
that appeared to be most complex have turned out to work perfectly well
when modeled as synchronous send-receive transactions on a bidirectional
communication channel. The software is much simpler to understand and debug
when structured in this manner.

My conclusion is that designing around asynchronous, loosely coupled
channels sounds like a good idea, but in practice it leads to unnecessary
complexity, and to systems that are difficult to debug and support.

$0.02

Dave


On Tue, Oct 29, 2013 at 01:19:17AM +0100, Igor Stasenko wrote:
 Maybe i'm off track, but here my thoughts:
 
  - one big misconception (and most problems which arising from it), that
 people tend to use a bidirectional communication models instead of
 unidirectional.
 In bidirectional you tend to think as if you using same channel for sending
 and receiving data.
 The truth is, that it is two unidirectional channels, wired in the way so
 they look like single bidirectional channel.
 
 And what it means, that coupling sending and receiving activities in a
 single thread is wrong:
  - you should not wait for data only if you sent something
  - you should not expect that activity from other end is possible only
 after you _send_ something.
 
 If you treat connection as two unidirectional, loosely coupled channels,
 instead of single bidirectional one,
 you will find that in such setup you don't have many problems.
 
 You should definitely go for (2), e.g. you should decouple a
 transport/communication layer
 from your program/logic flow: if some process wants to send data and block
 until some response to
 that message arrived, you should do so.
 You can simply put that process on semaphore, and adding it to the list of
 'awaiting answer'
 processes in your scheduler which runs in separate process to handle
 incoming data.
 Then you can have different strategies, what to do if
  a) data seems taking too long to arrive (signal timeout)
  b) connection lost
  c) etc..
 
 ohh.. with smalltalk it is so easy to operate with those things (the fact
 the VM is green-threaded doesn't
 changes much).
 
 
 
 On 28 October 2013 23:52, Norbert Hartl norb...@hartl.name wrote:
 
 
  Am 28.10.2013 um 23:50 schrieb Norbert Hartl norb...@hartl.name:
 
  Are there any approaches to simulate coroutines in a single thread
  environment or approaches to deal with multiple processes within one
  process?
 
 
  That should have been
 
  Are there any approaches to simulate coroutines in a single thread
  environment or approaches to deal with multiple processes within one test?
 
  Norbert
 
 
 
 
 -- 
 Best regards,
 Igor Stasenko.

Re: [Pharo-dev] Tests with multiple processes

[David T. Lewis]

 Am 29.10.2013 um 02:08 schrieb David T. Lewis le...@mail.msen.com:

 I have some experience with this, not related to Smalltalk but maybe
 still relevant. I have worked with systems that were designed around
 networked message passing, implemented on platforms like MS-DOS with
 time
 slicing kernels, Series I, OS/2, etc. These systems were designed to
 operate
 with asynchronous message passing protocols. The asynchronous designs
 were
 intended to support primitive OS environments, including MS-DOS with
 software interrupts for multitasking.

 These are production-critical manufacturing applications, and in the
 course
 of modernizing them I have reimplemented the network protocols in Java
 and
 moved them to new computing platforms. Along the way I have found that
 the
 asynchronous designs could always be replaced by bidirectional
 send-receive
 transactions running in the context of a thread or process. Even the
 systems
 that appeared to be most complex have turned out to work perfectly well
 when modeled as synchronous send-receive transactions on a bidirectional
 communication channel. The software is much simpler to understand and
 debug
 when structured in this manner.

 My conclusion is that designing around asynchronous, loosely coupled
 channels sounds like a good idea, but in practice it leads to
 unnecessary
 complexity, and to systems that are difficult to debug and support.

 I agree. That was the reason for me to look out for a simpler approach. At
 least for testing. Can you elaborate on the „bidirectional send-receive
 transaction“. I’m not sure I understand what happens there. I explained my
 case a bit more in my response to Igor.


I modeled the problem in terms of communication sessions, with message
objects that can be sent and received on a session. By “transaction” (poor
choice of terms here) I mean an object with an evaluate method that uses a
session to send a message object and receive a response message object,
and that answers an object that encapsulates the result of that completed
transaction (including errors if any).

Early in the project I implemented a kind of “session” that handled
asynchronous send and receive, keeping track of thread synchronization and
matching response objects to request objects. I expected this to be
necessary for interaction with old MS-DOS and OS/2 based applications that
had been written in that manner, and that were entirely asynchronous and
interrupt driven by design. I practice, I have never actually needed to
use this asynchronous type of processing, because the simple synchronous
session has always been proved sufficient even when interacting with
systems that appeared to be (and were designed to be) completely
asynchronous and event driven.

Re: [Pharo-dev] Tests with multiple processes

[David T. Lewis]

 On 29 October 2013 02:08, David T. Lewis le...@mail.msen.com wrote:

 I have some experience with this, not related to Smalltalk but maybe
 still relevant. I have worked with systems that were designed around
 networked message passing, implemented on platforms like MS-DOS with
 time
 slicing kernels, Series I, OS/2, etc. These systems were designed to
 operate
 with asynchronous message passing protocols. The asynchronous designs
 were
 intended to support primitive OS environments, including MS-DOS with
 software interrupts for multitasking.

 These are production-critical manufacturing applications, and in the
 course
 of modernizing them I have reimplemented the network protocols in Java
 and
 moved them to new computing platforms. Along the way I have found that
 the
 asynchronous designs could always be replaced by bidirectional
 send-receive
 transactions running in the context of a thread or process. Even the
 systems
 that appeared to be most complex have turned out to work perfectly well
 when modeled as synchronous send-receive transactions on a bidirectional
 communication channel. The software is much simpler to understand and
 debug
 when structured in this manner.

 My conclusion is that designing around asynchronous, loosely coupled
 channels sounds like a good idea, but in practice it leads to
 unnecessary
 complexity, and to systems that are difficult to debug and support.

 $0.02

 Dave


 Example from my life: real-time voice over IP project.
From very starting, it was designed as a bidirectional connection between
 two peers.
 Even though, that outgoing/incoming traffic was handled separately, the
 model was still there
 and treated as 'single conversation channel'.

 Later we wanted to introduce many-peer conversation and found that the
 above bi-directional model simply
 does not fits.. everything falls apart.
 Simple example: you have 3 peers A, B , C.
 So, you can represent is in this way:
   A-B , A-C , B-C,
 or this:
   A - (B,C)
   B - (A, C)
   C - (A,B)

 The bidirectional model implies that you have symmetry in number of
 outgoing and incoming channels (hence -)..
 and if you send data over one of them, only single node will receive it.
 Logically, that leads to sending same data to two different channels. So,
 at very beginning you nailing down
 how the traffic is going to flow.

 With unidirectional model, as illustrated when you sending data over one
 channel,
 it will be delivered to all participants, and you don't have to do it
 yourself - the transport layer handles it for you.

 The thing is, that the way how data gets to receivers is heavily depends
 on
 network topology:
 imagine that two peers, B, C belong to same subnet, and latency between
 them is miniscule comparing
 to latency between A and one of them. The logical choice then should be to
 route the signal from A
 to either B or C and then broadcast from middle point to the listener
 which
 is left.
 Not possible with bidirectional model, because A insists on having direct
 socket connection with both B and C.

 Same goes for proxies: imagine that node A is connected via proxy..
 and not directly to nodes B and C.. that means that to deliver its voice
 traffic to both of them it should
 just send a single packet to proxy, and then proxy will broadcast it to
 listeners.
 Not going to happen with bidirectional model, A will still send two very
 same packets to same proxy.. but with different destination
 address.

 Or even, it could be, that receiving traffic from A to C goes faster via
 B,
 while sending traffic from C to A is faster to do directly..
 So try to fit that into bidirectional model of communication and see where
 it goes.

 Yes, i agree it is easier to put everything into sequential bidirectional
 model which makes everything so simple.
 If it fits. But if its not, you should not even try to fit it..

 My point is, that IP is inherently asynchronous and unidirectional.


You give an excellent counter example, and I agree with what you are
saying. I am reminded of the quote attributed to Einstein, saying that
things should be as simple as possible but no simpler. For the examples
that I gave, simple was good enough. But that is not the case for all such
problems ;-)

Dave

Re: [Pharo-dev] Formatting dates

[David T. Lewis]

On Wed, Oct 30, 2013 at 03:03:53PM -0700, blake wrote:
 Chris,
 
 On Wed, Oct 30, 2013 at 2:02 PM, Chris Muller asquea...@gmail.com wrote:
 
  Date today mmdd.  prints '2013-10-30'
 
 
 I don't want the separator.
 

Date today mmdd copyWithoutAll: '-'

Re: [Pharo-dev] Github and Pharo

[David T. Lewis]

On Mon, Nov 04, 2013 at 01:58:29PM +0100, Goubier Thierry wrote:
 
 
 Le 04/11/2013 12:11, kilon alios a ?crit :
 yeap filetree did the trick here. However it does not allow to browse
 through the git commits as gitfiletree does, the only commit available
 is the last commit.
 
 I took a look at CommandShell and friends and they all look pretty much
 very broken. For example in workspace I executed
 [ CommandShellTranscript open.] and trying ls or dir it creates an
 error because it add C path inside pharo subdirectories. Dont know if
 this is normal behavior.

Use CommandShell open rather than CommandShellTranscript open.


 
 I hope someone with more knowledge than me of OSProcess under windows 
 will have a look :)
 

OSProcess support for Windows is incomplete, so this will probably not
do what you need. If the OSProcess is included in the Windows VM, it will
let you run a Windows program, but it will not do most of the other things
that you expect from OSProcess.

Check http://www.squeaksource.com/ProcessWrapper.html for a possible
alternative.

Dave

Re: [Pharo-dev] Loading Xtreams in Pharo

[David T. Lewis]

On Thu, Nov 21, 2013 at 12:29:36PM +0100, Torsten Bergmann wrote:
  https://ci.inria.fr/pharo-contribution/job/Xtreams/
  
  still a little rough, but it is a start.
 
 If possible I would like to see the project itself moved over from SqS to 
 STHub.
 
 Not as a Pharo fork but as a replacement for the Squeaksource repo. 
 Sorry but I was too often bitten by SqS problems...

For information: The squeaksource.com service is now hosted on a different
server, and is maintained by the Squeak community. The previous reliability
and performance problems have been resolved.

I don't want to discourage anyone from moving to other repositories, but you
should not feel like you have to do it on account of the previous reliability
issues.

FYI only,
Dave

[Pharo-dev] REPL (was: about miniimage)

[David T. Lewis]

 Tangentially can anyone point me to a headless image with a REPL for use
 in testing Spur?


Hi Eliot,

This is probably much too heavyweight for what you have in mind, but if
you are able to load OSProcess and CommandShell into the test image, you
can then evaluate ExternalCommandShell start, and talk to the image on
stdin/stdout from your terminal session.

You'll need to start the image in the foreground (no ) so that you are
connected to it from your terminal. The command evaluator accepts simple
Smalltalk expressions and various unix-ish inputs.

Dave

Re: [Pharo-dev] Stepping through with GLORP Proxies

[David T. Lewis]

On Fri, Nov 29, 2013 at 01:02:53PM -0800, Eliot Miranda wrote:
 Does the Pharo debugger use the mirror primitives yet?  Unless
 InstructionStream and ContextPart have been updated to use objectClass: and
 objectInstVarAt: et al then the debugger will be fooled by proxies.  I
 really need to push this code through to Squeak trunk.  Once I've done that
 (Monday?  it's thanksgiving here and its party time :-) ) Pharo can perhaps
 adopt the code too.
 
 David, does the Interpreter support the mirror primitives yet?  i.e. do the
 MirrorPrimitiveTests pass on the Interpreter now?
 

Hi Eliot,

Happy Thankgiving. Sorry I missed your question, but yes the mirror
primitives are implemented in the interpreter VM:

  Name: VMMaker-dtl.262
  Author: dtl
  Time: 5 January 2012, 12:41:29.745 am
  
  VMMaker 4.7.19
  
  Reference Mantis 7429: Add Mirror Primitives to the VM
  
  Update primitivePerformInSuperclass to support
  ContextPartobject:perform:withArguments:inClass:
  
  Implementation differs from that of oscog in that the original
  primitivePerformAt: is retained unmodified, and the necessary
  stack adjustments are done in primitivePerformInSuperclass for
  the special case of argumentCount 4 (mirror primitive call)
  rather than 3. The oscog approach may be prefered (not least
  for its clearer method naming), but making the change in
  primitivePerformInSuperclass is low risk and more easily
  implemented by a Sunday Squeaker.
  
  All MirrorPrimitiveTests pass.


So it theory it should work ... 

Dave



 
 On Fri, Nov 29, 2013 at 6:03 AM, Esteban A. Maringolo
 emaring...@gmail.comwrote:
 
  Does anybody else have issues while stepping through GLORP Proxy objects
  (collections in this case)?
 
  I simply can't use this feature unless the proxied collection was already
  materialized.
 
  I can't reproduce it right now, but it bites me everytime is step through
  blocks, because stepping into is really annoying :)
 
 
  Does anybody else have the same issue?
 
  This is Pharo 2.
 
 
  Esteban A. Maringolo
 
 
 
 
 -- 
 best,
 Eliot

Re: [Pharo-dev] Some squeaksource.com updates

[David T. Lewis]

On Sun, Dec 01, 2013 at 07:11:59PM -0800, Sean P. DeNigris wrote:
 David T. Lewis wrote
  The welcome page for http://squeaksource.com has been updated... The
  policy of disabling new project creation remains in effect.
 
 Thanks! It is good that so much of our history will remain accessible... 
 
 create any number of projects to host on our server should probably be
 removed


Thanks Sean, I fixed it.

Dave
 

Re: [Pharo-dev] copyscript from squeaksource to smalltalkhub

[David T. Lewis]

On Sat, Jan 04, 2014 at 07:37:36PM +0100, Hilaire Fernandes wrote:
 Le 04/01/2014 19:11, Johan Brichau a ?crit :
  Hi,
  
  Does anyone still have the script to copy a repository from squeaksource to 
  smalltalkhub?
  
  thx
  Johan
  
 
 Last time I saw was in the welcome page of the squeaksource.com web site.


Here is the script that was on the squeaksource.com welcome page.

Dave



| source goSource destination goDestination files destinationFiles |

source := MCHttpRepository location: 'http://www.squeaksource.com/YOURPROJECT'.
destination := MCSmalltalkhubRepository
owner: 'YOURNAME'
project: 'YOURPROJECT'
user: 'YOURNAME'
password: 'YOURPASSWORD'.

goSource := Gofer new repository: source.
goDestination := Gofer new repository: destination.

files := source allVersionNames.

(goSource allResolved select: [ :resolved | files anySatisfy: [ :each |
resolved name = each ] ]) do: [ :each | goSource package: each packageName 
].

goSource fetch. downloads all mcz on your computer

destinationFiles := destination allVersionNames. checks what files are already 
at destination
files reject: [ :file | destinationFiles includes: file ] thenDo: [ :file |
goDestination version: file ]. selects only the mcz that are not yet at 
destination

goDestination push. sends everything to SmalltalkHub

self assert: destination allVersionNames sorted = files sorted.
checks we have exactly the same files at source and destination


 

Re: [Pharo-dev] Is there a Smalltalk way to lock a file?

[David T. Lewis]

Hi Mariano,

It is supported directly in OSProcess. See the unit test for examples.

Dave

 Hi guys,

 Do we have a method to lock a file at the OS level? I searched in image
 but
 I cannot find anything. If it would work in gemstone too that would be
 nice.

 Thanks!

 --
 Mariano
 http://marianopeck.wordpress.com

Re: [Pharo-dev] Is there a Smalltalk way to lock a file?

[David T. Lewis]

 Hi Mariano,

 It is supported directly in OSProcess. See the unit test for examples.

 Dave

Sorry, I should have been more clear about this. The classes in the
OSProcess package are OSFileLock and OSFileRegionLock, as well as the
methods in category 'file locking' of class UnixOSProcessAccessor.

You can look at the unit tests to see how these are used, but you will not
be able to actually run any of the tests that depend on #forkSqueak
because these rely on a feature not present in Cog. So look at the tests
as examples, but do not expect them to run.

The actual file locking support should work fine on Pharo (but let me know
if that is not true).

HTH,
Dave


 Hi guys,

 Do we have a method to lock a file at the OS level? I searched in image
 but
 I cannot find anything. If it would work in gemstone too that would be
 nice.

 Thanks!

 --
 Mariano
 http://marianopeck.wordpress.com

Re: [Pharo-dev] [Moose-dev] Re: Re: Re: Fwd: Font problem is still there

[David T. Lewis]

On Fri, Apr 04, 2014 at 02:55:01AM +0200, Igor Stasenko wrote:
 
 there is one caveat, that if you really want to see exactly , say 16 points
 sized font on your screen,
 it is not possible without knowing the display resolution - how many pixels
 in one inch (hence
 #pixelsPerInch ).
 Unfortunately, our VMs don't give us a way to determine DPI of display..
 and so, it is always 96 :/
 
 pheww

I don't think there is any way for the VM to know the #pixelsPerInch of the
display, regardless of the display resolution. Maybe that implies that some
calibration would be needed in the image to relate font points to the
physical display.

Dave

Re: [Pharo-dev] Pharo RPi and FileSystem error

[David T. Lewis]

 There are no NativeBoost on Rpi, FFI or OSProcess.
 Basically the fileSystem work, but some data are fetch via env variable
 and then NativeBoost.
 The solution is to bypass all the request of env variable.
 You should use another way until, we finish to integrate a way to request
 correctly the env variable (NB on arm).

 I will studies the possibility to reintegrate fast the OSProcess plugin to
 offer better solution.


I would expect OSProcess to work without problems on Rpi, although I do
not have a way to test it. The plugin should be fine. Maybe some changes
will be needed on the image side for detecting the platform, I'm not sure.

Let me know if I can help, I will be glad to add any necessary patches to
OSProcess.

Dave

[Pharo-dev] Unix interop (was: a Pharo talk from a ruby conference)

[David T. Lewis]

On Tue, Apr 29, 2014 at 07:15:30AM -0700, Sean P. DeNigris wrote:
 
 Same goes with Unix interop. REPL is easy, but is it documented and
 marketed? What would it take to easily pipe output of other programs to
 Pharo? Maybe be able to sourceCodeString exportAsUnixCommand:
 '/usr/bin/my_cool_command'

That's what CommandShell/OSProcess is for. For example:

  sourceCodeString := 'ls -l | cat | cat | cat'.
  (ProxyPipeline command: sourceCodeString) upToEndOfFile

Dave

Re: [Pharo-dev] 32 bits and 64 bits VM

[David T. Lewis]

On Fri, May 16, 2014 at 10:20:47AM +0200, Esteban Lorenzano wrote:
 
 On 15 May 2014, at 18:46, Hilaire Fernandes hilaire.fernan...@gmail.com 
 wrote:
 
  Great.
  
  As we are discussing about build, is it possible to compile CogVM on 64
  bits architecture as it is already the case for the interpreted SqueakVM
  (https://packages.debian.org/sid/squeak-vm)?
 
 I do not understand. To compile a regular vm in a 64bits platform is trivial. 
 You just need to have the 32bits library installed.
 But to have a 64bits vm that runs on 64bits? that?s another very different 
 history:
 
 - you need a 64bits image (so trace, export, etc.)
 - you need to be 64bits word size aware (not so complicated, but a lot of 
 work). 
 - you need 64bits plugins (a lot of them)
 - you need 64bits JIT (or no JIT at all)
 
 Some time ago there was an experimental 64bits interpreter vm (not even 
 stack). Who worked with a special traced image.
 What squeak guys are doing in the link you provide is still building a 32bits 
 vm. 

It's experimental 64 bit image is not really all that experimental any more,
although Ian still labels it as such on squeakvm.org/unix. A compiled VM is
available there. Building it from source is just a matter of specifying 
--image64
in the configure step, then doing a make.

Up to date Squeak trunk images in 64-bit format are traced daily at
http://build.squeak.org/job/Squeak%2064-bit%20image/

Dave

Re: [Pharo-dev] Coral is green

[David T. Lewis]

Hi Stephan,

The precompiled Cog VM would need to come from Eliot's site (or follow the
links to Eliot's site from http://squeakvm.org). The build.squeak.org job
is really just me testing the VMMaker code generation, it's not intended
as a source of VMs.

I tested Eliot's change and it works, but I think I was testing it with a
Cog VM that I compiled myself. So I'm not sure if the up to date
precompiled VM is available.

In any case, the change affects one source file in the VM platform
sources, so if you can arrange to get that included in the Pharo VM
builds, that should provide a working VM that you can use for OSProcess on
Pharo.

Dave


 Thanks, David.

 Where do I find a precompiled cog vm with the changes to test with?
 (preferably mac,
 but linux will do). build.squeak.org doesn't seem to have a recent green
 build.

 Stephan

Re: [Pharo-dev] Coral is green

[David T. Lewis]

On Fri, May 23, 2014 at 10:26:54PM +0200, Stephan Eggermont wrote:
 David wrote:
 In any case, the change affects one source file in the VM platform 
 sources, so if you can arrange to get that included in the Pharo VM 
 builds, that should provide a working VM that you can use for OSProcess on 
 Pharo. 
 
 I noticed that this is only a unix vm change. Should this be working
 already on Mac and Windows?


Definitely not for Windows.

I'm honestly not sure where we stand on Mac VMs. If the VM is based on the
unix platform sources, then it will work, otherwise probably not.

The specific feature that affects the OSProcess tests is the ability to
support #forkSqueak, which uses a Unix fork() mechanism to start an exact
copy of the running VM and image with very little memory overhead. This is
used in the OSProcess test suite in order to set up test cases that deal
with the interaction of two or more OS processes.

Think about what is required to ask the VM to pause for a moment, split
itself in two like an amoeba, and then have both of the VM instances start
running again as if nothing had happened. That means pausing and restarting
some signal handlers, pausing the connection to the X display so a new one
can be created for the second VM process, and restarting the various threads
that were running in the original VM. Most of this was already working for
Cog. Eliot came up with a way to restart the timer threads for Cog and
wonder of wonders, that was all it took to get #forkSqueak working on Cog.

Dave 

Re: [Pharo-dev] Improving Pharo By Example

[David T. Lewis]

On Sun, Jun 01, 2014 at 09:09:21PM +0200, Damien Cassou wrote:
 On Sun, Jun 1, 2014 at 8:54 PM, kilon alios kilon.al...@gmail.com wrote:
  Whats the verdict with the Squeksource section ? Do I covert it to a
  SmalltalkHub section or do I continue to chapter 4 ?
 
 
 I think you can keep the section about squeaksource and replace it
 with ss3 and smalltalkhub instead.
 

That sounds right. Squeaksource.com remains operational, but readers should
be guided to use smalltalkhub and ss3 for new work.

Dave

Re: [Pharo-dev] ExternalPipemakePipe error

[David T. Lewis]

Check if your VM has the UnixOSProcessPlugin plugin. It looks like PythonParser 
is
using OSProcess to connect to Python, so the plugin would need to be present in
order for this to work.

As a quick check, inspect OSProcess thisOSProcess pid. If you get an integer
(the pid of your VM process), then the plugin is available in your VM.

Dave


On Sun, Jun 01, 2014 at 10:25:14PM +0200, Max Leske wrote:
 When trying to load PythonParser into 30 with
 
 Gofer it
 smalltalkhubUser: 'Pharo' project: 'MetaRepoForPharo30/';
 configurationOf: 'PythonParser';
 loadDevelopment
 
 I get the following error (tested with ?/30+vm and ?/30+vmLatest). Any ideas?
 I?m on a MacBook Pro, OS X 10.9.3
 
 
 ExternalPipe(Object)error:
 ExternalPipemakePipe
 ExternalPipeinitialize
 ExternalPipe classblockingPipe
 PipeableOSProcesscreateInputPipe
 PipeableOSProcess 
 classnew:arguments:environment:descriptors:workingDir:errorPipelineStream:
 PipeableOSProcess 
 classforkAndExec:arguments:environment:descriptors:workingDir:errorPipelineStream:
 PipeableOSProcess 
 classcommand:environment:workingDir:input:output:error:errorPipelineStream:shellSyntax:
 PipeableOSProcess 
 classcommand:environment:workingDir:input:output:error:errorPipelineStream:
 PipeableOSProcess classcommand:
 MCFileTreeGitRepositorygitVersions
 MCFileTreeGitRepositoryloadAncestry
 MCFileTreeGitRepositoryloadAllFileNames in Block: [ :bar | ...
 BlockClosurecull:
 Jobrun in Block: [ result := block cull: self ]
 BlockClosureon:do:
 Jobrun in Block: [ ...
 BlockClosureensure:
 Jobrun
 MorphicUIManager(UIManager)displayProgress:from:to:during:
 ByteString(String)displayProgressFrom:to:during:
 MCFileTreeGitRepositoryloadAllFileNames
 MCFileTreeGitRepositoryallFileNames
 MCFileTreeGitRepositorygoferVersionFrom:
 MetacelloCachingGoferResolvedReference(GoferResolvedReference)version
 MetacelloCachingGoferResolvedReferenceversion
 MetacelloFetchingMCSpecLoaderresolveDependencies:nearest:into: in Block: [ 
 :each | each version ]
 OrderedCollectiondo:
 MetacelloFetchingMCSpecLoaderresolveDependencies:nearest:into: in Block: [ 
 ...
 BlockClosureon:do:
 
 
 Cheers,
 Max

Re: [Pharo-dev] news from block :)

[David T. Lewis]

On Wed, Jun 11, 2014 at 11:15:37PM +0300, kilon alios wrote:

 But for me Morphic is the most beautiful Graphics API I have ever
 worked with.


It's nice to see someone say something positive in appreciation of Morphic.
It is quite a remarkable environment if you stop and think about it :-)

Dave

Re: [Pharo-dev] Implementing cull:cull: in symbol

[David T. Lewis]

On Thu, Jun 19, 2014 at 11:24:33AM -0300, Esteban A. Maringolo wrote:
 Can somebody please tell me the semantic difference between #cull: and 
 #value: ?
 'Cull' is not a word I ever used in any english based conversation :)

Hi Esteban,

I am an American English speaker, and I have never used the word 'cull' in
conversation or in writing. I recognize it as a real word, but I had to look
it up in a dictionary to see what it means.

Dave

Re: [Pharo-dev] Can forking speed up anything?

[David T. Lewis]

On Mon, Aug 04, 2014 at 02:53:07PM +0200, Levente Uzonyi wrote:  On Mon, 4 Aug 
2014, Frank Shearar wrote:  
 On 4 August 2014 09:38, Cl??ment Bera bera.clem...@gmail.com wrote:
 
 
 
 2014-08-04 10:17 GMT+02:00 Yuriy Tymchuk yuriy.tymc...@me.com:
 
 Hi guys,
 
 I have a script that runs very slow and does a lot of non dependent
 operation of a collection. I wander if I can speed it up by making it 
 run in
 another process, because as far as I understand everything runs on a 
 single
 thread, so I guess this won???t save me.
 
 I don't think forking can speed up anything.
 
 But that's only because we're single-threaded, hence single-cored. If
 we had a VM that handle multiple threads (like if we resuscitated
 Andreas' and Igor's Hydra work), then forking _would_ speed things up.
 But that would raise another huge issue, namely the shared state
 nature of the image.
 
 Hydra works differently. It's basically running multiple images from a 
 single VM, and provides channels between images for communication (like in 
 Erlang). There's no shared state at all between images, and one image can 
 only use one core just like now.
 It's also possible to use multiple cores by spawning new VMs. IIRC 
 OSProcess has some support for this.
 

OSProcess + CommandShell does this in class RemoteTask. A RemoteTask
spawns a VM with forked image, and uses Fuel to serialize the results
back to the parent image/VM. But Pharo may be missing the necessary VM
and plugin support, so I'm not sure if it works in Pharo now.

But I have to say that this is unlikely to useful as a performance
enhancement except in very specialized cases where large chunks of work
can be run in completely independent parallel tasks.

Dave

Re: [Pharo-dev] UndefinedObject(Object)doesNotUnderstand: #wait

[David T. Lewis]

On Fri, Aug 29, 2014 at 11:42:24AM +0200, Yuriy Tymchuk wrote:
 Sometimes when I save image I get this error:
 
 
 UndefinedObject(Object)doesNotUnderstand: #wait
 [ 
 event wait.
 processSynchronizationDelay wait. Avoids lost signals in heavy process 
 switching
 self changed: #childProcessStatus ] in [ 
 [ 
 event wait.
 processSynchronizationDelay wait. Avoids lost signals in heavy process 
 switching
 self changed: #childProcessStatus ] repeat ] in 
 UnixOSProcessAccessorgrimReaperProcess in Block: [ ...
 BlockClosurerepeat
 [ 
 [ 
 event wait.
 processSynchronizationDelay wait. Avoids lost signals in heavy process 
 switching
 self changed: #childProcessStatus ] repeat ] in 
 UnixOSProcessAccessorgrimReaperProcess in Block: [ ...
 [ 
 self value.
 Processor terminateActive ] in BlockClosurenewProcess in Block: [ ...
 
 
 Is it a known problem?

Hi Uko,

I have not seen this before, but I can say that the error is coming from
OSProcess. OSProcess uses a background process that wakes up when a child
OS process (that was started earlier by OSProcess) has exited.

I do not know the reason for the failure that you are seeing, but I am
guessing that it may be somehow related to image shutdown and startup processing
when you save the image.

Dave

Re: [Pharo-dev] Parsing Pharo syntax to C/C++

[David T. Lewis]

On Mon, Sep 15, 2014 at 11:28:24AM +0200, Santiago Bragagnolo wrote:
  I may be wrong, but I think the closest thing out there is Slang. Is the
 pseudo smalltalk used to develop the VM.
 

A bit off topic, but just for clarification: Slang is not a language. It
is just a casual name to refer to the subset of Smalltalk that can be easily
translated to a language such as C. Some of this Smalltalk may be written
in awkward ways in order to make sure it can be translated to a lower level
language, but it is real Smalltalk nonetheless.

The VM is not written in Slang, it is written in Smalltalk. When people talk
about running the VM simulator, they are talking about running the VM in its
native Smalltalk implementation, as opposed to running the code that is more
commonly translated to C and linked to the platform support code.

There is no psuedo Smalltalk involved, it is the real thing.

Dave

Re: [Pharo-dev] Issue 14160: Introduce isXXX dialect testing methods to SmalltalkImage to ease cross-dialect development

[David T. Lewis]

Speaking as the maintainer of an external package (OSProcess/CommandShell) I
agree with both Stef and Johan.

Feature detection is better, because pharo (or squeak, or gemstone) is
meaningless over time as the dialect changes. If you need to test for some
feature, it is better to test for it explicitly than to assume that it
exists in all versions of a specified dialect.

As Stef says, Seaside with Grease is the way to go. The test for any given
feature belongs in an external package (Grease), not in the core images. For
a significant external package such as Seaside, use the abstraction layer of
Grease. For simpler cases (e.g. OSProcess) put the required tests directly
into that external package. But either way the isFoo tests belong in the
external packages, and not in pharo/squeak/gemstone proper.

Dave


On Sun, Oct 05, 2014 at 10:29:26AM +0200, Johan Brichau wrote:
 It?s true that when you do a lot of cross-dialect development, such a method 
 is often what you desire.
 However, I think it?s better to do feature detection instead of dialect 
 detection. So, something like:
 
 ((Smalltalk includesKey: #CharacterWriteStream) ifTrue:[
   stream := CharacterWriteStream on: (String new: 10)
 ] ifFalse:[
   stream := WriteStream on: (String new: 10)
 ]
 
 And yes: use Grease where applicable. We try to maintain Grease across Pharo, 
 Squeak and Gemstone. I also think it?s actively ported to VA.
 
 cheers
 Johan
 
 On 05 Oct 2014, at 09:00, stepharo steph...@free.fr wrote:
 
  Hi jan
  
  Thanks for the proposal.
  I thought about it and I'm against because it goes again the idea of 
  dispatch.
  I prefer to have a class and some dispatch. Seaside with Grease is the way 
  to go from my perspective.
  Finally I do not want such methods in the systems because I spent my time 
  removing is because of bad design.
  Let us see what other people think.
  
  Stef
  
  On 4/10/14 22:14, Jan Vrany wrote:
  Hi guys,
  
  I've just opened:
  
  https://pharo.fogbugz.com/f/cases/14160/Introduce-isXXX-dialect-testing-methods-to-SmalltalkImage-to-ease-cross-dialect-development
  
  Introduce isXXX (isPharo, isVisualWorks, ...) dialect testing methods to
  SmalltalkImage to ease multi-dialect development.
  
  RATIONALE:
  
  Commonly used approach to solve differences among dialect is to use a
  sort of platform object and dispatch there for troublesome operations
  that has to be specialized. This platform object is usually in platform
  specific package.
  Other option is to load some library like Grease or Sport.
  
  The problem of the first approach is that brings to much (unnecessary)
  burden when used for two, three things. The amount of of the code and
  management is way to bigger then the amount of the code that has to be
  specialized. Loading Grease/Sport on the other hand introduces a
  dependency on an external package - not worth of for two or three
  things.
  
  The proposed dialect testing messages would allow for simple,
  #ifdef-like idiom like:
  
  | stream |
  ...
  ((Smalltalk respondsTo: #isSomeCoolDialect) and:[Smalltalk
  isSomeCoolDialect]) ifTrue:[
stream := CharacterWriteStream on: (String new: 10)
  ] ifFalse:[
stream := WriteStream on: (String new: 10)
  ]
  
  The #respondsTo: check, while not nice, is required as at the moment not
  all dialects could be trusted to have these testing messages.
  
  Putting these methods may not stick with Pharo standard (whatever it
  is), but Smalltalk global is probably one of the
  very few that are present in pretty much every Smalltalk implementation.
  Other option would be to place them to the class side of an Object
  (which is amost certainly present everywhere), however
  
  Smalltalk isPharo
  
  reads better than
  
  Object isPharo.
  
  Best, Jan
  
  
  
  
  
 

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

 Hi,

 Just tried to install the stable version of OSProcess using the
 configuration browser on a 3.0.
 I noticed it is still on http://www.squeaksource.com/OSProcess.
 Does it mean it is dead? if yes, what is the alternative?

Hi Noury,

OSProcess is alive and well. Let me know if you have any problems with it.

The Metacello configurations might be due for an update for Pharo, as
there is at least one recent update that would be of interest to users of
PipeableOSProcess (in www.squeaksource.com/CommandShell).

Dave

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

Thierry,

Thanks for doing those configuration updates, I hit send before I saw
your reply.

Dave

 Hi Noury,

 no, OSProcess on Squeaksource is not dead and is integrating all the
 changes needed for all the platforms it is running on.

 And as you can see, I very recently updated the configuration to make
 sure it was pointing to the very latest version.

 Regards,

 Thierry

 Le 09/10/2014 22:30, Noury Bouraqadi a écrit :
 Hi,

 Just tried to install the stable version of OSProcess using the
 configuration browser on a 3.0.
 I noticed it is still on http://www.squeaksource.com/OSProcess.
 Does it mean it is dead? if yes, what is the alternative?


 Thanx
 Noury

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

Darn, I think we must have both done our updates on the same day. The last
update I did was this (time stamp is GMT -4 hours) :

CommandShell-dtl.76 Dave Lewis, 5 October 2014 5:29:32 pm
CommandShell 4.6.9

Improve pipe closing for PipeableOSProcess classcommand: and related  
methods to support use as stand alone command processor. An expression
such as the following will now close all open handles when command
processing is complete:

  (PipeableOSProcess command: '/bin/echo this is a test') output

If you could update the configuration to pick this up, that would be
great. I think it will be helpful for people who want to easily execute
commands directly from the image.

I think the two sub-packages that you would update are
CommandShell-Base-dtl.38 and CommandShell-Piping-dtl.16

Thanks,
Dave

 Hi Dave,

 thanks to you. I updated OSProcess to 4.5.11 and CommandShell to 4.6.8;
 are those the latest? I did it last Sunday.

 Thierry

 Le 09/10/2014 22:42, David T. Lewis a écrit :
 Thierry,

 Thanks for doing those configuration updates, I hit send before I saw
 your reply.

 Dave

 Hi Noury,

 no, OSProcess on Squeaksource is not dead and is integrating all the
 changes needed for all the platforms it is running on.

 And as you can see, I very recently updated the configuration to make
 sure it was pointing to the very latest version.

 Regards,

 Thierry

 Le 09/10/2014 22:30, Noury Bouraqadi a écrit :
 Hi,

 Just tried to install the stable version of OSProcess using the
 configuration browser on a 3.0.
 I noticed it is still on http://www.squeaksource.com/OSProcess.
 Does it mean it is dead? if yes, what is the alternative?


 Thanx
 Noury

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

On Fri, Oct 10, 2014 at 08:00:29AM +0200, p...@highoctane.be wrote:
 Le 9 oct. 2014 23:15, David T. Lewis le...@mail.msen.com a ??crit :
 
  Darn, I think we must have both done our updates on the same day. The last
  update I did was this (time stamp is GMT -4 hours) :
 
  CommandShell-dtl.76 Dave Lewis, 5 October 2014 5:29:32 pm
  CommandShell 4.6.9
 
  Improve pipe closing for PipeableOSProcess classcommand: and related
  methods to support use as stand alone command processor. An expression
  such as the following will now close all open handles when command
  processing is complete:
 
(PipeableOSProcess command: '/bin/echo this is a test') output
 
 
 No more need for the closePipes thing then?

Right. But please do test it in your applications to be sure, I really only
did simple testing and I am sure there may still chances for problems in
this area.

 
 Also, I see that there is apparently no AioPlugin support in Pharo 3.0. Or
 so say the tests of OSProcess.
 
 What is the state of affairs there?
 

Ask someone to include AioPlugin in the VM builds.

Dave

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

 2014-10-10 14:09 GMT+02:00 David T. Lewis le...@mail.msen.com:


 Right. But please do test it in your applications to be sure, I really
 only
 did simple testing and I am sure there may still chances for problems in
 this area.


 Hi Dave,

 Would it has a chance of slowing down things a lot?

 There is apparently something going very very slow compared to OSProcess
 4.5.11 when used from GitFileTree. So slow that I killed the image
 building
 script before it was over. Reverting GitFileTree to 4.5.11 solved it.


I don't this so, but I am not certain. The update for PipeableOSProcess
affects only the methods in PipeableOSProcess classcommand: and closely
related methods. If GitFileTree is using that idiom, then it is certainly
possible that I have introduced a bug that does not show up in my unit
tests.

 The absence of the AioPlugin warning may also be problematic, since it
 appears on stdout or stderr when Pharo is used on the command line.


Yes I should get rid that annoyance.

Thanks,
Dave

 Thierry



 
  Also, I see that there is apparently no AioPlugin support in Pharo
 3.0.
 Or
  so say the tests of OSProcess.
 
  What is the state of affairs there?
 

 Ask someone to include AioPlugin in the VM builds.

 Dave

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

 Le 10/10/2014 17:52, David T. Lewis a écrit :
 2014-10-10 14:09 GMT+02:00 David T. Lewis le...@mail.msen.com:


 Right. But please do test it in your applications to be sure, I really
 only
 did simple testing and I am sure there may still chances for problems
 in
 this area.


 Hi Dave,

 Would it has a chance of slowing down things a lot?

 There is apparently something going very very slow compared to
 OSProcess
 4.5.11 when used from GitFileTree. So slow that I killed the image
 building
 script before it was over. Reverting GitFileTree to 4.5.11 solved it.


 I don't this so, but I am not certain. The update for PipeableOSProcess
 affects only the methods in PipeableOSProcess classcommand: and
 closely
 related methods. If GitFileTree is using that idiom, then it is
 certainly
 possible that I have introduced a bug that does not show up in my unit
 tests.

 Typical code in GitFileTree is this:

 [
 c := PipeableOSProcess command: ''.
 output := c output.
 ...
 ] ensure: [c closePipes]

 Maybe it's triggering something.

 Thierry


Hmmm... I wonder if the #closePipes is causing a problem now. I don't know
if I ever tested to see if sending closePipes works after the pipes are
already closed, and my recent change closes the pipes after #output is
evaluated. I'll check it when I get home.

Thanks,
Dave

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

On Fri, Oct 10, 2014 at 01:32:27PM -0400, David T. Lewis wrote:
  Le 10/10/2014 17:52, David T. Lewis a ?crit :
  2014-10-10 14:09 GMT+02:00 David T. Lewis le...@mail.msen.com:
 
 
  Right. But please do test it in your applications to be sure, I really
  only
  did simple testing and I am sure there may still chances for problems
  in
  this area.
 
 
  Hi Dave,
 
  Would it has a chance of slowing down things a lot?
 
  There is apparently something going very very slow compared to
  OSProcess
  4.5.11 when used from GitFileTree. So slow that I killed the image
  building
  script before it was over. Reverting GitFileTree to 4.5.11 solved it.
 
 
  I don't this so, but I am not certain. The update for PipeableOSProcess
  affects only the methods in PipeableOSProcess classcommand: and
  closely
  related methods. If GitFileTree is using that idiom, then it is
  certainly
  possible that I have introduced a bug that does not show up in my unit
  tests.
 
  Typical code in GitFileTree is this:
 
  [
  c := PipeableOSProcess command: ''.
  output := c output.
  ...
  ] ensure: [c closePipes]
 
  Maybe it's triggering something.
 
  Thierry
 
 
 Hmmm... I wonder if the #closePipes is causing a problem now. I don't know
 if I ever tested to see if sending closePipes works after the pipes are
 already closed, and my recent change closes the pipes after #output is
 evaluated. I'll check it when I get home.


I tried this in both Squeak trunk and Pharo 3.0:

   (1 to: 100) collect: [:e | | c output |
  [c := PipeableOSProcess command: 'ls -lt'.
  output := c output.
  ] ensure: [c closePipes].
  output ].

I get the expected results, and no open file handles left over.

The #closePipes is no longer needed, so this can now be simplified as follows,
which also leaves no open file handles:

   (1 to: 100) collect: [:e | (PipeableOSProcess command: 'ls -lt') output].

I am not sure what the issue is when used with GitFileTree, but it does not
look like the change to PipeableOSProcess class command: is directly involved.
But ... I'm probably missing something.

If you are fairly sure the that problem is related to the the PipeableOSProcess
change, then the only other thing I can think of right now is that there may
be some other places in GitFileTree where PipeableOSProcess is being used in
a different way, and maybe I caused a problem there due the the change in the
#command: methods. But this a a complete guess, I really do not know.

Dave
 

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

On Tue, Oct 14, 2014 at 07:19:10AM +0200, Thierry Goubier wrote:
 Hi Dave,
 
 took me a while to do some testing. It seems there is an issue, but I 
 couldn't find out where. First, the data:
 
 Loading a package from a local gitfiletree repository (get from the 
 command a zipped git archive containing the package) :
 
 4.5.11, 4.6.8: 2132 msec.
 4.5.12, 4.6.9: 50847 msec.
 
 I tried to rewrite the main command, remove the closePipes, etc... No 
 change in the runtime, so your right.
 
 Profiling doesn't help much: execution time is mostly idle time (but 
 with a significant increase on PipeableOSProcess commands: goes from 
 invisible to 2700 ms. A user break in the middle shows it delay waiting 
 on output of the external process).
 
 Only hint I could see is that it seems to be linked with the size of the 
 output of the command.

Thierry,

Aha! I am quite sure that you have located the problem. There must be
something preventing the OS process from running to completion with
a larger amount of output data. It definitely sounds like a bug in my
latest update to Pipeable OSProcess.

Thank you very much for finding this. I will get it fixed as quickly
as I can.

Dave


 
 Le 11/10/2014 15:34, David T. Lewis a ?crit :
 On Fri, Oct 10, 2014 at 01:32:27PM -0400, David T. Lewis wrote:
 Le 10/10/2014 17:52, David T. Lewis a ?crit :
 2014-10-10 14:09 GMT+02:00 David T. Lewis le...@mail.msen.com:
 
 
 Right. But please do test it in your applications to be sure, I really
 only
 did simple testing and I am sure there may still chances for problems
 in
 this area.
 
 
 Hi Dave,
 
 Would it has a chance of slowing down things a lot?
 
 There is apparently something going very very slow compared to
 OSProcess
 4.5.11 when used from GitFileTree. So slow that I killed the image
 building
 script before it was over. Reverting GitFileTree to 4.5.11 solved it.
 
 
 I don't this so, but I am not certain. The update for PipeableOSProcess
 affects only the methods in PipeableOSProcess classcommand: and
 closely
 related methods. If GitFileTree is using that idiom, then it is
 certainly
 possible that I have introduced a bug that does not show up in my unit
 tests.
 
 Typical code in GitFileTree is this:
 
 [
 c := PipeableOSProcess command: ''.
 output := c output.
 ...
 ] ensure: [c closePipes]
 
 Maybe it's triggering something.
 
 Thierry
 
 
 Hmmm... I wonder if the #closePipes is causing a problem now. I don't know
 if I ever tested to see if sending closePipes works after the pipes are
 already closed, and my recent change closes the pipes after #output is
 evaluated. I'll check it when I get home.
 
 
 I tried this in both Squeak trunk and Pharo 3.0:
 
 (1 to: 100) collect: [:e | | c output |
[c := PipeableOSProcess command: 'ls -lt'.
output := c output.
] ensure: [c closePipes].
output ].
 
 I get the expected results, and no open file handles left over.
 
 The #closePipes is no longer needed, so this can now be simplified as 
 follows,
 which also leaves no open file handles:
 
 (1 to: 100) collect: [:e | (PipeableOSProcess command: 'ls -lt') 
 output].
 
 I am not sure what the issue is when used with GitFileTree, but it does not
 look like the change to PipeableOSProcess class command: is directly 
 involved.
 But ... I'm probably missing something.
 
 If you are fairly sure the that problem is related to the the 
 PipeableOSProcess
 change, then the only other thing I can think of right now is that there 
 may
 be some other places in GitFileTree where PipeableOSProcess is being used 
 in
 a different way, and maybe I caused a problem there due the the change in 
 the
 #command: methods. But this a a complete guess, I really do not know.
 
 Dave
 
 
 
 

Re: [Pharo-dev] OSProcess in 3.0

[David T. Lewis]

On Tue, Oct 14, 2014 at 09:35:44PM +0200, Thierry Goubier wrote:
 
 
 Le 14/10/2014 14:15, David T. Lewis a ?crit :
 On Tue, Oct 14, 2014 at 07:19:10AM +0200, Thierry Goubier wrote:
 Hi Dave,
 
 took me a while to do some testing. It seems there is an issue, but I
 couldn't find out where. First, the data:
 
 Loading a package from a local gitfiletree repository (get from the
 command a zipped git archive containing the package) :
 
 4.5.11, 4.6.8: 2132 msec.
 4.5.12, 4.6.9: 50847 msec.
 
 I tried to rewrite the main command, remove the closePipes, etc... No
 change in the runtime, so your right.
 
 Profiling doesn't help much: execution time is mostly idle time (but
 with a significant increase on PipeableOSProcess commands: goes from
 invisible to 2700 ms. A user break in the middle shows it delay waiting
 on output of the external process).
 
 Only hint I could see is that it seems to be linked with the size of the
 output of the command.
 
 Thierry,
 
 Aha! I am quite sure that you have located the problem. There must be
 something preventing the OS process from running to completion with
 a larger amount of output data. It definitely sounds like a bug in my
 latest update to Pipeable OSProcess.
 
 The strange thing is that it runs to completion (i.e. I got no errors; 
 if any data was missing or corrupted on output, my load would fail); but 
 a lot slower than it should be.
 
 Thank you very much for finding this. I will get it fixed as quickly
 as I can.
 
 Thanks. I'm unable to help you much, I'm afraid :(

Thierry,

I do not yet have a solution to this, but I have been able to confirm
that the performance difference that you see is primarily associated with
the AioPlugin not being present in your VM.

When I changed PipeableOSProcess to handle pipe handle closing properly,
I used a buffered output stream that is driven by aio events. When data
is available to be read from the external OS process, an aio notification
pulls the available data into the buffered stream, and when no more data
is available, the pipe (file) handles are closed.

If the plugin is not present in the VM, I provided a polling mechanism
to read data from the external OS process. Apparently this is very
inefficient, and I am sure it could be greatly improved. That is the
cause of the extremely slow performance that you are seeing.

So in summary: The enhancement to PipeableOSProcess relies on an aio
event driven reader, and if the AIO plugin is missing, the fallback code
relies on polling, and is horribly inefficient.

More to follow, and thank you again for identifying the problem.

If you can convice your VM vendor to include AioPlugin in your VM, that
would be helpful also ;-)

Dave

Re: [Pharo-dev] PharoNOS

[David T. Lewis]

On Thu, Oct 16, 2014 at 06:52:48PM -0700, mikefilonov wrote:
 I'll take a look at AioPlugin issue. How I can test if it is loaded well?

With OSProcess installed, evaluate AioEventHandler aioPluginPresent

Dave

Re: [Pharo-dev] Command Shell in Pharo 4

[David T. Lewis]

On Tue, Oct 28, 2014 at 11:09:24AM +0200, kilon alios wrote:
 So I decided to give Command Shell a spin using today's Pharo 4 release and
 I have some issues
 
 *a) The text is too small* . After some investigation I found that the size
 is defined in ShellWindowMorph classtextStyle
 
 textStyle
 A fixed width font for the text morph
 
 ^ (TextStyle named: 'DefaultFixedTextStyle')
 ifNil: [TextStyle default]
 
 I assume there is a reason why a fixed size font is used . I changed it to
 just TextStyle default and works fine.

The intent is to provide a readable fixed font for a text terminal window.
It does not seem to work on Pharo. TextStyle named: 'DefaultFixedTextStyle'
does not look like a fixed font to me, and as you say it is too small to be
readable.

I'm not too familiar with the font handling in Pharo, so if you can suggest
a font that works better, that would be great.

 
 *b) no ~ folder.* Trying a cd ~ command does not work it complains that
 the folder does not exist.

CommandShell is a partial simulation of a unix shell. Many features of a
real shell are not implemented. In this case, the $~ character is not being
interpreted as it would be in a real shell, instead it is simply treated
as a text character, so it does nothing at all.

 
 *c) not able to run shell scripts or other scripts* (eg. python scripts).
 For example if I try ./blender.sh even though this command works fine in
 my macos terminal and iTerm2 , it does not work in Command Shell and
 complains that it cannot find the command.

It may be that default directory conventions are different either on
Pharo (compared to Squeak) or on OS X (compared to generic unix). I don't
have a Mac so I'm afraid that I am guessing at this.

FYI, CommandShell is keeping track of its own notion of the current working
directory. If you do a 'pwd' command in your command shell, it should show
you what the command shell thinks is its current working directory. This also
works in Widows by the way, which was something of a challenge handling
those drive letters ;-)

HTH,
Dave

Re: [Pharo-dev] Command Shell in Pharo 4

[David T. Lewis]

Hi Kilon,

On Tue, Oct 28, 2014 at 02:12:12PM +0200, kilon alios wrote:
 Why not use default as I did ? why the font needs to be fixed ? I am sorry
 I am not familiar with fonts in pharo so much, I did not even know what
 bitmap font is before I joined Pharo. For me it makes more sense to use the
 default because then it will use what the user have chosen in pharo
 settings.

I am not sure that I am reading your question correctly, but the reason for
selecting a fixed font rather than a proportional font is that it provides
a nice display output for many of the unix command line tools. Back in the
day, a terminal was a device with fixed width character displays, and programs
that write to the display would often format the output data so that it would
print nicely on a display or printer with fixed width character cells. The
CommandShell window is a simple emulation of devices like this. It tries
to use a fixed width font so that a unix utility command will display output
in the expected character positions.

In Squeak, TextStyle named: 'DefaultFixedTextStyle' answers a reasonable
looking fixed width font, so that's what I use as the default. If no reasonable
fixed font was available, I fell back on TextStyle default. The Pharo image
that I am looking at now seems to answer a proportional font as its
DefaultFixedTextStyle, so maybe there is something that needs to be fixed
there.

By the way, if you enter this in your CommandShell window, you will find
that it is evaluated as a Smalltalk expression rather than as a unix
shell command:

$ TextStyle named: 'DefaultFixedTextStyle'
a TextStyle Bitmap DejaVu Sans 7
$ 

That's one of the things I was trying to do with CommandShell, mix up
Smalltalk expression evaluation with unix commands, and be able to
mix them up in unix command pipelines like this:

$ { #one. #two . #three . Smalltalk . true } ! | cat
{#one. #two. #three. Smalltalk. true}
$ 


 
 I dont know if you noticed my reply but I did use pwd and did try to run
 the script with the bash command. it looks like that pwd is not in sync
 with what Command Shell thinks as its current working directory , actually
 my bash file complained that the tool tried to run the script from the path
 of the actual pharo image which should not be the case .

IIRC, there have been some changes made in Pharo versus Squeak that affect
default working directories, possibly the image directory as opposed to
the VM directory. I don't recall the details, but you may be seeing some
issue related to that. CommandShell tries to start with some initial
working directory, but it keeps track of its own current working directory
after it has started. This works very differently on Unix and Windows,
so it was necessary to keep track of it independent of what the image
and/or VM were using as defaults. It may be that changes in Pharo have
affected this, or maybe you are seeing issues related to OS X, I'm not
sure.

 
 By the way congratulation on your tool, I see you have put a  lot of effort
 in it and I am impressed by the amount of code and how well its organised.
 I will definetly keep exploring the code and further improve it if I can :)
 

Thanks! It was sort of a hobby experiment, so don't expect it to be a
professional grade replacement for /bin/bash ;-)

Dave

Re: [Pharo-dev] AioPlugin is back in linux VMs

[David T. Lewis]

On Tue, Oct 28, 2014 at 06:53:49PM +0200, kilon alios wrote:
 whats the purpose of the plugin ? How we can test it ?

AioPlugin provides support for asynchronous IO event notification. When
an external IO event occurs, the event registration causes a Smalltalk
semaphore to be signaled. A process can wait on this semaphore, thereby
acting as an event handler for the external event.

Code for the plugin is at http://www.squeaksource.com/AioPlugin.

For testing and examples, see class AioEventHandlerExample in package OSProcess,
and AioEventHandlerTestCase in OSProcess-Tests.

Dave


 
 On Tue, Oct 28, 2014 at 6:51 PM, Esteban Lorenzano esteba...@gmail.com
 wrote:
 
  Hi,
 
  I reintroduced AioPlugin into the linux VM.
  In Mac it is there since always.
  In windows is irrelevant.
 
  So well, if you use linux and download the latest vm, you can play with it
  and check that everything is ok (or the contrary) :)
 
  Esteban
 

Re: [Pharo-dev] SmalltalkHub bugs hangs :-(

[David T. Lewis]

On Thu, Nov 27, 2014 at 04:19:20PM +0100, Esteban Lorenzano wrote:
 
  On 27 Nov 2014, at 16:05, Levente Uzonyi le...@elte.hu wrote:
  
  On Thu, 27 Nov 2014, Noury Bouraqadi wrote:
  
  SmalltalkHub is slow, so don?t click too fast :-(
  
  While I?m grateful to developers of SmalltalkHub and previously 
  SqueakSource, I believe as a small community, we cannot afford developing 
  everything by ourselves. We don?t have enough man-power. We can see the 
  symptoms since SmalltalkHub is in beta stage since way too long...
  
  People on this list used to say the same about squeaksource.com. But it
  has been working flawlessly since it got updated to the latest version
  (which implies that a few bugs got fixed).
  
  What makes you think that tracking down the bugs in SmalltalkHub and fixing 
  them wouldn't be enough?
  
  Based on my experience a single image - running well designed software - is 
  able to serve about ten million requests a day. I think that the actual 
  load on smalltalkhub.com is about three magnitudes lower than that.
 
 because:
 1) the reason why squeaksource survived is because there is an smalltalkhub 
 absorbing most of the packages. 

No. The reason that squeaksource.com survived is that someone decided to put
effort into maintaining it.

The same will be true for SmalltalkHub. If you decide to make it work, then
you will succeed. If you decide that it is not worth the effort, then it will
fail.

Dave


 2) I never say it was not possible to keep it running. I said it does not 
 worth the effort, which is a complete different thing. Of course, if someone 
 wants to take smalltalkhub (there is even a configuration for it) and fix the 
 problems that emerge, I will be more than happy to update the production 
 image. 
 
 Esteban
 
  
  Levente
  
  
  It?s better to use some mainstream platform such as github.
  We?ll we have support for it in Pharo 4?
  
  It would be interesting also from the communication point of view to make 
  the world a little bit more aware of Pharo.
  
  Noury
 

Re: [Pharo-dev] Please delete .mcz file in http://www.squeaksource.com/MetacelloRepository

[David T. Lewis]

On Wed, Dec 03, 2014 at 02:01:28PM -0800, Dale Henrichs wrote:
 The .mcz is triggering Gofer errors for me during automated loads and of
 course the place to get a new version of Gofer from is
 http://www.squeaksource.com/MetacelloRepository ...
 
 If someone with admin privileges could deleted the .mcz file that'd be
 great!

Hi Dale,

I have admin access to squeaksource.com, so I will delete it. Note, I have
noticed problems in the past where deleted versions return several days
later like zombies. I will also manually delete the file from the ss file
directory, but let me know if it comes back a few days from now.

This is the file that was saved with the file name .mcz:

  File: .mcz
  Author:   
  Timestamp:2 December 2014 11:14:44 am
  UUID: ae6100fd-7bf3-4845-a640-d03e24b3588a
  Ancestors:ConfigurationOfRefactoringBrowser-SK.77
  Message:  added update methods to AAAObject and subclasses
  Diff: .diff
  Changeset:(@ConfigurationOfRefactoringBrowser-SK.77).cs
  Downloads:Web: 1

I saved a backup copy of the file in case it is needed, but whoever was
trying to commit the ConfigurationOfRefactoringBrowser update will need
to save it again with the intended file name.

HTH,
Dave

Re: [Pharo-dev] Question on AioPlugin

[David T. Lewis]

On Wed, Feb 04, 2015 at 07:18:03AM +0100, Thierry Goubier wrote:
 Hi all,
 
 anybody knows how to check if the AioPlugin is included in the current
 stable Pharo vm(*)? OSProcess is complaining of a missing AioPlugin, and I
 remember a thread where this was solved by integrating it in the vm build.

Hi Thierry,

Evaluate AioEventHandler aioPluginPresent to check if the plugin is present.

Dave

Re: [Pharo-dev] SmallInteger asByteArray for negative numbers

[David T. Lewis]

 I was going to complain about expecting such niceties as a built in
 2s-complement rather than program it myself (I am a bit rusty on such
 matters), but when I look what should I find?

 (-123 asTwosComplement: 16r) asByteArray.   = #[255 255 255
 133]

 cheers -ben


Hi Ben,

You might want to look at http://www.squeaksource.com/TwosComplement

Dave

Re: [Pharo-dev] [squeak-dev] OSProcess stdErr issue on Windows

[David T. Lewis]

On Sun, Jan 25, 2015 at 01:41:01PM +0800, Ben Coman wrote:
 On Mon, Jan 12, 2015 at 6:10 AM, Thierry Goubier thierry.goub...@gmail.com 
 wrote:
 
  2015-01-11 20:42 GMT+01:00 David T. Lewis le...@mail.msen.com:
 
  I copied ConfigurationOfOSProcess-ThierryGoubier.33 to the OSProcess
  repository. An update to ConfigurationOfOSProcess will be needed to
  get the latest version.
 
 Sorry for the delay getting back to this.  I updated the Configuration and
 tested it loads okay on latest Pharo. You can copy...
 
 ConfigurationOfOSProcess-BenComan.36

Done.

Thanks Ben!

Dave

Re: [Pharo-dev] OSProcess bug ?

[David T. Lewis]

Hi Jannik,

One other thing I just thought of: Check and see if your rostopic program
is actually writing to standard output, as opposed to writing directly to
/dev/tty. From a terminal window, I think you could do this:

$ rostopic  out.txt

If you run this get output written to the out.txt file, that is good. If
you run it and see output to your terminal window, it would mean that the
rostopic program was writing directly to /dev/tty (so there would never be
any output to read with a PipeableOSProcess).

I would not worry about the missing AioPlugin. Your code should still work
without the plugin.

Dave


 Hi Dave,

 2015-02-25 1:13 GMT+01:00 David T. Lewis le...@mail.msen.com:

 On Tue, Feb 24, 2015 at 08:33:41PM +0100, jannik laval wrote:
  Hi pharoers,
 
  I am playing with OSProcess.
  I am using an example, and I am annoyed with a strange bug:
 
  Here is the code I want to execute, and does not work:
  
  |thread process output|
  myCmd := 'rostopic bw /turtle1/pose'.
  thread := [process := (PipeableOSProcess command: myCmd) ] fork.
  3 second wait.
  output := process upToEnd.
  output inspect.
  thread terminate.
  ===

 Hi Jannik,

 I am not familiar with the rostopic command, but it is common for some
 Unix programs to flush their output when the stdout is connected to
 /dev/tty (a terminal), but not necessarily if connected to a pipe or
 to a file. So it may be that the running rostopic program is producing
 output, but that the output has not yet been flushed to the pipe
 connected
 to your PipeableOSProcess.


 Thank you for your explanation.
 So, what I can do is to send a sigint to terminate the process and receive
 the info.
 But sending process processProxy sigint. does nothing.

 By the way I have a warning on the AIOPlugin... Probably I should
 reinstall
 my VM.

 I also try process flush, but it does not do anything.




 Your #upToEnd call will read the available data from the OS pipe, but it
 will not read data that was written by the rotopic program, but not
 yet flushed out to the OSPipe. So I am guessing that this may be why the
 data seems to be missing.

 I notice also that you are putting your PipeableOSProcess into a
 separate
 Smalltalk process using #fork. This is probably not necessary, so try
 getting rid of the #fork and the later thread terminate. It will not fix
 the problem for you, but it will simplify the code.



 Ok, it is simpler like that. You are right, it does not help :)

 Thank you for your help.



 Dave


 
  'rostopic bw /turtle1/pose' is a command that is a loop displaying the
  current bandwidth of a ROS process. In a terminal, it works fine. With
  OSProcess the output value is empty.
 
  My source code is correct: if I replace myCmd by :
  myCmd := 'ping localhost'.
 
  It is a loop too, and it returns a correct string.
 
  I have clearly no idea of where I have to search... Any help or
 discussion
  about that would be great :)
 
  Thank you,
 
  Oups, some infos:
  I am using Ubuntu 14.04
  Pharo3.0 with Moose 5.0, latest update: #30863
  The script I use (rostopic) is a python script (does it impact ?)
 
 
  --
 
  ~~Jannik Laval~~
  ??cole des Mines de Douai
  Enseignant-chercheur
  http://www.jannik-laval.eu
  http://www.phratch.com
  http://www.approchealpes.info
  http://car.mines-douai.fr/




 --

 ~~Jannik Laval~~
 École des Mines de Douai
 Enseignant-chercheur
 http://www.jannik-laval.eu
 http://www.phratch.com
 http://www.approchealpes.info
 http://car.mines-douai.fr/

Re: [Pharo-dev] OSProcess bug ?

[David T. Lewis]

On Tue, Feb 24, 2015 at 08:33:41PM +0100, jannik laval wrote:
 Hi pharoers,
 
 I am playing with OSProcess.
 I am using an example, and I am annoyed with a strange bug:
 
 Here is the code I want to execute, and does not work:
 
 |thread process output|
 myCmd := 'rostopic bw /turtle1/pose'.
 thread := [process := (PipeableOSProcess command: myCmd) ] fork.
 3 second wait.
 output := process upToEnd.
 output inspect.
 thread terminate.
 ===

Hi Jannik,

I am not familiar with the rostopic command, but it is common for some
Unix programs to flush their output when the stdout is connected to
/dev/tty (a terminal), but not necessarily if connected to a pipe or
to a file. So it may be that the running rostopic program is producing
output, but that the output has not yet been flushed to the pipe connected
to your PipeableOSProcess.

Your #upToEnd call will read the available data from the OS pipe, but it
will not read data that was written by the rotopic program, but not
yet flushed out to the OSPipe. So I am guessing that this may be why the
data seems to be missing.

I notice also that you are putting your PipeableOSProcess into a separate
Smalltalk process using #fork. This is probably not necessary, so try
getting rid of the #fork and the later thread terminate. It will not fix
the problem for you, but it will simplify the code.

Dave


 
 'rostopic bw /turtle1/pose' is a command that is a loop displaying the
 current bandwidth of a ROS process. In a terminal, it works fine. With
 OSProcess the output value is empty.
 
 My source code is correct: if I replace myCmd by :
 myCmd := 'ping localhost'.
 
 It is a loop too, and it returns a correct string.
 
 I have clearly no idea of where I have to search... Any help or discussion
 about that would be great :)
 
 Thank you,
 
 Oups, some infos:
 I am using Ubuntu 14.04
 Pharo3.0 with Moose 5.0, latest update: #30863
 The script I use (rostopic) is a python script (does it impact ?)
 
 
 -- 
 
 ~~Jannik Laval~~
 ??cole des Mines de Douai
 Enseignant-chercheur
 http://www.jannik-laval.eu
 http://www.phratch.com
 http://www.approchealpes.info
 http://car.mines-douai.fr/

Re: [Pharo-dev] Transcript needs your love

[David T. Lewis]

On Fri, May 08, 2015 at 08:10:33PM +0200, Igor Stasenko wrote:
 
 i think there is a fundamental difference between 'stream' and 'update on
 the screen' . Choose one and stick to it.
 We shall separate such responsibilities and NEVER ever merge them again ,
 even if it was cool, nice, soft and puffy in the past.
 Yes, sure, we can then build the facade on top of that providing similar
 behavior of what was in the past..
 
 But if not, then it will always be something that will forever be causing
 problem, since it is conceptually wrong and looks like attempt to cross
 breed two different species.

OT
I have no particular opinions about Transcsript, but I would like to say
that I am very happy to have Igor back in the Squeak/Pharo/Cuis/Smalltalk
discussions. I may not agree with everything, but I really appreciate his
thoughtful perspective. There are a few people whose posts I will always
read, and Igor is one of them.
/OT

Dave
 

Re: [Pharo-dev] Transcript needs your love

[David T. Lewis]

On Mon, May 11, 2015 at 11:52:37PM +0200, Igor Stasenko wrote:
 On 11 May 2015 at 22:54, Igor Stasenko siguc...@gmail.com wrote:
 
  Speaking about penetrating multiple layers.. Lemme tell you the story:
 
  One day, a brave hero (let's call him Stef :) ) decided to save the
  world... err  improve Transcript and make it thread-safe.
  And everything vent well until he stumbled upon #endEntry logic, because
  he was unable to figure out how it is handled and where it is handled..
  He asked me for help and we sat together and looked for what can be done..
  After many months and fierce battles (longer that any developer would
  except), we finally discovered that #endEntry logic is handled ..
  (drumroll...) in TextMorph,
  as a separate case in case-statement in its #changed: method.
 
  Sure it was a long ago, and my memory could fail me, i might be completely
  wrong on some details, and maybe instead we were drinking ale with fairies
  in the pub, and there was no any fights. But that's what is fairy tale for
  :)
 
 
 I keep coming to this, so you guys can understand what is it..
 There's a fairy tale in our culture where hero needs to kill an Immortal
 King,
 and he asked for advice from old which that told him how to do it:
  - there's a Mammoth, inside it a wolf, and inside wolf a rabbit, and
 inside rabbit - a duck, and inside duck, an egg, and inside egg a needle,
 and once you break the needle, so the Immortal King will be dead..
 So, to recap, in order to kill an Immortal King, one must kill mammoth,
 wolf, rabbit, duck, break an egg and here we are.. job done!
 Now what is good and cool for fairy tale, not that cool and and suitable
 for daily programming tasks.. If one needs to break a needle - there should
 be a way to do just that, without need to peruse through piles of dead
 animal victims to achieve the goal.


Thanks for sharing the fairy tale :-)
 

Re: [Pharo-dev] FilePlugin maximum size

[David T. Lewis]

On Sun, Jun 07, 2015 at 04:02:10PM +0200, Norbert Hartl wrote:
 I just encountered a
 
  
 
 I guess it is because the position of 1073750701 is just slightly bigger than 
 (2 raisedTo: 30 = 1073741824). Is there a way to get a full 32 bit range?


You need a VM compiled with large file support 
http://en.wikipedia.org/wiki/Large_file_support

For gcc, the compiler flag is -D_FILE_OFFSET_BITS=64

Dave

 

Re: [Pharo-dev] GitFileTree dependencies

[David T. Lewis]

On Mon, Jul 06, 2015 at 11:03:06PM +0200, Peter Uhn??k wrote:
 On Mon, Jul 6, 2015 at 10:44 PM, Thierry Goubier thierry.goub...@gmail.com
 wrote:
 
 
  I may look at one point into adding support for pseudo-tty. Does anybody
  knows if pseudo-tty are available with Microsoft Windows?
 

I don't know of any direct equivalent on Windows. But people do write
good terminal emulators that run on Windows, so there must be a way to
do it.

 
 Maybe PuTTY? But I've only ever used that for ssh, so I don't know if it
 has the functionality you need.

Googling for putty api turns up quite a few links, so maybe something
there would help.

Dave

Re: [Pharo-dev] PID of VM ?

[David T. Lewis]

On Mon, Sep 14, 2015 at 02:08:54PM +0200, Sven Van Caekenberghe wrote:
> 
> > On 14 Sep 2015, at 11:12, Skip Lentz  wrote:
> > 
> > 
> >> On Sep 14, 2015, at 11:04 AM, Sven Van Caekenberghe  wrote:
> >> 
> >> Hi,
> >> 
> >> Is there a way to get the operating system PID of the running Pharo VM 
> >> from inside the image ?
> > 
> > If I???m not mistaken, it is what OSProcess is for. I think this should be 
> > it:
> > 
> >> OSProcess thisOSProcess pid.
> > 
> > The comment of thisOSProcess says:
> > 
> > "Answer a single instance of the class corresponding to the OS process in 
> > which this Smalltalk image is executing."
> 
> OK, that seems to works in Pharo 5. Thanks.
> 
> I don't know the overall status of OSProcess in Pharo though. Is is 100% 
> supported in all current versions, 3.0, 4.0 and 5.0 ? Is it based on a plugin 
> or on native boost ? Should it be included by default ?
> 

It should be working in all Pharo versions. It is based on a VM plugin.

I am the author of OSProcess, and I try to keep it working in Pharo as well
as Squeak. Thierry Goubier also works on OSProcess for Pharo (and he has write
access to the repository).

No, OSProcess should not be included in the image. It is intended to be
an external package.

Dave

Re: [Pharo-dev] PID of VM ?

[David T. Lewis]

> On Mon, Sep 14, 2015 at 8:25 PM, Esteban Lorenzano <esteba...@gmail.com>
> wrote:
>>
>>> On 14 Sep 2015, at 14:21, David T. Lewis <le...@mail.msen.com> wrote:
>>>
>>>
>>> No, OSProcess should not be included in the image. It is intended to be
>>> an external package.
>>
>> mmm… we want to include it as more and more people requires it and we
>> can profit from having it in the image (same as FFI).
>> why do you think it should be external?
>
> We should distinguish between having it traditionally-in-Image where
> that is the only place it exists and fixes are integrated, and the
> current trend (e.g. in-Image Glamorous Tools) where fixes are
> integrated "upstream" into an external package
>

Yes, that is what I meant.

Dave

Re: [Pharo-dev] Adapting the launcher to Spur VM

[David T. Lewis]

> On 17-12-15 18:21, Eliot Miranda wrote:
>> Hi Stephan,
>>
>>  I love you too ;-).  Just in case you weren't aware David Lewis'
>> ImageFormat package on http://source.squeak.org/VMMaker contains the
>> "official" approach to image version ids and will provide you with "the
>> official way" (tm) to test for a Spur image, including 64-bits.
>
> Ah, there is a lot more to do than testing for 6505, I see.
> I suspect ImageFormat needs an update, the last version is from 2011 and
> Spur is suspiciously missing. I'll add that instead.
>
> Stephan
>

Make sure you got the latest version from the repository, it should be
fully up to date, including the image format info for the 64-bit Spur
image.

Dave

Re: [Pharo-dev] Status of GitFileTree in Pharo 5.0 and Spur

[David T. Lewis]

On Sun, Jan 03, 2016 at 04:54:48PM +0100, Thierry Goubier wrote:
> Le 03/01/2016 16:49, Mariano Martinez Peck a ??crit :
> > ok, I am giving it a try but it seems it's not working or... mostly
> >likely, I am doing something wrong.
> >I just created a repo via github web interface, then I clone it to my
> >machine, and then I tried to add the gitfiletree repo in MC Browser, but
> >I get an error in: #currentBranch because #gitBranchs  answered an empty
> >array. But the `git branch` command was really executed as I can see the
> >output of it in the stdout of the VM. So I guess OSProcess stdout
> >setting may be what is not working?
> 
> You're on Mac when you do that, right?
> 
> OSProcess output redirection doesn't work in the latest Mac VM... my 
> fault (introduced in the OSProcess plugin) :( but I have no idea why the 
> fd dup / pipes work on Linux and not on Mac OS X.

Hi Thierry,

I cannot say if it is related to the problem you are discussing here for
Mac VMs, but I should mention that there is an error in the oscog branch
of OSProcessPlugin (the one that you are using for Pharo VMs). The following
test passes in the main branch of OSPP but fails for oscog version:

  (UnixProcessAccessorTestCase selector: #testRedirectStdOutTo) run

If you are compiling your own VM, you may want to try using the main
branch of OSPP (VMConstruction-Plugins-OSProcessPlugin-dtl.40) rather
than the oscog version (VMConstruction-Plugins-OSProcessPlugin.oscog-eem.50)
and see if it makes a difference.

Dave
 

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo Consortium Sponsored Development Effort

[David T. Lewis]

Mariano,

I am really happy to see this and I look forword to helping you in
any way I can.

Dimitris,

It is really interesting that you should suggest this. I think it may
be a bit off topic for Mariano's initial goals, but Mariano is the
driving force behind Fuel, which is the serializer that I use for
RemoteTask in OSProcess/CommandShell. This attempts to do much of
what you suggest, using Fuel to return the results of processing
executed in a forked image (in the unix sense of "forked", i.e.
extremely memory efficient).

So maybe if I can help Mariano with his project (which of course I
plan to do anyway), then he might be able to help me turn RemoteTask
into something reliable enough for general purpose use.

Another nice connection here is that Eliot's Spur image model will
make #forkSqueak (the primitive behind RemoteTask) more efficient
in cases where the cooperating images do a lot of processing, because
memory changes are more localized in the Spur memory model. In principle
it should be possible to run very large object memories in 64-bit Spur,
while forking cooperating images that consume very little additional
real memory.

Dave


On Sat, Dec 19, 2015 at 01:33:07PM +, Dimitris Chloupis wrote:
> I am very excited for such a project.
> 
> I hope you manage to fix the stdout problem on macos.
> 
> One thing I would like to see is an expansion of the ability of OSProcess
> to launch multiple instances of pharo and make them communicate with each
> other. Thats a way to have pharo running on multiple cores. Maybe establish
> a framework of communication of images that will be based on the existing
> design of the fork mechanism in Pharo.
> 
> Python is doing something similar with multiprocessing module  which
> launches multiple processes of cpython executable and makes them
> communication with each other through a similar api to its threading
> module. This way python takes advantage of multiple cores. Python threads
> like pharo are green threads and pharo like python has a GIL like
> mechanism.
> 
> Taking into account that current computers have at least 4 cores, its not a
> bad idea to speed up pharo like this at least  400% ;)
> 
> On Sat, Dec 19, 2015 at 3:24 PM Mariano Martinez Peck 
> wrote:
> 
> > Dear all,
> >
> > I am tremendously happy to announce you that Pharo Consortium will sponsor
> > yet another development effort. In this particular case, it's my honor to
> > carry on such an effort and I will be developing and improving a few things
> > and ideas. All the contract and paperwork has already been done so it's
> > time to start working.
> >
> > Regarding the developments itself, we discussed about these topics:
> >
> > 1) Experiment with a simpler yet more limited OSProcess alternative to
> > execute OS commands.
> > 2) Improving FileSystem in order to better deal with some POSIX stuff like
> > symbolic links, unix file permissions, etc etc.
> >
> > I have already started with 1). OSProcess is super complete and it
> > involves some packages (OSProcess / CommandShell) as well as some VM
> > plugins (OSProcessPlugin, AIOPlugin). OSProcess provides lots of features
> > (like forking the running image) but we would like to focus only in
> > executing OS commands. In addition, OSProcess dates from quite some years
> > ago when many of the current infrastructure features did not exist yet.
> >
> > So the idea is to think a simpler alternative to execute OS commands,
> > using new features such as threaded FFI (for example, for reading async
> > from pipes), pinned objects, etc. We want to use FFI as much as possible
> > rather than VM plugins.
> >
> > From the image side, we are thinking about an API which would look like a
> > builder-like API (FileSystem, XStream, etc).
> >
> > We have been discussing with David Lewis (OSProcess author) as well as
> > with Eliot Miranda, Esteban Lorenzano, Damien Pollet, Stephane Ducasse, etc
> > in order to agree in a project that would be worth doing (otherwise we
> > would continue using existing tools). And they have all been very kind with
> > me providing positive discussions.
> >
> > I will soon do a survey to measure the most common use cases of OSProcess
> > and related tools.
> >
> > If you have any thought, question, feedback, or whatever, please let us
> > know.
> >
> > Finally, note that I will be doing this project together with another
> > client's project (Quuve) and my life does not have much free time these
> > days...so please be patient!!!
> >
> > Best,
> >
> > --
> > Mariano
> > http://marianopeck.wordpress.com
> >

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo Consortium Sponsored Development Effort

[David T. Lewis]

On Sun, Dec 20, 2015 at 05:07:47PM +0800, Pierce Ng wrote:
> On Sun, Dec 20, 2015 at 08:56:43AM +, Dimitris Chloupis wrote:
> > Fuel I assume enters here the equation as a data exchange format , the
> > problem I have with fuel is that its not backward compatible which for me
> 

Fuel works well in this case. Version compatibility is not an issue, because
when you fork an image/VM the two cooperating images are essentially identical.

> Once you have a set of OS processes running Pharo how they talk to each other
> is up to you no? E.g. the Pharo instances could use ZeroMQ, XML-RPC, JSONRPC,
> msgpack, coordinate via Redis/SQLite/PostgreSQL, etc.

Any of those would work, but Fuel is a good way to enable all kinds of
objects to be copied from one image to another through a stream. In the
case of RemoteTask, all of the results of a remote computation can be put
into a single result object (an array or a dictionary or whatever), and
that result can be passed back to the parent image by reading the single
object. So - fork a copy of the image, have it do some work, and read the
result object from a stream. You know you are done when you have read
exactly one object.

Dave

Re: [Pharo-dev] endless loop again

[David T. Lewis]

On Wed, Jun 08, 2016 at 11:11:13AM +0200, Sven Van Caekenberghe wrote:
> 
> > On 08 Jun 2016, at 10:40, Henrik Johansen  
> > wrote:
> > 
> > 
> >> On 04 Jun 2016, at 9:13 , stepharo  wrote:
> >> 
> >> 
> >> 
> >> Le 3/6/16 ?? 19:16, Esteban A. Maringolo a ??crit :
> >>> Regarding the first part of your email, about UI lockups, it is almost
> >>> unavoidable, because in Pharo almost everything happens in the main
> >>> thread, as you pointed out.
> >> Yes. This is all my point.
> > 
> > IMHO, all Pharo beginner tutorials should start by showing off Cmd - dot 
> > (or Alt - dot, on Win / Linux).
> > 
> > Not just because it's cool, but because it's necessary :)
> > 
> > Cheers,
> > Henry
> 
> Very true.
> 
> Yet, and I have already mentioned this years ago, it is technically totally 
> possible to properly detect massive stack growth, before things get out of 
> control. Sadly, VM people think this is not needed (or too costly).
> 

Hi Sven,

I think that I remember this topic being discussed from time to time, but
I cannot recall specifics. Do you have a pointer to any discussions on the
mailing lists? I'm sure that there would be interest in doing this if it
can be done in an efficent manner.

Thanks,
Dave




> Consider the following in Common Lisp (LispWorks):
> 
> CL-USER 1 > (defun fac (n) (if (< n 2) 1 (* n (fac (1- n)
> FAC
> 
> CL-USER 2 > (fac 5)
> 120
> 
> CL-USER 3 > (fac 100)
> 9332621544394415268169923885626670049071596826438162146859296389521753229915608941463976156518286253697920827223758251185210916864
> 
> CL-USER 4 > (fac 400)
> 
> Stack overflow (stack size 31744).
>   1 (continue) Extend stack by 50%.
>   2 (abort) Return to level 0.
>   3 Return to top loop level 0.
> 
> Type :b for backtrace or :c  to proceed.
> Type :bug-form "" for a bug report template or :? for other options.
> 
> CL-USER 5 : 1 > :c 1
> 
> Stack overflow (stack size 48128).
>   1 (continue) Extend stack by 50%.
>   2 (abort) Return to level 0.
>   3 Return to top loop level 0.
> 
> Type :b for backtrace or :c  to proceed.
> Type :bug-form "" for a bug report template or :? for other options.
> 
> CL-USER 6 : 1 > :c 1
> 64034522846623895262347970319503005850702583026002959458684445942802397169186831436278478647463264676294350575035856810848298162883517435228961988646802997937341654150838162426461942352307046244325015114448670890662773914918117331955996440709549671345290477020322434911210797593280795101545372667251627877890009349763765710326350331533965349868386831339352024373788157786791506311858702618270169819740062983025308591298346162272304558339520759611505302236086810433297255194852674432232438669948422404232599805551610635942376961399231917134063858996537970147827206606320217379472010321356624613809077942304597360699567595836096158715129913822286578579549361617654480453222007825818400848436415591229454275384803558374518022675900061399560145595206127211192918105032491008000
> 
> CL-USER 7 > (defun infinite () (cons 'a (infinite)))
> INFINIT
> 
> CL-USER 8 > (infinite)
> 
> Stack overflow (stack size 80896).
>   1 (continue) Extend stack by 50%.
>   2 (abort) Return to level 0.
>   3 Return to top loop level 0.
> 
> Type :b for backtrace or :c  to proceed.
> Type :bug-form "" for a bug report template or :? for other options.
> 
> CL-USER 9 : 1 > :a
> 
> CL-USER 10 >
> 
> Just having an upper limit to the stack catches 99% of infinite loops - that 
> would be nice to have.
> 
> This has existed for 10s of years AFAIK.
> 
> Sven
> 
> PS: Of course, the stack size can be set larger to begin with, if necessary. 
> I believe it is even per execution thread.
> 

Re: [Pharo-dev] [ANN] OSSubprocess first milestone

[David T. Lewis]

On Thu, Jan 14, 2016 at 03:33:56PM -0300, Mariano Martinez Peck wrote:
> Dear all,
> 
> I am happy to tell you I have a first prototype of the tool for executing
> OS processes:
> 
> https://github.com/marianopeck/OSSubprocess
> 
>  As I said many times, the main idea was to be as much as possible
> FFI-based, only for executing OS commands and, from my point of view, a
> better end-user API. Also, as for the first step, I had to focus on
> Linux/OSX.
> 
> I would like to thank Pharo Consortium for the sponsor of this development
> effort. Here is the first milestone!

Bravo Mariano!

Dave

Re: [Pharo-dev] UUID Plugin ?

[David T. Lewis]

On Wed, Feb 10, 2016 at 02:46:23PM +0100, Guille Polito wrote:
> Yeap, linux VMs are not shipped with the plugin since a couple of years. 
> Then, the code in the image is needed.

For background - the UUID plugin is trivially simple, but there were
issues on certain Linux distributions that caused it to be a big pain
for packaging and distribution. Nowadays the in-image UUID is so fast
that the plugin is no longer worth the aggrivation.

Dave
 

Re: [Pharo-dev] wrong method trailer kind

[David T. Lewis]

On Sat, Feb 27, 2016 at 12:32:06PM +0100, Nicolai Hess wrote:
> 2016-02-27 0:36 GMT+01:00 Eliot Miranda :
> 
> >
> > Round about 2009 Igor Stasenko did a really nice reimplementation of this
> > scheme where he factored out the encoding tricks into
> > CompiledMethodTrailer, allowing one to generalize the scheme and add new
> > encoding types, for a total of 256 different kinds of trailer.
> >
> 
> Yes I do remember his work on this.
>

Igor's enchancement also enabled us to eliminate the size restriction
on the changes file.

Dave
 

Re: [Pharo-dev] Random is not random at startup

[David T. Lewis]

On Thu, May 19, 2016 at 07:57:37AM +0800, Ben Coman wrote:
> On Thu, May 19, 2016 at 7:05 AM, Peter Uhn??k  wrote:
> >
> > Not so random??? not random at all.
> 
> Obligitory cartoon...
> https://xkcd.com/221/
> 

Perfect :-)

Re: [Pharo-dev] Screenshot of Pharo 64 bits

[David T. Lewis]

On Wed, May 25, 2016 at 11:27:04AM +0200, Cl??ment Bera wrote:
> There are quite some progress on the plugin work.
> 
> Eliot made x64 FFI working a couple months ago.
> 
> Nicolas Cellier have already been looking for Number plugins for a while.
> 
> Many Slang compilation bugs related to 32-64 bits dependency were fixed, so
> many plugins work out of the box.

The commonly required plugins have been working for quite a few years on
all combinations of 32/64 bit VM and image. I'm sure there will still be
some issues to address as we move to Spur 64, but I expect that most plugins
will work.

Dave


> 
> Other plugins w
> 
> On Wed, May 25, 2016 at 11:02 AM, p...@highoctane.be 
> wrote:
> 
> > And this marked the start of a new era.
> >
> > Kudos guys.
> >
> > Now, plugin time I guess...
> >
> > Phil
> >
> > On Tue, May 24, 2016 at 3:25 PM, Cl??ment Bera 
> > wrote:
> >
> >> Today I helped Esteban porting the 32 to 64 bits bootstrap to Pharo. It
> >> looks like we have a Pharo 64 bits image on 64 bits VM running on Mac &
> >> Linux.
> >>
> >> Here is the wonderful screenshot showing the amazing result:
> >>
> >> [image: Inline image 1]
> >>
> >
> >

Re: [Pharo-dev] AioPlugin not present

[David T. Lewis]

On Sun, Jan 22, 2017 at 05:00:04PM +0100, Thierry Goubier wrote:
> 
> Le 22/01/2017 ? 15:50, Davide Varvello via Pharo-dev a ?crit :
> >Hi Thierry,
> >
> >I need it for the OSProcess. When my code calls OSProcess the warning
> >(http://imgur.com/a/9jhMy) appears.
> >
> >It happens only once. If I close the warning window It does not happen
> >again, but of course if I restart the image it appears once again, so I 
> >have
> >to find a way to include it or another workaround
> >
> >Cheers
> >Davide
>
> Hi Davide,
> 
> I do use OSProcess in some cases; last time, I just rewrote that part to 
> emit a notification (and not raise a popup).
> 
> Regards,
> 
> Thierry

I am not sure why the AIOPlugin is not being included in the VMs.

Whatever the reason for that, the notifier about the missing plugin was
too annoying, so I got rid of it in the latest OSProcess update:

  Name: OSProcess-Base-dtl.56
  Time: 17 January 2017, 8:47:36.321587 am
  
  OSProcess 4.6.7
  
  Add nil checks in #shutDown: to prevent failures when exiting existing
  image with a VM that does not include OSPP (e.g. SqueakJS).
  
  For AIO event hander, if the AIO plugin is not present, write an error
  to console rather than interrupting the image with a notifer. Reduces the
  annoyance level if VM provider provides OSPP but neglects to include the
  AIO plugin.

Dave
 

Re: [Pharo-dev] UnixOSProcessPlugin and accessing C array out of bounds

[David T. Lewis]

Holger,

Good catch, thank you. There were two arrays dimensioned wrong in 
OSProcessPlugin in
the Smalltalk slang source. I fixed this, and the warning should be gone after 
the
sources are next generated.

Esteban,

I fixed the oscog branch also. I could not easily test it, but it is a small 
change
so hopefully should be no problem when you next generate the plugin.

Dave


On Wed, Feb 22, 2017 at 04:25:05PM +0700, Holger Freyther wrote:
> Hi Esteban,
> 
> while building packages for OBS and going the compile warnings (one of the 
> nice things of newer compilers is the increased diagnostic) I noticed this:
> 
> [  196s] 
> /usr/src/packages/BUILD/src/plugins/UnixOSProcessPlugin/UnixOSProcessPlugin.c:4525:20:
>  warning: iteration 64u invokes undefined behavior 
> [-Waggressive-loop-optimizations]
> [  196s] if ((semaIndices[sigNum]) > 0) {
> [  196s] ^
> [  196s] 
> /usr/src/packages/BUILD/src/plugins/UnixOSProcessPlugin/UnixOSProcessPlugin.c:4524:9:
>  note: containing loop
> [  196s]while (sigNum <= (signalArraySize())) {
> 
> while (sigNum <= (signalArraySize())) {
> if ((semaIndices[sigNum]) > 0) {
> setSignalNumberhandler(sigNum, 
> (originalSignalHandlers())[sigNum]);
> }
> sigNum += 1;
> }
> 
> semaIndices has signalArraySize()/NSIG number of entries and is zero based, 
> it is being accessed with sigNum==NSIG which means one entry beyond the  
> memory of the array? Can you confirm this?
> 
> restoreDefaultSignalHandlers
> "Restore signal handlers to their original behaviors."
> 
> | sigNum |
> 
> semaIndices = nil "nil if in interpreter simulation"
> ifFalse: [sigNum := 1.
> [sigNum <= self signalArraySize] whileTrue:
> [((semaIndices at: sigNum) > 0) ifTrue:
> [self setSignalNumber: sigNum 
> handler: (self originalSignalHandlers at: sigNum).].
> sigNum := sigNum + 1]]
> 
> So it is one based but the array doesn't have an extra element to make it 
> work? How is this normally handled in slang code?
> 
> 
> holger

Re: [Pharo-dev] Running on Ubuntu?

[David T. Lewis]

On Tue, Nov 22, 2016 at 12:19:30PM +0200, Igor Stasenko wrote:
> ldd vm-display-X11
> linux-gate.so.1 =>  (0xf77fd000)
> libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf7799000)
> libGL.so.1 => not found
> libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf7664000)
> libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf74b5000)
> /lib/ld-linux.so.2 (0x5664e000)
> libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf7493000)
> libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf748e000)
> libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf748a000)
> libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf7482000)
> 
> 
> seems like i have everyting already except libgl...
> any ideas what package providing it?
> 

After some trial and error, this is what I ended up installing on my Ubuntu 
system:

  sudo apt-get install libgl1-mesa-dri-lts-utopic:i386
  sudo apt-get install libgl1-mesa-glx-lts-utopic:i386
  sudo apt-get install mesa-common-dev

Dave

Re: [Pharo-dev] ZnInvalidUTF8 on response from squeaksource

[David T. Lewis]

squeaksource.com is still running on a quite old image, and I know that it
has problems with multibyte characters. If you are seeing problems related
to this, it's not the fault of Zinc.

If you can confirm that this is what is happening, then I guess it is time
to update that trusty old squeaksource.com image :-)

Dave

> On Wed, Mar 15, 2017 at 8:19 PM, Patrick R.  wrote:
>>
>> Hi everyone,
>>
>> I have been working on bringing http://squeaksource.com/ical/ up to
>> speed
>> for Squeak and wanted to make sure that it also works for Pharo.
> Therefore,
>> I have created a travis build job for Squeak and Pharo
>> (https://travis-ci.org/codeZeilen/ical-smalltalk/jobs/211298950) which
> pulls
>> the source from squeaksource.com.
>>
>> Now the issue is that loading the package in Pharo fails with a
>> GoferException wrapping a ZnInvalidUTF8 Exception. We figured that this
>> might be the result of the squeaksource page delivering the page as
>> iso-8859-1 as it contains special characters. Any ideas on how to get
>> this
>> to work? I do not have access to the ical repository description and I
> would
>> like to avoid mirroring the whole repository on GitHub.
>
>
> In a fresh 60437 image, in Playground evaluating...
>
>   Metacello new
>configuration: 'ICal';
>repository: 'github://codeZeilen/ical-smalltalk:master/repository';
>onConflict: [:ex | ex allow];
>load.
>   ==> Could not resolve: ICal-Core [ICal-Core-PaulDeBruicker.5] in
> /home/ben/.local/share/Pharo/images/60437-01/pharo-local/package-cache
> http://squeaksource.com/ical ERROR: 'GoferRepositoryError: Could not
> access
> http://squeaksource.com/ical: ZnInvalidUTF8: Illegal continuation byte for
> utf-8 encoding'
>
>
> In a new fresh 60437 Image (i.e. empty package-cache)
>   World menu > Monticello > +Repository > squeaksource.com...
>  MCSqueaksourceRepository
> location: 'http://squeaksource.com/ical'
> user: ''
> password: ''
>==> open repository then errors "MCRepositoryError: Could not access
> http://squeaksource.com/ical: ZnInvalidUTF8: Illegal continuation byte for
> utf-8 encoding"
>
>
> In Chrome, opening http://www.squeaksource.com/ical
> then clicking 
> and the browser's View Page Source,
> I see...
>
>
> Googling: zinc iso-8859-1
> finds...
> http://forum.world.st/Problem-using-Zinc-in-Pharo-4-Moose-5-1-td4825329.html
> but "ZnByteEncoder iso88591"
> errors with "KeyNotFound: key 'iso88591' not found in Dictionary"
> and inspecting "ZnByteEncoder byteTextConverters keys sorted"
> confirms this key is missing (@Sven, I'm curious why was this removed? )
>
>
> Now https://en.wikipedia.org/wiki/ISO/IEC_8859-1
> indicates IBM819 is an alias
> and " ZnByteEncoder newForEncoding: 'ibm819' "
> works okay
>
> So in MCHttpRepository>>#loadAllFileNames
> changing...
>  queryAt: 'C' put: 'M;O=D' ;
>  get.
> to...
>  queryAt: 'C' put: 'M;O=D' .
>  ZnDefaultCharacterEncoder
>   value: (ZnByteEncoder newForEncoding: 'ibm819')
>   during: [client get].
>
> Then from Monticello opening the previously defined
> http://squeaksource.com/ical
> works!!
>
>
> Now I was hoping that reverting #loadAllFileNames
> and in Playground doing...
> converters := ZnByteEncoder byteTextConverters.
> converters at: 'iso-8859-1' put: (converters at: 'ibm819').
> might alleviate the problem, but no luck.
>
>
> Anyone know a better way to deal with this that hardcoding the encoding
> into #loadAllFileNames?
>
> cheers -ben
>

[Pharo-dev] File I/O performance (Re: [VM] Windows VM SurfacePlugin.dll LoadLibrary failure)

[David T. Lewis]

On Sat, Apr 01, 2017 at 10:06:34AM +0200, Nicolas Cellier wrote:
> AFAIR, flushing after each write was a workaround against some linux
> problem.

This was specifically for the case of writing a chunk to e.g. the changes
file. The extra flush for that case is protecting for two processes writing
to the same stream. This would not affect overall read/write performance
though.

> I think that flush is not exagerately expensive on linux, because there's a
> difference between flush (transfer data from user space buffers to kernel
> space buffers) and sync (transfer data to the disc).

The Windows VM uses a different FilePlugin. The I/O calls are made at the
HANDLE level, not at the (FILE *) level as in the Unix plugin. A HANDLE on
Windows would be loosely equivalent to a file descriptor number on Unix.

If #primitiveFileFlush is intended to mean "flush everything from the VM out
to the operating system", and #primitiveFileSync is intended to mean "tell the
operating system to flush it all way to disk", then flush would be a no-op
on Windows, and you would call the sync primitive only in cases where you
really need to guarantee a flush to disk.

An interesting project would be to adapt the Unix plugin to run on Windows
(maybe it already just works, I never tried), or conversely to write a Unix
plugin that reads and writes to the file descriptor level as in done with
the Windows plugin.

My guess is that both approaches are valid, and they probably will have
differing performance advantages based on the I/O workload. But someone
would have to try it to find out :-)

Dave


> 
> In Win32 VM, the fileFlush and fileSync primitives do the same call to
> FlushFileBuffers.
> The msdn tells it's a bad thing to repeatedly call FlushFileBuffers
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa364439(v=vs.85).aspx
> I don't see such thing as a "cheap" flush on windows.
> 
> But do we really need to flush?
> Couldn't we rather implement a linuxWorkaroundFlush: aFile method that
> would flush on linux and do nothing anywhere else?
> 
> 2017-04-01 6:40 GMT+02:00 Martin Dias :
> 
> > Hi all. I rarely use Windows but this night entered again to repeat the
> > load test in Pharo.
> >
> > Cyril: I uninstalled the antivirus and excluded .changes and .images files
> > in the "Windows Defender" (didn't know it exists). I may note some speed up
> > but not much, so it wasn't THE solution.
> >
> > Peter: I didn't check firewall but "Compiling methods..." is what stays on
> > screen so it shouldn't be the network.
> >
> > Eliot: this time I profiled it and can highlight from report that Leaves
> > shows 91% in StandardFileStream>>primFlush:. I guess it isn't too
> > surprising since file writing is expensive but it's a lot. However I didn't
> > compare it yet with the respective report in linux. For the record, I
> > attach the report.
> >
> > Code evaluated:
> > AndreasSystemProfiler spyOn: [
> > Gofer new
> > url: 'http://www.smalltalkhub.com/mc/ObjectProfile/Roassal2/main';
> > package: 'ConfigurationOfRoassal2';
> > load.
> > #ConfigurationOfRoassal2 asClass load.]
> >
> > Regards
> > Mart??n
> >
> >
> > On Thu, Mar 16, 2017 at 8:03 PM, Cyril Ferlicot D. <
> > cyril.ferli...@gmail.com> wrote:
> >
> >> Le 16/03/2017 ?? 05:11, Martin Dias a ??crit :
> >> > And it takes >30 minutes to load Roassal2, while takes <1 minyte in
> >> > Linux (same notebook). Script:
> >> >
> >>
> >> Do you have an anti-virus? If yes did you excluded the .changes and ombu
> >> files from the files to check? If not the anti-virus might analyze your
> >> file at each compilation.
> >>
> >> --
> >> Cyril Ferlicot
> >>
> >> http://www.synectique.eu
> >>
> >> 2 rue Jacques Pr??vert 01,
> >> 59650 Villeneuve d'ascq France
> >>
> >>
> >

Re: [Pharo-dev] 60437 empty catalog

[David T. Lewis]

On Sat, Mar 11, 2017 at 10:14:58PM +0800, Ben Coman wrote:
> Anyone else getting an empty Catalog?
> 
> In 60437 &  CoInterpreter * VMMaker.oscog-eem.2143 uuid:
> fe064b6b-e530-4766-837d-799ffe1e8dcd Mar  3 2017
> (i386itimer-201703031554.zip)
> 
> Sorry I'm too tired to bisect it myself right now.
> (Just spent 10 hours sanding the wooden floors of my brothers house)
> 

It looks ok to me. Try deleting the file from your local package cache,
sometime an error condition can leave junk there.

Dave

[Pharo-dev] Software maintenance costs in the real world (Re: ftp repositories broken)

[David T. Lewis]

For students of software engineering, I offer this as an example of one of the
most common and costly types of error that you will encounter in real life.

A small error may be introduced early in the development process, possibly
slipping through the cracks through lack of review (as was the case in this
example). The change is motivated by good intentions, but some aspect of the
overall system design is overlooked.

Once the mistake has been made, it becomes increasingly difficult to change
as time goes by, and increasingly costly to support as new people need to
understand the issue and work around its consequences. Over the life of the
software, the accumulated costs of effort to understand and address the
problem (typically budgeted separately under "software maintenance") far
exceed the costs associated with the original change (typically budgeted
as "development").

In addition to cost of maintenance, this kind of error can also defeat the
intentions of the original developer. In the example we have here, the
original change was motivated by an interest in making the system cleaner
and easier to understand. But nearly a decade later, we see workarounds
such as the #translatedPrimitives method below that are far worse than the
original problem that someone was trying to solve.

For the historical background on this particular exammple, see the earlier
post by Henrik Sperre Johansen in this thread.

Dave

p.s. I am not a software engineer :-)


On Tue, Apr 04, 2017 at 06:45:27PM +0200, Esteban Lorenzano wrote:
> 
> > On 4 Apr 2017, at 18:01, Ben Coman  wrote:
> > 
> > 
> > 
> > On Tue, Apr 4, 2017 at 10:12 PM, Denis Kudriashov  > > wrote:
> > >
> > >
> > > 2017-04-04 15:47 GMT+02:00 Esteban Lorenzano  > > >:
> > >>
> > >> no it doesn't: squeak does not have that primitive, that???s why it does 
> > >> not fails :)
> > >
> > >
> > > I think you are wrong because I check it carefully in Squeak and I found 
> > > it on github 
> > > https://github.com/OpenSmalltalk/opensmalltalk-vm/search?utf8=???=primitiveFindSubstring=
> > >  
> > > .
> 
> still is not there, because is not the same primitive :)
> look, this is the way where it is generated: 
> 
> translatedPrimitives
>   "an assorted list of various primitives"
>   (ByteString compiledMethodAt: 
> #findSubstringViaPrimitive:in:startingAt:matchTable: ifAbsent: []) ifNotNil:
>   [^#("Pharo uses 
> findSubstringViaPrimitive:in:startingAt:matchTable:"
>   (Bitmap compress:toByteArray:)
>   (Bitmap decompress:fromByteArray:at:)
>   (Bitmap encodeBytesOf:in:at:)
>   (Bitmap encodeInt:in:at:)
>   (ByteString compare:with:collated:)
>   (ByteString translate:from:to:table:)   
>   (ByteString findFirstInString:inSet:startingAt:)
>   (ByteString indexOfAscii:inString:startingAt:)
>   (String 
> findSubstringViaPrimitive:in:startingAt:matchTable:)
>   (ByteArray hashBytes:startingWith:)
>   (SampledSound convert8bitSignedFrom:to16Bit:))].
>   ^#(
>   (Bitmap compress:toByteArray:)
>   (Bitmap decompress:fromByteArray:at:)
>   (Bitmap encodeBytesOf:in:at:)
>   (Bitmap encodeInt:in:at:)
>   (ByteString compare:with:collated:)
>   (ByteString translate:from:to:table:)   
>   (ByteString findFirstInString:inSet:startingAt:)
>   (ByteString indexOfAscii:inString:startingAt:)
>   (ByteString findSubstring:in:startingAt:matchTable:)
>   (ByteArray hashBytes:startingWith:)
>   (SampledSound convert8bitSignedFrom:to16Bit:)
>   )
> 
> means that Pharo uses #indSubstringViaPrimitive:in:startingAt:matchTable: 
> while squeak uses #findSubstring:in:startingAt:matchTable:. 
> Now, there is a bug there (because source is in String and not in 
> ByteString), and I can fix that by asking correctly, but: 
> 
> 1) this means that sources for pharo and squeak will be different and pharo 
> needs to be generated in pharo. This is contrary to what we are trying to do 
> with the VM: we want all sources to be the same, and differences should come 
> from compilation flags and/or the presence of certain plugins (this is 
> because is a lot easier to debug the VM like that). 
> 2) I wonder??? why we are using this primitive an squeak is not? maybe we 
> need to deprecate the use?
> 
> > I believe Esteban's point Its not so much whether the VM has it 
> > (since they are essentially the same VM + extra C-libs for Pharo)
> > but whether the Squeak Image is using it.
> > 
> > Squeak does use 

Re: [Pharo-dev] P6 crashed image

[David T. Lewis]

Hi Hilaire,

I do not know if it will help in this case, but sometimes I have recovered from
similar problems by temporarily deleting a problem plugin (probably FT2Plugin.so
in this case). Sometimes that will allow you to open the image and save it 
again.
After that, you may be able to put the plugin back and run the image again.

Dave


On Sun, Jul 02, 2017 at 05:56:05PM +0200, Hilaire wrote:
> Hi,
> 
> I got problem with my Pharo6 image when I save it, after some Glamour
> try out, don't know if it is related.
> 
> When I saved the image, it grows to 292MB then stalled, with the mouse
> writing pointer. Then Pharo display became garbage. I have to kill the
> process.
> 
> The image does not start anymore, get segmentation fault with Smalltalk
> stack dump, Freetype seems to be involved:
> 
> Segmentation fault Sun Jul  2 17:52:39 2017
> 
> 
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/lib/pharo/5.0-201705310241/pharo
> Pharo VM version: 5.0-201705310241  Wed May 31 04:43:29 UTC 2017 gcc 4.6.3 
> [Production Spur 64-bit VM]
> Built from: CoInterpreter VMMaker.oscog-eem.2231 uuid: 
> de62947a-7f40-4977-a232-e06a3a80c939 May 31 2017
> With: StackToRegisterMappingCogit VMMaker.oscog-eem.2227 uuid: 
> 7ea146b4-39ce-4de7-afa3-a76ed1d1da35 May 31 2017
> Revision: VM: 201705310241 
> https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Tue May 30 
> 19:41:27 2017 -0700 $
> Plugins: 201705310241 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
> Build host: Linux testing-gce-7baaea33-cad4-44b6-b0e5-7ad5f93c3335 
> 3.13.0-115-generic #162~precise1-Ubuntu SMP Fri Mar 24 16:47:06 UTC 2017 
> x86_64 x86_64 x86_64 GNU/Linux
> plugin path: 
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241
>  [default: 
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/lib/pharo/5.0-201705310241/]
> 
> 
> C stack backtrace & registers:
> rax 0x49063940 rbx 0x490637d0 rcx 0x490639f8 rdx 0x49063888
> rdi 0x490635a8 rsi 0x490635a8 rbp 0x49063718 rsp 0x49063ab0
> r8  0x49062fe8 r9  0x490630a0 r10 0x49063158 r11 0x49063210
> r12 0x490632c8 r13 0x49063380 r14 0x49063438 r15 0x490634f0
> rip 0x49063b68
> *[0x7fff49063b68]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo[0x41cac1]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo[0x41ce4f]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7f4e70d53330]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(FT_Stream_ReadULong+0x78)[0x7f4e6cc9c168]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(+0x52116)[0x7f4e6ccdb116]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(+0x26742)[0x7f4e6ccaf742]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(+0x152e3)[0x7f4e6cc9e2e3]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(FT_Open_Face+0x2a0)[0x7f4e6cca0070]
> /usr/lib/x86_64-linux-gnu/libfreetype.so.6(FT_New_Memory_Face+0x39)[0x7f4e6cca0669]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/FT2Plugin.so(primitiveNewMemoryFaceFromExternalMemoryAndIndex+0xea)[0x7f4e6cf309aa]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo[0x4572a7]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo(interpret+0xbd80)[0x467d80]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo[0x468651]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo(interpret+0x268)[0x45c268]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo(main+0x2ab)[0x419d6b]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f4e70695f45]
> /home/hilaire/Travaux/Developpement/Pharo/vm/pharo64-linux-threaded-stable/bin/../lib/pharo/5.0-201705310241/pharo[0x41a055]
> [0x7fff49061d60]
> 
> 
> Smalltalk stack dump:
> 0x7fff490695d8 I FreeTypeFace(FT2Face)>newFaceFromExternalMemory:index: 
> 0x4a83478: a(n) FreeTypeFace
> 0x7fff49069628 I FreeTypeFace>create 0x4a83478: a(n) FreeTypeFace
> 0x7fff49069668 I FreeTypeFace>validate 0x4a83478: a(n) FreeTypeFace
> 0x7fff490696a8 I FreeTypeFont>face 0x13c41758: a(n) FreeTypeFont
> 0x7fff490696e8 I FreeTypeFont>validate 0x13c41758: a(n) FreeTypeFont
> 0x7fff49069728 I LogicalFont>realFont 0x3015870: a(n) LogicalFont
> 0x7fff49069768 I LogicalFont>descent 0x3015870: a(n) LogicalFont
> 
> 
> -- 
> Dr. Geo
> http://drgeo.eu
> 

Re: [Pharo-dev] Maybe better pinning messages? (inspired by [Pinning Objects in Pharo])

[David T. Lewis]

On Thu, Sep 14, 2017 at 12:18:34AM +0200, Norbert Hartl wrote:
> 
> > Am 14.09.2017 um 00:09 schrieb Eliot Miranda :
> > 
> > Hi Denis,
> > 
> > On Tue, Sep 12, 2017 at 9:29 AM, Denis Kudriashov  > > wrote:
> > Hi Eliot. 
> > 
> > I know and I only talk about new messages. I am not trying to rethink full 
> > meta model of Smalltalk.
> > By the way #class is very common message and it is handy to use short name. 
> > But pinning messages will be used rarely in very specific applications. So 
> > no much sense to preserve them in short version.  
> > 
> > Agreed.  So we have to decide whether to go with pinInMemory or pinObject, 
> > pinObject being suggested by Norbert because it matched isReadOnlyObject.  
> > Personally I like pinInMemory.  Norbert, do you feel strongly about 
> > pinObject et al?
> >  
> No I don't. It feels only good to me if there is a requirement not to 
> implement selectors that are likely to be used in user code. I'm ok with 
> pinInMemory although I asked myself where can it be pinned elsewhere if not 
> in memory. So the suffix in memory doesn't add anything but also moves the 
> selector out of user space. 
> 

+1

I also like pinInMemory because it is easy to read and it describes
exactly what the method does. It pins the object to some location in the
object memory. Once you stick a pin in it, the object stops moving around
in the object memory. Nice.

Dave

Re: [Pharo-dev] [ci-announces] [CI] July 28, 2017 Incident Report

[David T. Lewis]

Kudos for providing a good incident report and explanation of the cause of the 
problem.

I know little about this actual incident, but that it not important. It is 
really good
to see constructive responses to problems like this with a focus on inprovement 
and
problem prevention in the future.

Dave

On Thu, Sep 21, 2017 at 03:35:19PM +0200, Christophe Demarey wrote:
> July 28, 2017 Incident Report
> 
> This summer, we experienced a hard incident on the Continuous Integration 
> service infrastructure.Today we???re providing an incident report that 
> details the nature of the incident and our response.
> We understand this service issue has impacted all Inria developers using the 
> CI service and their valued time, and we apologize to everyone who was 
> affected.
> 
> Issue summary
> Due to a manipulation error, a lot of Virtual Machines (VM) hosted on 
> CloudStack were destroyed and not recoverable!
> Jenkins servers are not concerned by this issue, so all Jenkins jobs and 
> history are safe. 
> 
> Root cause
> After the successful migration of Jenkins servers and security fixes (July 
> 18-19, 2017), a few projects were not able to reach their slaves hosted on 
> the CI build farm. This problem was due to a synchronization problem between 
> the CI database and CloudStack (powering the CI build farm) having its own 
> way to manage projects and users (through domains).
> An attempt to reproduce and debug this problem on the qualification 
> infrastructure failed. So, we added some logging on the production 
> infrastructure. To avoid troubles to the production infrastructure, we 
> limited the synchronization to one project. It was the mistake!
> The synchronization of one project led to the deletion of all other 
> CloudStack domains (i.e. projects). Indeed, the synchronization code expected 
> to get the full environment (all CI projects) and if it finds a domain not 
> bound to a CI project, it deletes it...
> The synchronization process was aborted before its termination but it was too 
> late. Some user Virtual Machines were still alive during some hours but were 
> finally purged by CloudStack. 
> It means we lost most VM and templates hosted on the CI build farm.
> 
> Resolution and recovery
> It was impossible to recover destroyed virtual machines. CloudStack is 
> configured to keep VM data 24 hours before actually destroying it but it does 
> not work when the domain hosting the VM is destroyed.
> Primary storage hosting running VM is a high-performance and very expensive 
> storage. That???s why the CI team chose (at the CI service setup) to do not 
> backup VM but rather to rely on both the expunge delay and the snapshot / 
> template mechanism to save VM state. This mechanism was useless in relation 
> to this incident.
> Templates and snapshots are hosted on the secondary storage that is a 
> redundant storage in two different buildings to ensure data reliability and 
> recovery. The incident led CloudStack to perform a ?? clean ?? deletion of 
> all the domain data including templates. That???s why they also became 
> unavailable.
> 
> We were able to rebuild all domains from the CI database but CI service users 
> had to create new VM to replace the destroyed ones.
> 
> Corrective and preventative measures
> All members of the CI team (DSI, SED) worked and are still working all 
> together to find the best solution to mitigate the incident and prevent same 
> situations in the future.
> The synchronization code responsible of the deletion of CloudStack domain has 
> been deactivated. CloudStack domain deletion is a critical action and will no 
> longer be automated. Deletions will be reviewed and approved by the CI team 
> before being completed.
> This incident showed us that backup mechanism in place are not strong enough 
> and we are now evaluating the cost to backup, with history:
> all VM, snapshots and templates or
> all snapshots and templates.
> We are also working on providing a way to download templates created on 
> CloudStack so that you can easily get a copy of them. We encourage you to 
> create templates for virtual machines that are time consuming to set up from 
> scratch.
> 
> 
> Sincerely,
> The CI Team

Re: [Pharo-dev] Problem with primitive instVarAt: / instVarAt:put:, what should we do ?

[David T. Lewis]

On Mon, Nov 27, 2017 at 12:11:13PM -0800, Eliot Miranda wrote:
> Hi Cl??ment,
> 
> On Wed, Nov 22, 2017 at 5:35 AM, Cl??ment Bera 
> wrote:
> 
> > Hi All,
> >
> > It seems Pharo is lacking the slotAt: / slotAt:put: primitive and that the
> > instVarAt: / instVarAt:put: primitive is not doing what it says it does.
> > When I look at Object>>#instVarAt: I see:
> >
> > "[...]Answer a fixed variable in an object. The numbering of the variables
> > corresponds to the named instance variables, followed by the indexed
> > instance
> > variables. Fail if the index is not an Integer or is not the index of a
> > fixed variable[...]"
> >
> > Now in Pharo 7 when one does:
> >
> > #(iv1) instVarAt: 1, one gets #iv1.
> >
> 
> But the failure code should indeed answer #iv1.
> 
> 
> > So the comment is wrong.
> >
> > The reason for this is that instVarAt: / instVarAt:put: should be
> > primitive 73/74, while they are in Pharo 173/174 (primitive
> > slotAt:/slotAt:put:).
> >
> 
> Why do you say "should"?  The 173/174 primitives work well and avoid
> unnecessary primitive failures (which is why they're there).
> 
> 
> > The slot primitive access pointer slots of objects, whichever inst var or
> > variable fields. A good example would be:
> >
> > | t |
> > t := #temp.
> > [ t ] instVarAt: 4, one gets #temp
> >
> > So I have two solutions for this problem:
> > 1) We change the instVarAt: / instVarAt:put: primitives comments to match
> > the slotAt: / slotAt:put: behavior, and in Pharo instVarAt: /
> > instVarAt:put: means in factslotAt: / slotAt:put: .
> >
> 
> This is the performant solution, because...
> 
> 
> > 2) We change the instVarAt: / instVarAt:put: primitives to primitive 73
> > and 74, we introduce slotAt: / slotAt:put: with primitive 173/174 and we
> > ask everyone to fix their code relying on instVarAt: / instVarAt:put:
> > acting as slotAt: / slotAt:put:
> >
> 
> Early in Squeak Spur testing Chris Muller noticed a slow down in his code
> He was accessing an Array using instVarAt:, which was failing because he
> was using it to access an indexed field in an Array.  Now in Spur the VM
> must scan the object graph of a primitive's  receiver and arguments to some
> depth on primitive failure to follow forwarders.  Because a) at that time
> the VM computed a depth > 0 for instVarAt:[put:] it scanned the whole array
> on each access, and b) the Array was large, this meant that the failures
> were very expensive.  So I added the slot primitives, since the old 73/74
> primitive pair were only used in the instVarAt:[put:] methods that accessed
> indexable slots on failure.  So the new 173/174 pair subsumed the 73/74
> pair.
> 
> I eventually added an accessorDepth: pragma rot specify the accessor depth
> manually when the automatically computed depth is too deep.  It is for
> at:[put:] because the String version of the at: 60 primitive appears to
> access the innards of non-immediate Characters (it doesn't do this in Spur,
> but the depth inferencer isn't clever enough to notice).  So primitiveAt
> looks like:
> 
> primitiveAt
> 
> self commonAt: false
> 
> So we could go back to using 73/74 for instVarAt:[put:] provided that their
> accessorDepth ends up being 0, which likely means they will require the
> pragma.  But I think the simplest thing is to keep using 173/174.  Here's
> the Squeak code:
> 
> Object>>instVarAt: index
> "Primitive. Answer a fixed variable in an object. The numbering of the
> variables
> corresponds to the named instance variables, followed by the indexed
> instance
> variables. Fail if the index is not an Integer or is not the index of a
> fixed or
> indexed variable. Essential. See Object documentation whatIsAPrimitive."
> 
> 
> self primitiveFailed
> 
> Object>>instVarAt: index put: anObject
> "Primitive. Store a value into a fixed variable in an object. The numbering
> of the
> variables corresponds to the named instance variables, followed by the
> indexed
> instance variables. Fail if the index is not an Integer or is not the index
> of a fixed
> or indexed variable. Essential. See Object documentation whatIsAPrimitive."
> 
> 
> self primitiveFailed
> 
> I've attached these methods.
> 
> 
> What do you think ?
> >
> 
> I think for Spur it makes sense to use 173/174.
> 

+1

The 73/74 primitives (primitiveInstVarAt and primitiveInstVarAtPut) are 
semantically
different from the 173/174 primitives (primitiveSlotAt and primitiveSlotAtPut).

The 173/174 primitives are preferred for Spur images, and the 73/74 primitives 
are
used for V3 images. The VM can support both, and the image should call the 
primitive
that it needs or wants to use. In any case, the primitive table numbering 
should not
be confused between them.

Squeak has the correct implementation for calling the primitives from a Spur 
image.
See Cuis has an implementation of the image calling the appropriate primitive 
for
whatever image format it happens to be using.

Dave

Re: [Pharo-dev] [ Umbrella Issue ] Cleanup remaining DeprecatedFileSystem users

[David T. Lewis]

Hi Alistair,

On Fri, May 11, 2018 at 03:09:20PM +0200, Alistair Grant wrote:
> Hi Dave,
> 
> On 11 May 2018 at 03:58, David T. Lewis <le...@mail.msen.com> wrote:
> > On Wed, May 09, 2018 at 01:41:47PM +0200, Thierry Goubier wrote:
> >> 2018-05-09 13:14 GMT+02:00 Sven Van Caekenberghe <s...@stfx.eu>:
> >> > You mean you made a subclass of StandardFilestream yourself ?
> >>
> >> There are a few subclasses of StandardFileStream in OSProcess:
> >> AttachableFileStream, AsyncFileReadStream and
> >> BufferedAsyncFileReadStream. They will need to be ported.
> >>
> >> > If yes, can you describe briefly why you did so ?
> >>
> >> As far as my analysis go, AttachableFileStream refers to and
> >> manipulates the underlying ioHandle of a unix pipe stream,
> >> AsyncFileReadStream interacts with the asynchronous io extensions of
> >> the vm to observe that io handle, and a BufferedAsyncFileReadStream
> >> add a thread-safe, buffered, blocking API for smalltalk processe on
> >> top.
> >
> > That's right.
> >
> > I'm sorry I have not been keeping up with this discussion, but one thought
> > that comes to mind - the AttachableFileStream hierarchy is something that
> > would better be replaced by some kind of delegation. It was a hack that I
> > did a long time ago to allow OSProcess to work without modifying the base
> > image. But since then, we have added multibyte characters and streams, and
> > we have different file and stream models in Squeak/Pharo/Cuis (Cuis is an
> > important target platform from my POV).
> >
> > I don't have a solution in mind, my sense is that if we find a way to make
> > AttachableFileStream go away, then that solution would very likely make
> > it unnecessary to develop and maintain separate Squeak/Pharo/Cuis forks
> > moving forward.
> >
> > Note that the only fundamental thing that AttachableFileStream does is this:
> >
> > AttachableFileStream class>>name: aSymbolOrString attachTo: anIOHandle 
> > writable: readWriteFlag
> > "Create a new instance attached to anIOHandle, where anIOHandle
> > represents an open IO channel. For write streams, this represents 
> > two
> > Smalltalk streams which write to the same OS file or output stream,
> > presumably with interleaved output. The purpose of this method is to
> > permit a FileStream to be attached to an existing IOHandle, such as
> > the IOHandle for standard input, standard output, and standard 
> > error."
> >
> > ^ (super basicNew
> > name: aSymbolOrString
> > attachTo: anIOHandle
> > writable: readWriteFlag) initialize
> >
> >
> > The method comment refers to "IOHandle" which was another old project I was
> > doing, but IOHandle basically means the #fileID field of a file stream.
> >
> > If it was just AttachableFileStream that needed to be refactored out of 
> > existence,
> > it would be easy. But I later added quite a bit a behavior to 
> > BufferedAsyncFileReadStream
> > and I'm not sure what might be needed to make this work properly.
> >
> > Sven, I suspect that you are the most knowledgeable in this area, so any
> > suggestions would be welcome.
> 
> Are the two primitives that were added to the VM a couple of months
> ago (primitiveConnectToFileDescriptor & primitiveConnectToFile) enough
> to allow AttachableFileStream to be removed?

No, I was referring only to the actual file stream hierarchy. Long ago,
I tapped into StandardFileStream with some subclasses that do useful things
for OSProcess. That is clearly a hack, and I'm sure we could do better.
Having said that, I have not looked at Pharo 7 yet so maybe I don't know
what I'm talking about ;-)


> 
> Also, borrowing primitiveSQFileSetNonBlocking from OSProcess I was
> able to open a named pipe and read from it in a non-blocking way using
> the standard Pharo 7 file stream (BinaryFileStream).  I'd like to look
> at getting AsyncFile working with it, but that's a long way down the
> ToDo list.

Good, primitiveSQFileSetNonBlocking certainly should work for that, and
I'm glad to hear that it does.

Dave

> 
> HTH,
> Alistair

Re: [Pharo-dev] [ Umbrella Issue ] Cleanup remaining DeprecatedFileSystem users

[David T. Lewis]

On Wed, May 09, 2018 at 01:41:47PM +0200, Thierry Goubier wrote:
> 2018-05-09 13:14 GMT+02:00 Sven Van Caekenberghe :
> > You mean you made a subclass of StandardFilestream yourself ?
> 
> There are a few subclasses of StandardFileStream in OSProcess:
> AttachableFileStream, AsyncFileReadStream and
> BufferedAsyncFileReadStream. They will need to be ported.
> 
> > If yes, can you describe briefly why you did so ?
> 
> As far as my analysis go, AttachableFileStream refers to and
> manipulates the underlying ioHandle of a unix pipe stream,
> AsyncFileReadStream interacts with the asynchronous io extensions of
> the vm to observe that io handle, and a BufferedAsyncFileReadStream
> add a thread-safe, buffered, blocking API for smalltalk processe on
> top.

That's right.

I'm sorry I have not been keeping up with this discussion, but one thought
that comes to mind - the AttachableFileStream hierarchy is something that
would better be replaced by some kind of delegation. It was a hack that I
did a long time ago to allow OSProcess to work without modifying the base
image. But since then, we have added multibyte characters and streams, and
we have different file and stream models in Squeak/Pharo/Cuis (Cuis is an
important target platform from my POV).

I don't have a solution in mind, my sense is that if we find a way to make
AttachableFileStream go away, then that solution would very likely make
it unnecessary to develop and maintain separate Squeak/Pharo/Cuis forks
moving forward.

Note that the only fundamental thing that AttachableFileStream does is this:

AttachableFileStream class>>name: aSymbolOrString attachTo: anIOHandle 
writable: readWriteFlag 
"Create a new instance attached to anIOHandle, where anIOHandle
represents an open IO channel. For write streams, this represents two
Smalltalk streams which write to the same OS file or output stream,
presumably with interleaved output. The purpose of this method is to
permit a FileStream to be attached to an existing IOHandle, such as
the IOHandle for standard input, standard output, and standard error."

^ (super basicNew
name: aSymbolOrString
attachTo: anIOHandle
writable: readWriteFlag) initialize


The method comment refers to "IOHandle" which was another old project I was
doing, but IOHandle basically means the #fileID field of a file stream.

If it was just AttachableFileStream that needed to be refactored out of 
existence,
it would be easy. But I later added quite a bit a behavior to 
BufferedAsyncFileReadStream
and I'm not sure what might be needed to make this work properly.

Sven, I suspect that you are the most knowledgeable in this area, so any
suggestions would be welcome.

Apologies once again for not keeping up with this, but I am glad to see the
discussion.

Dave

Re: [Pharo-dev] Blame support P7

[David T. Lewis]

On Sat, Jan 13, 2018 at 04:49:07PM +0100, Sven Van Caekenberghe wrote:
> 
> 
> > On 13 Jan 2018, at 15:13, Eliot Miranda  wrote:
> > 
> > Hi Sven,
> > 
> >> On Jan 13, 2018, at 6:01 AM, Sven Van Caekenberghe  wrote:
> >> 
> >> 
> >> 
> >>> On 13 Jan 2018, at 05:22, Eliot Miranda  wrote:
> >>> 
> >>> Isn't it important to preserve the ability to exchange code between 
> >>> Pharo, Squeak and Cuis?  Don't you care that the VM development is 
> >>> directly affected by this?  Is the VM and plugin support not important to 
> >>> Pharo?
> >> 
> >> Eliot,
> >> 
> >> Not trying to minimise your contributions (we all love the VM), and I know 
> >> that you favour collaboration, but 
> >> 
> >> (1) what non-trivial code is actively maintained from a single code base 
> >> between Pharo, Squeak and Cuis ?
> > 
> > Alien
> 
> Hmm, not that active. And it requires Pharo/Cuis to use an 'older' loading 
> mechanism (from their perspective evolving to git SCM).
> 
> Other than that, a very short list. You know just as well as that it is close 
> to impossible to do this (Seaside and FUEL are the only real counter 
> examples, and even there it is a lot of work).
> 

Indeed, Seaside and FUEL are good examples.

My own experience is with OSProcess/CommandShell, which I develop in Squeak,
version control in Monticello, and intermittently try to maintain for Pharo
and Cuis.

As you say, this can be difficult to do. Multiple version control approaches
do not make the job any easier, and I can assure you that it does not make
it any more enjoyable. I am probably not a good example, because I am just one
person, not a development team. But for what it's worth, that is my experience.

Dave

Re: [Pharo-dev] Blame support P7

[David T. Lewis]

On Sun, Jan 14, 2018 at 12:52:52AM +0800, Ben Coman wrote:
> 
> @Juan, IIUC Cuis already primarily uses git for Smalltalk source code 
> management.
> How is this currently done and would you partake in a Smalltalk
> implementation of git?
>

You will find Juan's explanation of Git and GitHub for Cuis code management at
https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/blob/master/Documentation/CuisAndGitHub.md

The introduction to that explanation is worth quoting here:

> Cuis includes tools and procedures for managing Smalltalk code. Central
> to this is the management of Packages and Package Files (.pck). But Cuis
> doesn't do version control. Instead, we suggest using external VCS tools.
> In particular, we're using GitHub, and the first project we're hosting
> there is StyledTextEditor.
> 
> The guiding principle is to not duplicate concepts and behavior. As we're
> using an external tool (Git) for version control, then we use it as it
> meant to be used. Most people use Git for version control and a file based
> IDE such as Eclipse for development. Such IDEs don't do version control
> themselves. It is done by Git. Do the same: do not include package version
> control in Cuis. This is a departure from the Monticello /Git integration
> (smallsource and MonticelloFileTree) by Otto Behrens, Dale Henrichs, etc.

Dave

Re: [Pharo-dev] Pharo6 with Ubuntu and OSSubprocess/threaded heartbeat crashes and problems

[David T. Lewis]

On Tue, Mar 06, 2018 at 11:49:14AM +0100, Sabine Manaa wrote:
> Hi,
> 
> I can report the following:
> 
> The problem with the Buffer overflow remains also with OSProcess.
> Also with the normal vm (https://get.pharo.org ).
> 
> Summary: I have buffer overflow crashes with my application on Linux.
> The same code runs without problems on MacOS and on Windows.
> I have the same test data/database and do exactly the same.
> 
> The problem occurs only after calling several times OSProcess/OSSubprocess.
> This is the reason why I first had a suspicion that OSSubprocess is
> responsible for the crashes.
> 
> Can anyone tell me how to proceed to find the reason for the buffer
> overflow?
> There is no PharoDebug.log
> 
> Help is very appreciated!
> 
> This is the message when terminating:
> 
> root@Pharo_ubuntu_16:/spf/build# ./pharo Pharo-Productive-SPF.image
> --no-quit
> 
> SmalltalkImageInstanceID class>>startUp*** buffer overflow detected ***:
> /spf/build/pharo-vm/lib/pharo/5.0-201707201942/pharo terminated
>

What is class SmalltalkImageInstanceID? I have a Pharo 6.1 image, and I
am running on Ubuntu. I loaded OSProcess to check this out, but I am not
sure what that #startUp message is doing.

Thanks,
Dave
 

Re: [Pharo-dev] Pharo6 with Ubuntu and OSSubprocess/threaded heartbeat crashes and problems

[David T. Lewis]

Thank you for the follow up on this. This kind of problem can be very difficult
to find and debug!

Dave


On Wed, Mar 07, 2018 at 04:30:33PM +0100, Sabine Manaa wrote:
> Now I know the reason for the crashes. I did a lot of log writing when
> creating my reports (they use OSSubprocess).
> This was only for debugging. Removing them removed the problem. OSProcess
> and OSSubprocess are not guilty :-)
> 
> I can crash the image with this (try several times)
> (in /opt/local/bin/convert there is imagemagick):
> 
> 
> 1000 timesRepeat: [
> | theFileName theStream thePath theFullPathAndFilename isFileNew |
> OSSUnixSubprocess new
> shellCommand: '/opt/local/bin/convert -background white -alpha remove
> -density 200
> /Library/WebServer/Documents/reports/201801231322-419088218-250457/20170227_19_messdefuerhunde_Fahrten.pdf
> /Library/WebServer/Documents/reports/201801231322-419088218-250457/20170227_19_messdefuerhunde_Fahrten_%02d.png';
> redirectStdout;
> run.
> thePath := '/data/db/'  asFileReference.
> thePath exists
> ifFalse: [ thePath createDirectory ].
> theFileName := 'HALLO_' , Date today mmdd , '.txt'.
> theFullPathAndFilename := ('/data/db/'  , theFileName) asFileReference .
> theFullPathAndFilename asFileReference exists
> ifTrue: [ isFileNew := false.
> theStream := MultiByteFileStream oldFileNamed: theFullPathAndFilename.
> theStream position: theStream size.
> theStream crlf ]
> ifFalse: [ isFileNew := true.
> theStream := MultiByteFileStream newFileNamed: theFullPathAndFilename ].
> [ isFileNew
> ifTrue: [ theStream
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> nextPutAll: Time now printString;
> crlf ].
> theStream nextPutAll:'' ]
> ensure: [ theStream close ].
> ].
> 
> 
> 2018-03-07 2:16 GMT+01:00 David T. Lewis <le...@mail.msen.com>:
> 
> > On Tue, Mar 06, 2018 at 11:49:14AM +0100, Sabine Manaa wrote:
> > > Hi,
> > >
> > > I can report the following:
> > >
> > > The problem with the Buffer overflow remains also with OSProcess.
> > > Also with the normal vm (https://get.pharo.org ).
> > >
> > > Summary: I have buffer overflow crashes with my application on Linux.
> > > The same code runs without problems on MacOS and on Windows.
> > > I have the same test data/database and do exactly the same.
> > >
> > > The problem occurs only after calling several times
> > OSProcess/OSSubprocess.
> > > This is the reason why I first had a suspicion that OSSubprocess is
> > > responsible for the crashes.
> > >
> > > Can anyone tell me how to proceed to find the reason for the buffer
> > > overflow?
> > > There is no PharoDebug.log
> > >
> > > Help is very appreciated!
> > >
> > > This is the message when terminating:
> > >
> > > root@Pharo_ubuntu_16:/spf/build# ./pharo Pharo-Productive-SPF.image
> > > --no-quit
> > >
> > > SmalltalkImageInstanceID class>>startUp*** buffer overflow detected ***:
> > > /spf/build/pharo-vm/lib/pharo/5.0-201707201942/pharo terminated
> > >
> >
> > What is class SmalltalkImageInstanceID? I have a Pharo 6.1 image, and I
> > am running on Ubuntu. I loaded OSProcess to check this out, but I am not
> > sure what that #startUp message is doing.
> >
> > Thanks,
> > Dave
> >
> >
> >

Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

[David T. Lewis]

OS pipes are a similar case. On Pharo, you can run CommandShellTestCase to
provide some test coverage for this.

Dave

On Wed, Apr 11, 2018 at 03:13:35PM +0200, Denis Kudriashov wrote:
> Thanks for explanation.
> 
> I think it would be same scenario for socket stream where #atEnd is not
> blocking. So I agree that it is expected behaviour.
> 
> The example is general enough to expect it to be working for any given pair
> of in and out streams. So our streams should support this.
> 
> 
> 2018-04-11 14:56 GMT+02:00 Alistair Grant :
> 
> > Hi Sven & Dennis,
> >
> > On 11 April 2018 at 12:04, Denis Kudriashov  wrote:
> > > Hi Alistair.
> > >
> > > I don't think anybody is annoyed by you. You are doing really good job.
> > And
> > > nice thing that you are super patient to continue :)
> > >
> >
> > On 11 April 2018 at 12:13, Sven Van Caekenberghe  wrote:
> > >
> > > Yes, Alistair, you are a top notch open source contributor !
> > >
> > > For me, this discussion is about the difference between looking from low
> > level details/issues/changes up, vs, from a higher level down.
> >
> > Thanks for your kind words.
> >
> >
> >
> > > What I try to understand is why blocking atEnd is bad?
> > > Here is code from VMMaker:
> > >
> > > [stdin atEnd] whileFalse:
> > > [| nextChunk |
> > > stdout nextPutAll: 'squeak> '; flush.
> > > nextChunk := stdin nextChunkNoTag.
> > > [nextChunk notEmpty and: [nextChunk first isSeparator]] whileTrue:
> > > [nextChunk := nextChunk allButFirst].
> > > Transcript cr; nextPutAll: nextChunk; cr; flush.
> > > [stdout print: (Compiler evaluate: nextChunk); cr; flush]
> > > on: Error
> > > do: [:ex| self logError: ex description inContext: ex signalerContext to:
> > > stderr]].
> > > quitOnEof ifTrue:
> > > [SourceFiles at: 2 put: nil.
> > > Smalltalk snapshot: false andQuit: true]
> > >
> > >
> > > I am not see why it breaks with blocking #atEnd. Can you explain?
> >
> >
> > First consider the case where #atEnd doesn't block and we just want to
> > evaluate 4+3:
> >
> > 1. #atEnd will return false
> > 2. the loop will print the prompt
> > 3. wait for input (stdin nextChunkNoTag)
> > 4. print the result
> > 5. goto 1.
> >
> > So the screen will look like:
> >
> > squeak> 4+3!
> > 7
> > squeak> [cursor here]
> >
> > Which is what we expect (prompt, input, result, prompt).
> >
> > If #atEnd is blocking the VM will hang at step 1 until the user enters
> > something in the terminal.  In Ubuntu at least terminal input appears to
> > be line buffered, so for the example above the terminal will look like:
> >
> > 4+3!
> > squeak> 7
> > [cursor here]
> >
> > We don't get the prompt when the program is started, the result is
> > printed after the prompt, and then there's just a cursor sitting at the
> > start of the next line.
> >
> > Obviously the program could be re-written to have the correct output
> > with #atEnd blocking.  But I'm arguing that this program is
> > representative of many others, and we don't want to break backward
> > compatibility in this case.
> >
> > Cheers,
> > Alistair
> >
> >
> >
> >
> >
> >
> >
> >
> > > 2018-04-11 11:41 GMT+02:00 Alistair Grant :
> > >>
> > >> Hi Sven,
> > >>
> > >> Oh dear.  I feel as though I'm not getting my concerns across at all
> > >> well, and I'm pushing hard enough that all I'm going to do is make
> > >> people annoyed.  So let me try to restate the issue one last time
> > >> before answering your questions directly.
> > >>
> > >> Pharo & Squeak have unwritten rules about stream usage that I suspect
> > >> have just emerged over time without being designed.
> > >>
> > >> If you want to be able to iterate over any stream, and in particular
> > >> stdin from a terminal (which, as far as I know, is the outlier that
> > >> causes all the problems) you have to follow these rules:
> > >>
> > >> 1.  If the stream is character / byte oriented you have to check for
> > >> EOF using "stream next == nil".  #atEnd can be used, but you'll still
> > >> have to do the nil check.
> > >>
> > >> 2.  All other streams have to check for EOF (end of stream) using
> > >> #atEnd.  "stream next == nil" can be used, but you'll still need to
> > >> test #atEnd to determine whether nil is a value returned by the
> > >> stream.
> > >>
> > >> If you write code that you want to be able to consume characters,
> > >> bytes or any other object, you'll have to test both "stream next ==
> > >> nil" and #atEnd.
> > >>
> > >> The rules are the result of the original blue book design being that
> > >> #atEnd should be used, and then character input from a terminal being
> > >> added later, but always returning an EOF character (nil) before #atEnd
> > >> answers correctly.
> > >>
> > >> At the moment, ZnCharacterEncoder uses #atEnd on character / byte
> > >> streams, so fails for stdin on a terminal.
> > >>
> > >> Back to your questions:
> > >>
> > >> On 11 April 2018 at 11:12, Sven Van Caekenberghe 

Re: [Pharo-dev] [ANN] The STON Specification

[David T. Lewis]

This is very clear and well written.

Dave

> Hi,
>
> Since there can never be enough documentation I finally took some time to
> write a more formal description of STON as a data format.
>
>   https://github.com/svenvc/ston/blob/master/ston-spec.md
>
> The idea is to let this stabilise a bit and to then update the two other
> documents describing STON, where necessary:
>
>   https://github.com/svenvc/ston/blob/master/ston-paper.md
>   
> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html
>
> Also, the latest changes in STON have to make their way to the Pharo image
> as well.
>
>   https://github.com/svenvc/ston
>
> All feedback is welcome.
>
> Sven
>
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>
>
>
>

Re: [Pharo-dev] Better management of encoding of environment variables

[David T. Lewis]

On Fri, Jan 18, 2019 at 01:40:26PM +0100, Sven Van Caekenberghe wrote:
> Dave,
> 
> > On 18 Jan 2019, at 01:54, David T. Lewis via Pharo-dev 
> >  wrote:
> > 
> > 
> > From: "David T. Lewis" 
> > Subject: Re: [Pharo-dev] Better management of encoding of environment 
> > variables
> > Date: 18 January 2019 at 01:54:34 GMT+1
> > To: Pharo Development List 
> > 
> > 
> > On Thu, Jan 17, 2019 at 04:57:18PM +0100, Sven Van Caekenberghe wrote:
> >> 
> >>> On 16 Jan 2019, at 23:23, Eliot Miranda  wrote:
> >>> 
> >>> On Wed, Jan 16, 2019 at 2:37 AM Sven Van Caekenberghe  
> >>> wrote:
> >>> 
> >>> The image side is perfectly capable of dealing with platform differences
> >>> in a clean/clear way, and at least we can then use the full power of our
> >>> language and our tools.
> >>> 
> >> Agreed.  At the same time I think it is very important that we don't reply
> >> on the FFI for environment variable access.  This is a basic cross-platform
> >> facility.  So I would like to see the environment accessed through 
> >> primitives,
> >> but have the image place interpretation on the result of the primitive(s),
> >> and have the primitive(s) answer a raw result, just a sequence of 
> >> uninterpreted
> >> bytes.
> >> 
> >> OK, I can understand that ENV VAR access is more fundamental than FFI
> >> (although FFI is already essential for Pharo, also during startup).
> >> 
> >>> VisualWorks takes this approach and provides a class UninterpretedBytes
> >>> that the VM is aware of.  That's always seemed like an ugly name and
> >>> overkill to me.  I would just use ByteArray and provide image level
> >>> conversion from ByteArray to String, which is what I believe we have 
> >>> anyway.
> >> 
> >> Right, bytes are always uninterpreted, else they would be something else.
> >> We got ByteArray>>#decodedWith: and ByteArray>>#utf8Decoded and our 
> >> ByteArray
> >> inspector decodes automatically if it can.
> >> 
> > 
> > Hi Sven,
> > 
> > I am the author of the getenv primitives, and I am also sadly uninformed
> > about matters of character sets and strings in a multilingual environment.
> > 
> > The primitives answer environment variable variable values as ByteString
> > rather than ByteArray. This made sense to me at the time that I wrote it,
> > because ByteString is easy to display in an inspector, and because it is
> > easily converted to ByteArray.
> > 
> > For an American English speaker this seems like a good choice, but I
> > wonder now if it is a bad decision. After all, it is also trivially easy
> > to convert a ByteArray to ByteString for display in the image.
> > 
> > Would it be helpful to have getenv primitives that answer ByteArray
> > instead, and to let all conversion (including in OSProcess) be done in
> > the image?
> > 
> > Thanks,
> > Dave
> 
> Normally, the correct way to represent uninterpreted bytes is with a 
> ByteArray. Decoding these bytes as characters is the specific task of a 
> character encoder/decoder, with a deliberate choice as to which to use.
> 
> Since the getenv() system call uses simple C strings, it is understandable 
> that this was carried over. It is probably not worth or too risky to change 
> that - as long as the receiver understands that it is a raw OS string that 
> needs more work.
> 
> Like with file path encoding/decoding, environment variable encoding/decoding 
> is plain messy and complex. IMHO it is better to manage that at the image 
> level where we are more agile and can better handle that complexity.
> 

Thanks Sven, that makes perfect sense to me.

>
> BTW: using funny Unicode chars, like  
> [https://www.fileformat.info/info/unicode/char/1f388/index.htm] is something 
> even English speakers do.
>

You are right. I wrote those getenv primitives 20 years ago and
back then we were still doing our emoticons like this:

;-)

Thanks,
Dave
 

Re: [Pharo-dev] Green tests

[David T. Lewis]

Also useful for "quick and dirty" debugging with concurrent processes:

  OSProcess trace: 'my debugging note'

This prints to console, identifying the process, receiver, method, and
whatever additional notes you may want to add.

Dave

On Wed, Apr 03, 2019 at 07:34:22AM +0200, Tudor Girba wrote:
> Instead of printing, you can also use Beacon. We actually developed Beacon 
> specifically to be able to debug a parallel distributed system.
> 
> Cheers,
> Doru
> 
> > On Apr 2, 2019, at 11:56 PM, Ben Coman  wrote:
> > 
> > 
> > On Tue, 2 Apr 2019 at 23:36, ducasse  wrote:
> > I???m inherently race condition incompatible and just avoid concurrent 
> > programming because I???m too dull on this. 
> > 
> > I've solved a few Pharo race conditions before, so just a tip (that you 
> > maybe already know).
> > You can't troubleshoot a race condition by standard debugging, because it 
> > alters the timing.
> > Best that I've done before is resorting to plain old print-line tracing 
> > between every line of the suspect code,
> > or the equivalent threadsafe collection if its not a hard crash.
> > e.g
> > someMethod
> > threadsafeCollection add: "methodname-A" , testCondition
> > original method line 1
> > threadsafeCollection add: "methodname-B", testCondition
> > original method line 2
> > threadsafeCollection add: "methodname-C", testCondition
> > 
> > and then looking for cyclic patterns in that data and breaks in those 
> > patterns.
> > 
> > I can have a look in a few weeks.  Right now working 13 hour days to the 
> > end of the week
> > and then vacationing in China for 10 days.
> >  
> 
> --
> www.feenk.com
> 
> "Sometimes the best solution is not the best solution."
> 

Re: [Pharo-dev] OSProcess for Pharo 7

[David T. Lewis]

Hi Mariano,

On Sun, Mar 17, 2019 at 08:29:59PM -0300, Mariano Martinez Peck wrote:
> >
> > ducasse wrote
> > > this is not because we do not care of Pharo that we will fix it
> > > immediately.
> >
> > Yes of course, just want to keep it on the radar (and maybe Mariano will
> > read he he)
> >
> >
> I always do :)

Me too :-)


> But honestly, I don't have enough time to carry on that myself. As you
> could see with latest Guillermo pass, I reviewed every PR and tried to give
> my advice and support as much as possible.

I saw Sean's earlier email (see below) which reminded me of your ideas about
having a child process listener that could provide child exit notifications
to both OSSubprocess and OSProcess. This seems to me to be the right thing
to do, but it would take some work by somebody (not by me, but it could
be done).

Meanwhile, prompted by Sean's , I think maybe I see another way that
we could handle the problem, at least in the near term.


On Fri, Mar 15, 2019 at 01:42:26PM -0500, Sean P. DeNigris wrote:
> Max Leske wrote
> >> I was talking about OSSubProcess
> > Well, I'm not working with that at the moment ;)
>
> [OT] Just want to bump the problem that OSProcess and OSSubProcess can't
> currently be used in the same image. This is an important problem because
> users can't control which library dependencies use. Mariano hinted at a
> solution. It didn't seem like a ton of work, but it's beyond my
> understanding.
>

I think I see another way that this can be handled for an image that
contains both OSSubprocess and OSProcess. I loaded both into a Pharo 7
image, then added this:

OSSVMProcess>>update: aParameter
"If subscribed to events from the OSProcess child watcher, the
changed/update: mechanism will dispatch child exit notification
here. To subscribe to notification from OSProcess add the singleton
OSSVMProcess as a dependent of the OSProcess accessor singleton.
When any child process exits, this method will be called with
parameter #childProcessStatus"

"OSProcess accessor addDependent: OSSVMProcess vmProcess"

(aParameter == #childProcessStatus) ifTrue:
[^ self updateActiveChildrenAndNotifyDead].


I stopped the child watcher in OSSVMProcess (by commenting out the last
two lines in OSSVMProcess>>initialize just to see if this would work),
then added the changed/update: dependency with
  "OSProcess accessor addDependent: OSSVMProcess vmProcess"

After restarting the image, it appears to be working, at least so far
has having both OSP and OSSP handle their child process exits properly
in the same image.

I don't have time to play with this any more right now, but it does
look like it might be a workable approach. It should be possible to
have the initializer for OSSVMProcess use its normal child exit handler,
but hook into the OSP handler instead if that happens to be present
in the image.

Dave 

[Pharo-dev] OSProcess and CommandShell available on GitHub for Pharo users

[David T. Lewis]

Alistair Grant and I, with the support of Feenk, have made GitHub repositories
for OSProcess and CommandShell at:

  https://github.com/dtlewis290/OSProcess-Tonel
  https://github.com/dtlewis290/CommandShell-Tonel

Alistair did the conversions using Peter Uhn??k's migration tool, and I set up
the repositories so that they can now be loading in Pharo as follows:

  Metacello new
repository: 'github://dtlewis290/OSProcess-Tonel/src';
baseline: 'OSProcess';
load.

  Metacello new
repository: 'github://dtlewis290/CommandShell-Tonel/src';
baseline: 'CommandShell';
load.

Note, the two respositories are named *-Tonel because I also maintain a GitHub
repository for OSProcess on Cuis, and will probably do repositories in Squot
format in the future. 

@Thierry Goubier - If you have an account on GitHub I will add you as a
collaborator (but my own development work remains on squeaksource so I prefer
contributions there anyway).

Enjoy,

Dave and Alistair

Re: [Pharo-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]

[David T. Lewis]

Guille,

This is excellent work. Thank you also for your careful explanations.

Dave

On Wed, May 15, 2019 at 10:33:38AM +0200, Guillermo Polito wrote:
> Hi Eliot,
> 
> > El 15 may 2019, a las 3:20, Eliot Miranda  
> > escribi??:
> > 
> > Hi Guille,
> > 
> > On Tue, May 14, 2019 at 7:30 AM Guillermo Polito  > > wrote:
> > 
> >  - I had to review the AST-to-AST transformation, checking the output, 
> > comparing it to what squeak does and so on...
> > 
> > Can you show a little of th differences in output between Squeak compiler 
> > derived sources and RB parser derived sources?  I'm not going to demand 
> > changes or fixes or that they be exactly the same.  But I am curious to 
> > know what kind of changes there are.
> 
> From what I have now, I can see three main kind of differences (I actually 
> compared with source code I generated myself from squeak and the existing 
> sources in the git repo).
> I did not work on them because the VM seemed to work, so I left them for 
> later:
> 
> 1) some comments are placed in different places in the output source code. 
> This is entirely cosmetic, but it brings unnecessary noise to C source diffs.
> 
> 2) labels in the gnu???ed code are numbered differently. I did not spot any 
> ???bug??? in that yet, I did not dig enough into the gnuification to see why 
> the difference, and I???m not entirely sure that this difference is caused by 
> that either :)???
> Anyhow, this is mostly annoying because it also brings lots of noise to diffs.
> 
> 3) some for loops inline the condition while they probably shouldn't. For 
> example:
> 
> 0 to: (self numSlotsOf: array1) - 1 do: [ ??? ]
> 
> gets in my output translated to 
> 
> for (i = 0; i < (numSlotsOf(array1)); i += 1) { ...
> 
> while in the version generated from squeak
> 
> for (i = 0, iLimiT = ((numSlotsOf(array1)) - 1); i <= iLimiT; i += 1) { ???
> 
> This seems the most ???dangerous" difference I have so far specially if the 
> limit condition has some side effect.
> We will probably agree in that this third point should be fixed ^^.
> 
> >  
> >  - I???ve written several unit tests to ensure that future migrations are 
> > easier to do
> >  - I???ve introduced several compatibility classes/methods related to 
> > PackageInfo, Time and so on???
> > 
> > This is great1  Thank you.  Perhaps we can move some of the source 
> > generation tests into the core.  It would be good for there to be tests 
> > that apply to the Squeak compiler side too.  I never took the time to run 
> > tests (I look at the generated source every time I make a change Slang), 
> > and writing a really good set of tests always seemed like a lot of work.  
> > But I might be tempted to add to the ones you've written. 
> 
> Maybe in the next weeks I can try to massage the tests to make them less 
> dependent on the source (wether squeak's or pharo???s) AST.
> The thing is that Pharo???s AST is closer to the source code, while 
> Squeak???s has already some pre-cooked code transformations (like #ifNil: & 
> co), and my tests assume a specific AST structure to traverse the tree and 
> compare???
> 
> Cheers,
> Guille

Re: [Pharo-dev] Are people using the Pharo VM builds from BinTray?

[David T. Lewis]

On Fri, Dec 13, 2019 at 10:33:29AM +0100, Esteban Lorenzano wrote:
> Hi,
> 
> > On 12 Dec 2019, at 23:20, David T. Lewis  wrote:
> > 
> > Hi,
> > 
> > There is an automated CI build for opensmalltalk VMs with artifacts for
> > Pharo, Squeak, and Newspeak at https://bintray.com/opensmalltalk/vm/cog/.
> > I suspect that the Pharo builds may be redundant with the official PharoVM
> > distributions.
> 
> Those VMs are also copied into our file sever and we take them from there. 
> 
> > Is anyone using the Pharo VMs from BinTray? Otherwise if it's not a problem
> > for any Pharo users, I think the CI proprietors would like to turn off
> > some of their unnecessary builds to save resources. My understanding is
> > that the builds can easily be turned back on as needed in the future.
> 
> I do not think we use the bintray, but we still use the osvm builds as our 
> standard distribution.
> 
> > 
> > Fabio asked Marcel who asked me, so I am asking you :-)
> 
> So I???m answering you so you can answer Marcel who can tell Fabio ;)
> 


Thanks Esteban :-)

Dave