Re: [Pharo-dev] [squeak-dev] Why do we have SmallDictionary?

[Andres Valloud]

In addition, open addressing with linear probing has superior cache line 
read behavior (no indirection / random traversal, and if the first probe 
misses the second one was likely cached by the first one).


On 6/8/18 1:37 , Levente Uzonyi wrote:

On Fri, 8 Jun 2018, Clément Bera wrote:


Hi Max,
Theoretically, for a small number of elements, usually somewhere
between 3 and 30 depending on implementations, a linear search is
faster than a hash search, especially in the Pharo dictionary hash search
implementation.

Efficient dictionary implementations are usually bucket-based. The
dictionary holds a certain number of buckets, and based on the key
hash, the bucket where the key value is present is determined. Small
buckets are linear (arrays or linked list). Large buckets are
typically balanced binary trees (red-black trees typically). Under a
certain number of elements there is a single bucket, which means a linear
search is performed, as for the SmallDictionary. When it grows the
dictionary search becomes a combination between a hash search and a
linear or tree search.


The bucket-based list you described is called separate chaining
collision resolution, and it won't have any better performance in Squeak
than the currently used open addressing.
However, it is preferrable, when you can't trust your hash function.
When you do "crazy" things, like using a tree as a bucket, you just
exchange the possible O(n) worst case time with O(log(n)).



Pharo dictionary search is first hash-based, then all the buckets are
represented next to each other in the same arrays and a linear search
is performed there, leading to many collisions and slower search
time (especially when the element is not found), sometimes the code
searches over multiple buckets because the dictionary is too full or
there are too many near-collisions. The memory consumption is
competitive with the advanced implementations though (precise
measurements would need to be made).


This is called open addressing, and if your hash function is good
enough, the lists are short, so those "many collisions and slower search
time" just don't happen. And no, separate chaining needs a lot more
memory than open addressing, unless you manage to get rid of
Associations, but that's a lot harder than it sounds like.



Method dictionaries are represented differently to optimize the
look-up logic.


MethodDictionary is optimized to let the VM do lookups more easily.
Memory consumption is worse when the dictionary is sparse, but gets
better as it gets filled up compared to the current Dictionary
implementation (assuming Pharo still uses the same representation as
Squeak does).



If you want to improve things and have one dictionary implementation
instead of two, implement or look for a bucket based dictionary and
put it in the base image instead of Dictionary. This is quite some work
since there are many APIs to port. You can look at the Pinnochio
implementation, it's quite good but they've not implemented large
buckets.


Well, as an experiment, I'm sure it'll be fun. But don't expect better
performance nor better memory usage.

Levente





On Fri, Jun 8, 2018 at 8:46 AM, Max Leske  wrote:
  Hi,

  I was messing around with SmallDictionary when I suddenly
realised that I can't find a single reason to use it over a normal
Dictionary. While its name and class comment imply that it is somehow
  an optimised Dictionary, I don't see any measurement where that
actually holds up. The following was run in a Pharo 7 image on a
recent VM (see below):

  | d |
  d := SmallDictionary new.
  d sizeInMemory. "24"
  [10 timesRepeat: [
  1 to: 100 do: [ :i | d at:i put: i] ] ] timeToRun.
"0:00:00:05.226"

  [10 timesRepeat: [
  d at: 48 ifAbsent: [] ] ] timeToRun. "0:00:00:00.041"



  | d |
  d := Dictionary new.
  d sizeInMemory. "16"
  [10 timesRepeat: [
  1 to: 100 do: [ :i | d at:i put: i] ] ] timeToRun.
"0:00:00:00.385"
  [10 timesRepeat: [
  d at: 48 ifAbsent: [] ] ] timeToRun.  "0:00:00:00.006"


  As you can see, SmallDictionary is 8 bytes larger per instance
and significantly faster while reading and writing (I know that this
isn't a good benchmark but it suffices to make my point).


  Is anyone aware of a reason for hanging on to SmallDictionary?
I'm also curious to know how SmallDictionary came to be. There must
have been some advantage over Dictionary at some point in the
  past.


  Cheers,
  Max





  Image version: Pharo 7.0
  Build information:
Pharo-7.0+alpha.build.961.sha.a69e72a97136bc3f93831584b6efa2b1703deb84
(32 Bit)

  VM version: CoInterpreter VMMaker.oscog- nice.2281 uuid:
4beeaee7-567e-1a4b-b0fb-bd95ce302516 Nov 27 2017
  StackToRegisterMappingCogit VMMaker.oscog-nice.2283 uuid:
2d20324d-a2ab-48d6-b0f6-9fc3d66899da Nov 27 2017
  VM: 201711262336

Re: [Pharo-dev] float & fraction equality bug

[Andres Valloud]

Because by definition, a floating point value is of the form

+/- 2^e * m

for some 0 <= m < M, and e in some range of values (some positive, some 
negative).  There are other special cases that don't matter here, such 
as NaN, INF, denormals, etc.


So now set up the equality you want.  Since it's positive we can skip 
the sign.  Hence,


2^e * m = 1/10

Or, rather,

10 * 2^e * m = 1

The Fundamental Theorem of Arithmetic shows this is impossible.

Andres.

On 11/9/17 6:48 , Tudor Girba wrote:

Hi,

Thanks for the answer. The example I provided was for convenience.

I still do not understand why it is wrong to expect 0.1 = (1/10) to be true.

Doru



On Nov 9, 2017, at 3:36 PM, Nicolas Cellier 
 wrote:

Nope, not a bug.

If you use Float, then you have to know that (x -y) isZero and (x = y) are two 
different things.
Example; Float infinity

In your case you want to protect against (x-y) isZero, so just do that.

2017-11-09 15:15 GMT+01:00 Tudor Girba :
Hi,

I just stumbled across this bug related to the equality between fraction and 
float:
https://pharo.fogbugz.com/f/cases/20488/x-y-iff-x-y-0-is-not-preserved-in-Pharo

In essence, the problem can be seen that by doing this, you get a ZeroDivide:
x := 0.1.
y := (1/10).
x = y ifFalse: [ 1 / (x - y) ]

The issue seems to come from the Float being turned to a Fraction, rather than 
the Fraction being turned into a Float:

Fraction(Number)>>adaptToFloat: rcvr andCompare: selector
"If I am involved in comparison with a Float, convert rcvr to a
Fraction. This way, no bit is lost and comparison is exact."

rcvr isFinite
ifFalse: [
selector == #= ifTrue: [^false].
selector == #~= ifTrue: [^true].
rcvr isNaN ifTrue: [^ false].
(selector = #< or: [selector = #'<='])
ifTrue: [^ rcvr positive not].
(selector = #> or: [selector = #'>='])
ifTrue: [^ rcvr positive].
^self error: 'unknow comparison selector'].

^ rcvr asTrueFraction perform: selector with: self

Even if the comment says that the comparison is exact, to me this is a bug 
because it seems to fail doing that. What do you think?

Cheers,
Doru


--
www.tudorgirba.com
www.feenk.com

"Problem solving should be focused on describing
the problem in a way that makes the solution obvious."








--
www.tudorgirba.com
www.feenk.com

"We are all great at making mistakes."









.

Re: [Pharo-dev] Odd (and wrong) implementation of Float>>sign

[Andres Valloud]

Martin,

On 1/30/17 16:41 , Martin McClure wrote:


IEEE-754 does not define a "sign" operation. ANSI Smalltalk and ISO/IEC
10967 do, so it makes sense to provide a #sign that conforms to those
standards -- answering -1, 0, or 1.

IEEE-754 does define an isSignMinus operation, answering true or false,
so it makes sense to provide that operation in addition to the ones
specified by the other standards.


Yes on isSignMinus.  It may also be useful to define #signBit, and 
implement #isSignMinus as ^self signBit = 1.


These types of messages are very useful for testing the implementation 
is doing the right thing IEEE-wise (hence VM-primitive-wise), so that 
it's possible to implement e.g. ISO 10967 with confidence.


Andres.

Re: [Pharo-dev] Odd (and wrong) implementation of Float>>sign

[Andres Valloud]

The 2007 update of 10967 is somewhat more helpful. It replaces the
"sign" operation with one called "signum" which returns 1, -1, or NaN.
It returns 1 for positive zero and positive infinity, and -1 for
negative zero and negative infinity. If given a qNaN, it returns a qNaN.
If given a signaling NaN, it returns a qNaN but notifies the application
of an "invalid" situation.


That's ISO 10967.  However, IEEE-758-2008 specifies implementations 
shall provide isSignMinus(x).  This operation is true if and only if x 
has negative sign, and applies to zeros and NaNs.


IMO, values that operate according to IEEE-758-2008 should behave as 
specified by IEEE-758-2008 by default.  The ISO 10967 standard seems 
more of a layer on top of things like IEEE-758-2008, because it's a 
standard for language agnostic arithmetic.


So, shouldn't #sign behave as in IEEE-758-2008?

^self isSignMinus ifTrue: [-1] ifFalse: [1]

It looks like the real question is whether Float's methods are merely 
renaming IEEE-758-2008 functionality (if yes, why rename at all and add 
to the confusion?), ISO 10967 functionality, or are something else 
entirely (e.g. Smalltalk provided behavior in the context of the 
Smalltalk experience).


For example, does it make sense -0.0 < 0.0 to answer false (because 
zeros must be numerically equivalent) while -0.0 ~= 0.0 sign answers 
true?  That depends on where #sign belongs to.  This is very important 
because if #sign isn't IEEE-758 or ISO 10967, then why use it or care 
about how it behaves?  And if there's no good explanation, then why is 
#sign there in the first place?


Andres.

Re: [Pharo-dev] roundTo: strange behavior

[Andres Valloud]

Hey Martin... I agree on the confusion.

> I still recommend not rounding the number itself, but
rounding the printing of the number

Yes.

If "rounding" had well specified behavior that was at least somewhat 
reasonable (e.g. return the closest floating point value and round to 
even if tied), I'd accept floating point results that don't print 
"pretty" in base 10.  That's the way things are, there's no point in 
pretending IEEE-754 is not base 2.


If "rounding" is being done in place of "printing", or obtaining a 
"string representation", I think that won't work as expected because 0.1 
not representable etc.


For "printing", the algorithms were Dragon4 and Grisu3 last time I checked.

For "floating point that works nice in base 10", there's IEEE-854.  For 
those unfamiliar with the spec, it's been implemented in non-exotic 
hardware already.


> you did proof that, as you said "1.2002 *is* the correct 
answer for 1.19 roundTo: 0.1.".


As a suggestion for these types of problems, try iterating over a 
floating point bit range converting each to a true fraction and see 
which one is closest?


> testIfCompletelyBroken [doesn't round to even]

I'd agree rounding to even would be better.


Guillermo,

> And also: I'd argue that you NEVER EVER want to use simple Floats for 
financial applications


Define "financial" applications?  In the risk management world, tying 
down that last $1 difference will likely cost more electricity (and way 
more time) than the $1 is worth.



John,

> While ANSI says that the numbers should be converted using the 
default conversion table, all of the Smalltalk’s that I’ve tried don’t 
follow that. I’ve tried Dolphin, VW, Pharo, & VAST, and all return 
fractions. I prefer this behavior over the ANSI behavior.


I agree that ANSI doesn't always make the best sense.  If it were up to 
me, I'd follow the relevant standard to the letter.  That's more 
important than providing a custom interpretation of an interoperability 
mechanism just for Smalltalk (or a particular dialect) by default.



Nicolas,

> printShowingDecimalPlaces:

That's quite a long selector.  Has anyone considered an approach along 
the lines of this?


1.2345 %f #'3.2' => '  1.23'
1.2345 %f #'03.2' => '001.23'


A reminder: calling printf() via FFI is undefined behavior.

Andres.

On 10/26/16 23:31 , Martin McClure wrote:

On 10/26/2016 08:27 AM, stepharo wrote:

So what is the definition of printShowingMaxDecimalDigit:?
I have the impression that there is a confusion between the rounding a
number and the printing of his value in a given format.


Yes, I see quite a bit of confusion in this thread about that.


we can deprecate roundTo: if necessary.


I think roundTo: is OK. #round: is not, and should be deprecated (at
least for Floats). For Floats, the idea of rounding to a specific number
of decimal digits is a fantasy. Here's why: Floats cannot exactly
represent most common decimal fractions. For example:

0.1 -- not representable

0.2 -- not representable

0.3 -- not representable

0.4 -- not representable

0.5 -- hey, representable!

0.6 -- not representable

0.7 -- not representable

0.8 -- not representable

0.9 -- not representable

1.0 -- representable.

*Printing* a Float to a specific rounded decimal format is a sensible
idea, and should be encouraged. But trying for a "rounded Float"
*number* just can't be done exactly (except by converting to a Fraction).

Fractions can be rounded to exact decimal fractions, so something like
"myFraction roundTo: 1/100" makes sense. But the current implementation
of round: on Fraction that converts to a Float just gets you the misery
detailed above.


On 10/26/2016 01:00 PM, Nicolas Cellier wrote:

I've put a single slice in the inbox for the 3 issues because they all
are related.
See slice comment:

For Float, implement guard to prevent overflow (15471), and use exact
representation for intermediate results (asFraction) so as to avoid
double rounding problems (15473)


The double rounding problem is not the fundamental problem, the
fundamental problem is that what is desired does not exist, because
Floats cannot exactly represent most decimal fractions. So this can't
really fix it.



For Fraction (15472), choose an exact representation for result rather
than Float. This is because such method is particularly usefull for
financial/monetary applications.


Yes. Fraction or ScaledDecimal. Not Floats, unless the developer
understands the needs of the financial world and the limitations of
Floats very well.


Regards,

-Martin

Re: [Pharo-dev] C Variadic Functions via UFFI?

[Andres Valloud]

The identifier printf() can also be a macro...

On 9/23/16 1:15 , Esteban Lorenzano wrote:

along with what Andres says, also no, you do not have support for variadic 
functions.
what you do this is to declare your functions as if they were non-variadic:

pritnf: format intValue:  number
self ffiCall: #(int printf(String format, int number))

that will work… yes is not the best in the word, but it works :)

Esteban



On 23 Sep 2016, at 08:11, Andres Valloud <avall...@smalltalk.comcastbiz.net> 
wrote:

I wouldn't do that if I were you.  The manual says fcntl() can be a macro, and 
you can't call a macro from an FFI.

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/fcntl.h.html

On 9/22/16 15:12 , Mariano Martinez Peck wrote:

Hi guys,

I am wondering if I can wrap some variadic functions (with optional
arguments) with UFFI. Is this possible? If true, how so? For my
particular use case, I am checking if I can call fcntl()
which has a third optional argument.

Thanks in advance,

--
Mariano
http://marianopeck.wordpress.com

Re: [Pharo-dev] C Variadic Functions via UFFI?

[Andres Valloud]

I wouldn't do that if I were you.  The manual says fcntl() can be a 
macro, and you can't call a macro from an FFI.


http://pubs.opengroup.org/onlinepubs/009695399/basedefs/fcntl.h.html

On 9/22/16 15:12 , Mariano Martinez Peck wrote:

Hi guys,

I am wondering if I can wrap some variadic functions (with optional
arguments) with UFFI. Is this possible? If true, how so? For my
particular use case, I am checking if I can call fcntl()
which has a third optional argument.

Thanks in advance,

--
Mariano
http://marianopeck.wordpress.com

[Pharo-dev] Marquette Camp Smalltalk September 15-18th

[Andres Valloud]

Hi!  Your Camp Smalltalk crew has been at it for a while, and we are
very happy to announce a Camp Smalltalk at Marquette, Michigan, on
September 15-18th.  The event's website is at http://mqt.st.  We are
generously hosted by Northern Michigan University (http://www.nmu.edu),
where John Sarkela teaches Smalltalk courses.  Of course, NMU is also
home to the Modtalk project, https://www.modtalk.org.  You can help us
provide a better experience for everyone by registering here:
https://www.picatic.com/event14606861846650.  Don't miss the date!

Re: [Pharo-dev] Pharo website broken on HTTPS

[Andres Valloud]

Also FYI the Pharo Consortium's website at http://consortium.pharo.org/ 
displays a Proxy Error.


On 8/5/16 7:35 , Esteban A. Maringolo wrote:

Hi all,

The Pharo website doesn't work properly if you access it vía HTTPS
(and Chrome), apparently some resources (JS and CSS) are explicitly
linked to HTTP, so if you access it by HTTPS those resource are
blocked by the client.

See attached screenshot.

Regards!

Esteban A. Maringolo

[Pharo-dev] [Smalltalks 2015] --- Invitation

[Andres Valloud]

The Fundación Argentina de Smalltalk proudly invites you to one of the 
premier Smalltalk conferences in the world.  Let's meet at Buenos Aires, 
November 11-13!  For more details, see the invitation here:


http://www.fast.org.ar/Smalltalks2015-invitation.pdf

[Pharo-dev] Camp Smalltalk PDX --- August 21st through the 23rd

[Andres Valloud]

Camp Smalltalk Portland 2015 is go for August 21st through the 23rd! 
(and if you arrive earlier on the 20th that's cool too) Please register 
for the event here:


https://www.picatic.com/event14352674835773927

Help us by filling in the questions so you can get your event shirt. And 
also see that we're having a BBQ with live music on Saturday. You can't 
miss it!

[Pharo-dev] Camp Smalltalk PDX

[Andres Valloud]

  
  
A group of Smalltalkers have been discussing the possibility of
having a Camp Smalltalk in Portland Oregon, August 20-23. If you
think you might be interested in attending such a gathering, please
let us know filling our short survey:

http://goo.gl/forms/Di0rFPMcLu


We also encourage you to join the SCONA mailing list here:

http://scona.us/mailman/listinfo/scona-list


The proposed format is:

* Thursday 20th evening prior: An informal welcoming dinner at a
local pub or similar.

* Friday 21st morning - Sunday 23rd afternoon: The event proper,
consisting of a blend of scheduled talks or discussions and
free-form pair programming.

* Sunday 23rd evening - Farewell dinner.

The whole event is envisioned as being informal enough that showing
up for only part of the time is perfectly acceptable.

Your responses will help us with our planning efforts. Feel free to
forward this message to any other Smalltalk forums or individual
Smalltalkers you think might be interested.

Andres.

  

Re: [Pharo-dev] Parsing Pharo syntax to C/C++

[Andres Valloud]

Side comments...

Some code just has to be written in C, otherwise you cannot rely on the 
behavior being correct and ultimately that's a major show stopper for 
anything you might want to call production code.  Anything having to 
do with calling operating system libraries is a typical example of this 
phenomenon.  So, in those cases, translating Smalltalk to C (or 
interfacing to OS libraries with FFI interfaces) only gives you huge 
maintenance headaches in the long run.  The resulting distractions due 
to bug reports will obliterate any engineering team's performance, and 
it's a race that cannot be won as framed.  POSIX is there for a reason, 
let the C compiler do the job it's been doing for 40+ years.


But suppose one wants to write C code with Smalltalk syntax anyway. 
Nevertheless, C99 and newer still apply and one must be aware of the 
subject matter.  The danger is that Smalltalk is so much more user 
friendly than C, that once one starts writing C in Smalltalk one will 
never look at the automatically generated C.  But is the mechanically 
generated C code valid?  Because if not, then compiling such source code 
can be no better than garbage in, garbage out, and then you cannot 
have production quality.  Similar arguments apply to FFI interfaces. 
Please excuse my ignorance, but I'd like to assume someone's looking 
into this?


What seems to follow is that the architecture of the system should try 
to minimize the dependencies on external libraries, precisely because 
the dependencies cause exposure to foreign rules and regulations that 
slow one down.  Creating dependencies in a reasonable manner should be 
straightforward _for system users_, though.  Letting externals create 
reliable dependencies on one's system should also be facilitated to 
promote healthy modes of interaction with the community at large.


I feel one of our current challenges is to wholeheartedly admit there is 
no royal road to inventing our future, then act accordingly.


Andres.

[Pharo-dev] [Smalltalks 2014] --- Invitation

[Andres Valloud]

  
  
1. Invitation.

The Fundación Argentina de Smalltalk (FAST, http://www.fast.org.ar)
invites you to the 8th International Conference on Smalltalk
Technologies (Smalltalks), to be held from November 5th through
November 7th at the Universidad Tecnológica Nacional, Facultad
Regional Córdoba, located in the city of Córdoba, Argentina.
Everyone, including teachers, students, researchers, developers and
entrepreneurs, are welcome as speakers or attendees.

This year, we are extremely happy to announce Allen and Rebecca
Wirfs-Brock will attend the conference. Allen was instrumental in
developing Digitalk's Visual Smalltalk, as well as a leader in
modular Smalltalk design. Rebecca created the field of
Responsibility Driven Design, which spawned a variety of modern
disciplines such as TDD and BDD. Their presence is just a preview of
an oustanding presentation program you cannot afford to miss.

2. Registration.

Registration is free and now open at

http://www.fast.org.ar/smalltalks2014.

Please make sure to register early to receive the conference's
shirt, as well as to help us plan the conference's social events.

We are accepting donations from participants to help fund the
conference's costs. Please see the Donate section at FAST's website,

http://www.fast.org.ar/smalltalks2014/donate.

Contributions are greatly appreciated, and can be received both in
pesos for local attendees, as well as via Paypal for those coming
from abroad. Please note that donors, including those that have
already sent us their contribution (thank you!), will receive a set
of thank you gifts as well as the conference's shirt. For those of
you that need a receipt, we can provide those on site.

3. Sponsors.

In addition our generous attendee donation contributors, we would
like to thank our sponsors. We would not be able to organize the
conference without their help.

Platinum sponsors:
+ Caesar Systems
+ ESUG
+ GemTalk Systems
+ Instantiations

Silver sponsors:
+ InfOil

Bronze sponsors:
+ 10 Pines
+ Arduino Software
+ Mercap
+ Precision System Design
+ Smallworks

Moreover, this year our official airline carrier is Aerolineas
Argentinas. To book plane tickets with Aerolineas Argentinas, please
go to

http://www.aerolineas.com.ar/Congresos

and select Smalltalks 2014 from the list of events (available soon).

4. Call for participation.

Talk proposal submission for the Industry Track is now open at our
website:

http://www.fast.org.ar/smalltalks2014/technical-session

If you need special arrangements (e.g. because you would like to
hold a workshop session), please indicate so in the abstract. The
Industry Track's submission deadline is October 13th.

Abstract submitters: please send us a photo and a short bio by
answering to this email.

5. Related events.

For more information about related events, such as a Pharo Sprint or
talks and presentations around the dates of the conference, please
visit

http://www.fast.org.ar/smalltalks2014/events

In particular, we like to invite to you Squeakfest 2014, to be held
at the same conference site on from November 3rd through November
4th. This companion event highlights Squeak and Etoys used in the
context of education. To register, please go to the SqueakFest page
under the related events for Smalltalks 2014.

We will update related event information as we get closer to the
conference, so please check for updates.

For any additional questions please reply to this email.

See you in Córdoba!

  

Re: [Pharo-dev] Memory limit problems on Windows VM

[Andres Valloud]

Keep in mind that, depending on how images are loaded, you won't be able 
to load images larger than about 2gb... you might want to test that.


On 7/19/14 13:39 , Tudor Girba wrote:

Wow! I will certainly try this patch.

Doru


On Fri, Jul 18, 2014 at 5:15 PM, Oscar E. A. Callaú
oscar.apo...@gmail.com mailto:oscar.apo...@gmail.com wrote:

Well,

   I fixed the problem by modifying the Pharo.exe file with the 4GB
patch program (http://ntcore.com/4gb_patch.php).

As mentioned before, I think the best solution is that Pharo vms
must be compiled with the flag LARGEADDRESSAWARE, so more memory
space is enabled for 64bits Windows OSs.

Thanks guys

Oscar


On Thu, Jul 17, 2014 at 11:03 PM, Andres Valloud
avall...@smalltalk.comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net wrote:

I worked on that switch before, and IIRC it's just a linker
switch that marks executables a certain way.  So, if you have
imagecfg.exe handy...

http://support.microsoft.com/__kb/297812
http://support.microsoft.com/kb/297812


On 7/17/14 19:25 , Oscar E. A. Callaú wrote:

Well. It seems that Pharo vms must be compiled with that
option :/

Cheers


On Wed, Jul 16, 2014 at 8:44 PM, b...@openinworld.com
mailto:b...@openinworld.com
mailto:b...@openinworld.com mailto:b...@openinworld.com
wrote:

 __
 Eliot Miranda wrote:




 On Wed, Jul 16, 2014 at 2:21 PM, Oscar E. A. Callaú
 oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com__ wrote:

 Hi,

 I'm using Windows 8.1 (update1) 64bits with 4GB
of physical
 RAM. I only need 1.5GB of RAM, so I think I
don't need to
 configure my OS to take /3GB of RAM.


 Well my experience (and others) is that one can't
grow the heap
 much above 1.1 Gb without using the /3Gb switch on
Windows XP.  I
 can't talk for 8.1.  But if 8.x also limits memory
to 2Gb/process
 unless the /3Gb switch is in effect then yes,
you'll need to use
 the /3Gb switch.

 HTH
 Eliot


 Cheers

 On Wednesday, July 16, 2014, Eliot Miranda wrote:

 Hi Oscar,


 On Wed, Jul 16, 2014 at 9:11 AM, Oscar E.
A. Callaú
 oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com wrote:

 Hi guys,

I'm running some experiments in
Pharo. My data
 is aprox. 1.3GB. When I load my image
on Mac,
 everything works perfectly. But, when I
try to load
 the same image on Windows, I get this
error:

 Unable to commit memory (1326649344
bytes requested)

 I tried the Pharo VM, cogMT and  NBcog
with and
 without the option

 AddressSpaceLimit = 2048

 in the ini file

 Please help.

 What OS are you using?  Have you read e.g.

http://msdn.microsoft.com/en-__us/library/windows/hardware/__dn613959(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959(v=vs.85).aspx


http://msdn.microsoft.com/en-__us/library/windows/hardware/__dn613959%28v=vs.85%29.aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959%28v=vs.85%29.aspx?



 Here are some additional links...

 /LARGEADDRESSAWARE (Handle Large Addresses)
http://msdn.microsoft.com/en-__us/library/wz223b1z.aspx
http://msdn.microsoft.com/en-us/library/wz223b1z.aspx

 Memory Limits for Windows and Windows Server Releases

http://msdn.microsoft.com/en-__us/library/windows/desktop/__aa366778(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx
 I learnt something new reading this article. For Win
8.1 64-bit, the
 /3GB switch is not applicable or required. However the
 /LARGEADDRESSAWARE  linker option

Re: [Pharo-dev] Memory limit problems on Windows VM

[Andres Valloud]

Does the VM make a single memory allocation to load the image file?

On 7/19/14 17:20 , Ben Coman wrote:

Andres Valloud wrote:

Keep in mind that, depending on how images are loaded, you won't be
able to load images larger than about 2gb... you might want to test that.


Can you hint what those scenarios are?
cheers -ben



On 7/19/14 13:39 , Tudor Girba wrote:

Wow! I will certainly try this patch.

Doru


On Fri, Jul 18, 2014 at 5:15 PM, Oscar E. A. Callaú
oscar.apo...@gmail.com mailto:oscar.apo...@gmail.com wrote:

Well,

   I fixed the problem by modifying the Pharo.exe file with the 4GB
patch program (http://ntcore.com/4gb_patch.php).

As mentioned before, I think the best solution is that Pharo vms
must be compiled with the flag LARGEADDRESSAWARE, so more memory
space is enabled for 64bits Windows OSs.

Thanks guys

Oscar


On Thu, Jul 17, 2014 at 11:03 PM, Andres Valloud
avall...@smalltalk.comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net wrote:

I worked on that switch before, and IIRC it's just a linker
switch that marks executables a certain way.  So, if you have
imagecfg.exe handy...

http://support.microsoft.com/__kb/297812
http://support.microsoft.com/kb/297812


On 7/17/14 19:25 , Oscar E. A. Callaú wrote:

Well. It seems that Pharo vms must be compiled with that
option :/

Cheers


On Wed, Jul 16, 2014 at 8:44 PM, b...@openinworld.com
mailto:b...@openinworld.com
mailto:b...@openinworld.com mailto:b...@openinworld.com
wrote:

 __
 Eliot Miranda wrote:




 On Wed, Jul 16, 2014 at 2:21 PM, Oscar E. A. Callaú
 oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com__ wrote:

 Hi,

 I'm using Windows 8.1 (update1) 64bits with 4GB
of physical
 RAM. I only need 1.5GB of RAM, so I think I
don't need to
 configure my OS to take /3GB of RAM.


 Well my experience (and others) is that one can't
grow the heap
 much above 1.1 Gb without using the /3Gb switch on
Windows XP.  I
 can't talk for 8.1.  But if 8.x also limits memory
to 2Gb/process
 unless the /3Gb switch is in effect then yes,
you'll need to use
 the /3Gb switch.

 HTH
 Eliot


 Cheers

 On Wednesday, July 16, 2014, Eliot Miranda
wrote:

 Hi Oscar,


 On Wed, Jul 16, 2014 at 9:11 AM, Oscar E.
A. Callaú
 oscar.apo...@gmail.com
mailto:oscar.apo...@gmail.com wrote:

 Hi guys,

I'm running some experiments in
Pharo. My data
 is aprox. 1.3GB. When I load my image
on Mac,
 everything works perfectly. But, when I
try to load
 the same image on Windows, I get this
error:

 Unable to commit memory (1326649344
bytes requested)

 I tried the Pharo VM, cogMT and  NBcog
with and
 without the option

 AddressSpaceLimit = 2048

 in the ini file

 Please help.

 What OS are you using?  Have you read e.g.
http://msdn.microsoft.com/en-__us/library/windows/hardware/__dn613959(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959(v=vs.85).aspx


http://msdn.microsoft.com/en-__us/library/windows/hardware/__dn613959%28v=vs.85%29.aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959%28v=vs.85%29.aspx?




 Here are some additional links...

 /LARGEADDRESSAWARE (Handle Large Addresses)
http://msdn.microsoft.com/en-__us/library/wz223b1z.aspx
http://msdn.microsoft.com/en-us/library/wz223b1z.aspx

 Memory Limits for Windows and Windows Server Releases
http://msdn.microsoft.com/en-__us/library/windows/desktop/__aa366778(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx

 I learnt something new reading this article. For Win
8.1 64-bit, the
 /3GB switch

Re: [Pharo-dev] Memory limit problems on Windows VM

[Andres Valloud]

I worked on that switch before, and IIRC it's just a linker switch that 
marks executables a certain way.  So, if you have imagecfg.exe handy...


http://support.microsoft.com/kb/297812

On 7/17/14 19:25 , Oscar E. A. Callaú wrote:

Well. It seems that Pharo vms must be compiled with that option :/

Cheers


On Wed, Jul 16, 2014 at 8:44 PM, b...@openinworld.com
mailto:b...@openinworld.com wrote:

__
Eliot Miranda wrote:




On Wed, Jul 16, 2014 at 2:21 PM, Oscar E. A. Callaú
oscar.apo...@gmail.com mailto:oscar.apo...@gmail.com wrote:

Hi,

I'm using Windows 8.1 (update1) 64bits with 4GB of physical
RAM. I only need 1.5GB of RAM, so I think I don't need to
configure my OS to take /3GB of RAM.


Well my experience (and others) is that one can't grow the heap
much above 1.1 Gb without using the /3Gb switch on Windows XP.  I
can't talk for 8.1.  But if 8.x also limits memory to 2Gb/process
unless the /3Gb switch is in effect then yes, you'll need to use
the /3Gb switch.

HTH
Eliot


Cheers

On Wednesday, July 16, 2014, Eliot Miranda wrote:

Hi Oscar,


On Wed, Jul 16, 2014 at 9:11 AM, Oscar E. A. Callaú
oscar.apo...@gmail.com wrote:

Hi guys,

   I'm running some experiments in Pharo. My data
is aprox. 1.3GB. When I load my image on Mac,
everything works perfectly. But, when I try to load
the same image on Windows, I get this error:

Unable to commit memory (1326649344 bytes requested)

I tried the Pharo VM, cogMT and  NBcog with and
without the option

AddressSpaceLimit = 2048

in the ini file

Please help.

What OS are you using?  Have you read e.g.

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959%28v=vs.85%29.aspx?




Here are some additional links...

/LARGEADDRESSAWARE (Handle Large Addresses)
http://msdn.microsoft.com/en-us/library/wz223b1z.aspx

Memory Limits for Windows and Windows Server Releases

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx
I learnt something new reading this article. For Win 8.1 64-bit, the
/3GB switch is not applicable or required. However the
/LARGEADDRESSAWARE  linker option is critical for both 32-bit and
64-bit VMs. Its only the default setting that changes between
compiling 32-bit or 64-bit applications. Some extracts...

Limits on memory and address space depend on whether the
IMAGE_FILE_LARGE_ADDRESS_AWARE value of the LOADED_IMAGE structure
and 4-gigabyte tuning (4GT) are in use.
IMAGE_FILE_LARGE_ADDRESS_AWARE is set or cleared by using the
/LARGEADDRESSAWARE linker option. 4-gigabyte tuning (4GT), also
known as application memory tuning, or the /3GB switch, is a
technology (**only applicable to 32 bit systems**) that alters the
amount of virtual address space available to user mode applications.

On x64-bit MS Windows, User-Mode virtual address space for each
32-bit process
4 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE set
2 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE cleared (default)

On x64-bit MS Windows, User-Mode virtual address space for each
64-bit process
8 TB With IMAGE_FILE_LARGE_ADDRESS_AWARE set (default):
2 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE cleared

cheers -ben



--
best,
Eliot




--
best,
Eliot

Re: [Pharo-dev] Memory limit problems on Windows VM

[Andres Valloud]

/LARGEADDRESSAWARE?... I don't think /3GB is even recognized in 64 bit 
Windows...


http://support.microsoft.com/default.aspx?scid=889654

On 7/16/14 14:49 , Eliot Miranda wrote:




On Wed, Jul 16, 2014 at 2:21 PM, Oscar E. A. Callaú
oscar.apo...@gmail.com mailto:oscar.apo...@gmail.com wrote:

Hi,

I'm using Windows 8.1 (update1) 64bits with 4GB of physical RAM. I
only need 1.5GB of RAM, so I think I don't need to configure my
OS to take /3GB of RAM.


Well my experience (and others) is that one can't grow the heap much
above 1.1 Gb without using the /3Gb switch on Windows XP.  I can't talk
for 8.1.  But if 8.x also limits memory to 2Gb/process unless the /3Gb
switch is in effect then yes, you'll need to use the /3Gb switch.

HTH
Eliot


Cheers

On Wednesday, July 16, 2014, Eliot Miranda wrote:

Hi Oscar,


On Wed, Jul 16, 2014 at 9:11 AM, Oscar E. A. Callaú
oscar.apo...@gmail.com wrote:

Hi guys,

I'm running some experiments in Pharo. My data is
aprox. 1.3GB. When I load my image on Mac, everything works
perfectly. But, when I try to load the same image on
Windows, I get this error:

Unable to commit memory (1326649344 bytes requested)

I tried the Pharo VM, cogMT and  NBcog with and without the
option

AddressSpaceLimit = 2048

in the ini file

Please help.

What OS are you using?  Have you read e.g.

http://msdn.microsoft.com/en-us/library/windows/hardware/dn613959(v=vs.85).aspx?


--
best,
Eliot




--
best,
Eliot

Re: [Pharo-dev] Division isn't correct

[Andres Valloud]

You say starting from a more consistent place, but how consistent are =
and = among Numbers without transitivity?


How are numbers consistent to begin with, when for all integers you have

x + 1 ~= x

but for floats, there are a multitude of values such that

x + 1.0 = x

holds?  Addition is hardly the only operation that exhibits this kind of 
behavior.  Or how about


x = x

being true for the vast majority of cases across all sorts of numbers, 
except when x is NaN?  Integers and fractions don't even have a notion 
of -0.0.  All of these are already pretty inconsistent without 
considering obvious issues such as


17 / 20 = 0.85

which in at least some Smalltalks evaluates to true even though it is 
mathematically impossible.


The point is that classes like Float, Integer and Fraction may very well 
have a common superclass.  Nonetheless, floating point numbers and 
effectively rationals obey very different rules.  Both have their strong 
points, those strengths are maximized with consistency.


Andres.



If we can maintain the invariant with a pair of double dispatching
methods and coordinated hash, why shouldn't we?
Why lispers did it? (Scheme too)

For me it's like saying: since float are inexact, we have a license to
waste ulp.
We have not. IEEE 754 model insists on operations to be exactly rounded.
These are painful contortions too, but most useful!
Extending the contortion to exact comparison sounds a natural extension
to me, the main difference is that we do not round true or false to the
nearest float, so it's even nicer!


On 7/11/14 17:19 , Nicolas Cellier wrote:




2014-07-12 1:29 GMT+02:00 Andres Valloud
avalloud@smalltalk.__comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net
mailto:avalloud@smalltalk.__comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net:


  I don't think it makes sense to compare floating point
 numbers to
  other types of numbers with #=... there's a world of
 approximations
  and other factors hiding behind #=, and the
occasional true
 answer
  confuses more than it helps.  On top of that, then
you get
 x = y =
  x hash = y hash, and so the hash of floating point
values
 has to
  be synchronized with integers, fractions, scaled
decimals,
 etc...
  _what a mess_...


 Yes, that's true, hash gets more complex.
 But then, this has been discussed before:

 {1/2  0.5. 1/2 = 0.5. 1/2  0.5} -  #(false false false).

 IOW, they are unordered.
 Are we ready to lose ordering of numbers?
 Practically, this would have big impacts on code base.


 IME, that's because loose code appears to work.  What
enables that
 loose code to work is the loose mixed mode arithmetic.  I could
 understand integers and fractions.  Adding floating point
to the mix
 stops making as much sense to me.

 Equality between floating point numbers does make sense.
  Equality
 between floating point numbers and scaled decimals or
fractions...
 in general, I don't see how they could make sense.  I'd
rather see
 the scaled decimals and fractions explicitly converted to
floating
 point numbers, following a well defined procedure, and then
compared...

 Andres.


Why do such mixed arithmetic comparisons make sense?
Maybe we used floating points in some low level Graphics for
optimization reasons.
After these optimized operations we get a Float result by
contagion, but
our intention is still to handle Numbers.
It would be possible to riddle the code with explicit
asFloat/asFraction
conversions, but that does not feel like a superior solution...

OK, we can as well convert to inexact first before comparing for
this
purpose.
That's what C does, because C is too low level to ever care of
transitivity and equivalence relationship.
It's not even safe in C because the compiler can decide to
promote to a
larger precision behind your back...
But let's ignore this feature and see what lispers recommend
instead:

http://www.lispworks.com/__documentation/lcl50/aug/aug-__170.html 
http://www.lispworks.com/documentation/lcl50/aug/aug-170.html

It says:

In general, when an operation involves both a rational and a
floating-point argument, the rational number is first converted to
floating-point format, and then the operation is performed

Re: [Pharo-dev] Division isn't correct

[Andres Valloud]

Being consistent then means either abandonning 1/2 = 0.5 but we saw this
has nasty side effects.


Well ok, so letting 1 / 2 = 0.5 answer false has nasty side effects. 
Why is that?  Isn't the code that breaks trying to tell us something? 
That's what I've been trying to point out.  Why rush to defend 
numerically unstable code?  What are we going to preserve, and what will 
be the example given to others?


Andres.


Or denying the generality of Dictionary: not all objects can be used as
key...
That's a possible choice, but IMO, this will generate bad feedback from
customers.
Speaking of consistency, I strongly believe that Squeak/Pharo are on the
right track.

OK, we can not magically erase inexactness of floating point operations.
They are on purpose for the sake of speed/memory footprint optimization.
Exceptional values like NaN and Inf are a great deal of
complexification, and I'd allways preferred exceptions to exceptional
values...
But when we can preserve some invariants, we'd better preserve them.
Once again, I did not invent anything, that's the approach of lispers,
and it seems wise.

And last thing, I like your argumentation, it's very logical, so if you
have more, you're welcome
but I pretty much exhausted mine ;)

Nicolas


If we can maintain the invariant with a pair of double dispatching
methods and coordinated hash, why shouldn't we?
Why lispers did it? (Scheme too)

For me it's like saying: since float are inexact, we have a
license to
waste ulp.
We have not. IEEE 754 model insists on operations to be exactly
rounded.
These are painful contortions too, but most useful!
Extending the contortion to exact comparison sounds a natural
extension
to me, the main difference is that we do not round true or false
to the
nearest float, so it's even nicer!


 On 7/11/14 17:19 , Nicolas Cellier wrote:




 2014-07-12 1:29 GMT+02:00 Andres Valloud
 avalloud@smalltalk.__comcastb__iz.net
http://comcastbiz.net
 mailto:avalloud@smalltalk.__comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net
 mailto:avalloud@smalltalk.
mailto:avalloud@smalltalk.__c__omcastbiz.net
http://comcastbiz.net

 mailto:avalloud@smalltalk.__comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net:


   I don't think it makes sense to compare
floating point
  numbers to
   other types of numbers with #=... there's
a world of
  approximations
   and other factors hiding behind #=, and the
 occasional true
  answer
   confuses more than it helps.  On top of
that, then
 you get
  x = y =
   x hash = y hash, and so the hash of
floating point
 values
  has to
   be synchronized with integers, fractions,
scaled
 decimals,
  etc...
   _what a mess_...


  Yes, that's true, hash gets more complex.
  But then, this has been discussed before:

  {1/2  0.5. 1/2 = 0.5. 1/2  0.5} -  #(false
false false).

  IOW, they are unordered.
  Are we ready to lose ordering of numbers?
  Practically, this would have big impacts on
code base.


  IME, that's because loose code appears to work.  What
 enables that
  loose code to work is the loose mixed mode
arithmetic.  I could
  understand integers and fractions.  Adding
floating point
 to the mix
  stops making as much sense to me.

  Equality between floating point numbers does make
sense.
   Equality
  between floating point numbers and scaled decimals or
 fractions...
  in general, I don't see how they could make sense.
  I'd
 rather see
  the scaled decimals and fractions explicitly
converted to
 floating
  point numbers, following a well defined procedure,
and then
 compared...

  Andres.


 Why do such mixed arithmetic comparisons make sense?
 Maybe we used floating points in some low level
Graphics for
 optimization reasons.
 After

Re: [Pharo-dev] Division isn't correct

[Andres Valloud]

I don't think it makes sense to compare floating point numbers to other 
types of numbers with #=... there's a world of approximations and other 
factors hiding behind #=, and the occasional true answer confuses more 
than it helps.  On top of that, then you get x = y = x hash = y hash, 
and so the hash of floating point values has to be synchronized with 
integers, fractions, scaled decimals, etc... _what a mess_...


On 7/11/14 10:46 , stepharo wrote:

I suggest you to read the Small number chapter of the Deep into Pharo.

Stef

On 11/7/14 15:53, Natalia Tymchuk wrote:

Hello.
 I found interesting thing:
Why it is like this?

Best regards,
Natalia

Re: [Pharo-dev] Division isn't correct

[Andres Valloud]

I don't think it makes sense to compare floating point numbers to
other types of numbers with #=... there's a world of approximations
and other factors hiding behind #=, and the occasional true answer
confuses more than it helps.  On top of that, then you get x = y =
x hash = y hash, and so the hash of floating point values has to
be synchronized with integers, fractions, scaled decimals, etc...
_what a mess_...


Yes, that's true, hash gets more complex.
But then, this has been discussed before:

{1/2  0.5. 1/2 = 0.5. 1/2  0.5} -  #(false false false).

IOW, they are unordered.
Are we ready to lose ordering of numbers?
Practically, this would have big impacts on code base.


IME, that's because loose code appears to work.  What enables that loose 
code to work is the loose mixed mode arithmetic.  I could understand 
integers and fractions.  Adding floating point to the mix stops making 
as much sense to me.


Equality between floating point numbers does make sense.  Equality 
between floating point numbers and scaled decimals or fractions... in 
general, I don't see how they could make sense.  I'd rather see the 
scaled decimals and fractions explicitly converted to floating point 
numbers, following a well defined procedure, and then compared...


Andres.


I'm pretty sure a Squeak/Pharo image wouldn't survive that long to such
a change
(well I tried it, the Pharo3.0 image survives, but Graphics are badly
broken as I expected).

That's allways what made me favour casual equality to universal inequality.

Also should 0.1 = 0.1 ? In case those two floats have been produced by
different path, different approximations they might not be equal...
(| a b | a := 0.1. b := 1.0e-20. a+b=a.)
I also prefer casual equality there too.
Those two mathematical expressions a+b and a are different but both
floating point expressions share same floating point approximation,
that's all what really counts because in the end, we cannot distinguish
an exact from an inexact Float, nor two inexact Float.
We lost the history...

Also, the inexact flag is not attached to a Float, it's only the result
of an operation.
Statistically, it would waste one bit for nothing, most floats are the
result of an inexact operation.
But who knows, both might be the result of exact operations too ;)



On 7/11/14 10:46 , stepharo wrote:

I suggest you to read the Small number chapter of the Deep into
Pharo.

Stef

On 11/7/14 15:53, Natalia Tymchuk wrote:

Hello.
  I found interesting thing:
Why it is like this?

Best regards,
Natalia

Re: [Pharo-dev] about ~=

[Andres Valloud]

Sp, speaking of mathematical precedence and all that, anyone up to 
looking at the IEEE-754 spec and making one's Smalltalk of choice comply 
with _all_ the requirements?  Or how about implementing decimal floats?


On 6/26/14 12:01 , p...@highoctane.be wrote:


Couldn't we just have an expression evaluator done with PetitParser
doing these things?

Tcl and Bash have this expr: thing.


(Expression parse: '5+3*2') value.
(Expression parse: ':a | 5+:a*2') value:10.
(Expression parse: ':a | 5+:a*2') value:self x.

With the thesis of Lukas Renggli these was this Helvetia story.
Is there any chance we see something around these lines now that there
is a new compiler thing inside Pharo?

So:

someMethod: anX
expr
^ 5 + anX * 2

Phil





On Thu, Jun 26, 2014 at 6:11 PM, Richard Sargent
richard.sarg...@gemtalksystems.com
mailto:richard.sarg...@gemtalksystems.com wrote:

Esteban A. Maringolo wrote
  If one thing confuses people in that realm is non arithmetic
precedence:
  Eg. 2 + 3 * 4 = 20 instead of the expected 14.
 
  And we're not going to change that either. It's not worthy, and I
  doubt if it is possible at all.

I'm probably late to the party with this reply. The primary reason
for not
changing this is that it would be incorrect. It comes down to intrinsic
versus extrinsic meaning. A multiple operator has an intrinsic
meaning: it
means multiple. But a message name does not have intrinsic meaning
(other
than it is a good idea to have the name represent what it does). The
meaning
of a message is determined by the receiver. e.g. if PetitParser
defines #*
to mean 0 or more repetitions, what happens when someone has
decided that
it should be evaluated before #+?

Message sending precedence can only be defined in terms of the type of
message: unary, binary (or infix), and keyword, since the
interpretation of
the message is the receiver's responsibility, not the compiler's.




--
View this message in context:
http://forum.world.st/about-tp4764892p4765070.html
Sent from the Pharo Smalltalk Developers mailing list archive at
Nabble.com.

Re: [Pharo-dev] [squeak-dev] Re: String #=

[Andres Valloud]

Hey Philippe,


Yes but #= is blissfully unaware of normalization in Squeak/Pharo. In
fact AFAIK Squeak/Pharo is unaware of normalization. Having a short look
at it doesn't even look as if case insensitivity worked in Squeak/Pharo
outside of Latin-1 (I could be wrong though).


Yes, that's what I am thinking about.  To be more explicit, suppose 
Unicode series of characters got into the image via the keyboard, a 
file, a socket... once decoded, what could one do with them?  Are all 
types of decoded character series going to be represented as instances 
of a single class, although they have inherently different behavior?



In addition you probably don't want #= to do normalization because
performance. And even if you did you probably still want a fast path
for ByteString receiver and ByteString argument in which case #size is safe.


Assuming all fixed width representation strings (e.g. byte strings) will 
always have the same encoding (e.g. same code page), then the size check 
for those seems ok to me.


Just to make sure, I am not celebrating all this complexity in the 
world... however, given that it's there, how are we going to deal with 
it?  I'm concerned about the long term consequences of making things 
more complex than they are by reinterpreting them.  The problem I see is 
that ultimately programs just won't Work(TM).


Andres.

Re: [Pharo-dev] [squeak-dev] Re: String #=

[Andres Valloud]

What is going to happen when one compares two general Unicode series of 
characters that represent the same string but differ in normalization? 
Wouldn't the size test would result in false negatives?


http://unicode.org/reports/tr15/

I'm asking because I haven't seen any discussion on the subject, and the 
decision to change the code as proposed could have side effects.


On 5/27/14 11:59 , Eliot Miranda wrote:




On Tue, May 27, 2014 at 6:54 AM, J. Vuletich (mail lists)
juanli...@jvuletich.org mailto:juanli...@jvuletich.org wrote:

__

Quoting Eliot Miranda eliot.mira...@gmail.com
mailto:eliot.mira...@gmail.com:


Hi Phillipe,


On Mon, May 26, 2014 at 12:51 AM, Philippe Marschall
philippe.marsch...@netcetera.ch
mailto:philippe.marsch...@netcetera.ch wrote:

Hi

I have been investigating why Dictionary look up performance
with String keys is not as good as I would expected. Something
I noted is that String  #= is implemented in terms of
#compare:with:collated:. There is no short circuit if Strings
are not the same size. In my case some Strings have the same
prefix but a different length eg 'Content-Type' and
'Content-Length'. In that case a #compare:with:collated: is
performed even though we know in advance the answer will be
false because they have different sizes.

Why not rewrite
String= aString
Answer whether the receiver sorts equally as aString.
The collation order is simple ascii (with case differences).
aString isString ifFalse: [ ^ false ].
^ (self compare: self with: aString collated: AsciiOrder) = 2
as
String= aString
Answer whether the receiver sorts equally as aString.
The collation order is simple ascii (with case differences).
(aString isString
and: [self size = aString size]) ifFalse: [^false].
^ (self compare: self withSize: with: aString collated:
AsciiOrder) = 2
?



This makes a huge difference, over 3 times faster:

| bs t1 t2 |
bs := ByteString allInstances first: 1.
t1 := [bs do: [:a| bs do: [:b| a = b]]] timeToRun.
(FileStream fileNamed: '/Users/eliot/Squeak/Squeak4.5/String-=.st') fileIn.
t2 := [bs do: [:a| bs do: [:b| a = b]]] timeToRun.
{ t1. t2 } #(13726 4467)
4467 - 13726 / 137.26 -67.46%


One /could/ add a replacement compare:with:collated:
primitive primitiveCompareString which took the sizes as arguments
to avoid asking twice.  But it wouldn't be safe.  One could abuse
the primitive and lie about the size.  So I suspect it is best to
add the size check to String#= and accept the duplication of
the primitive finding the sizes of the two strings.  The cost in
the primitive is minimal.  A WideString version of the primitive
might pay its way, but if Spur and Sista arrive soon the primitive
shouldn't be faster than the optimised Smalltalk code.
--
best,
Eliot


BTW, any good reason for not prefixing all the implementors of #=
with this?

Any object is equal to itself
self == argument ifTrue: [ ^ true ].


It doesn't make much difference:

| bs t1 t2 |
bs := ByteString allInstances first: 1.
t1 := [bs do: [:a| bs do: [:b| a = b]]] timeToRun.
(FileStream fileNamed: '/Users/eliot/Squeak/Squeak4.5/String-=.st') fileIn.
t2 := [bs do: [:a| bs do: [:b| a = b]]] timeToRun.
{ t1. t2 } #(4628 4560)

4560 - 4628 / 46.28 -1.47%

So is it worth it?  If you feel it is I've no objection other than it
feels a little kludgey for such little benefit.  And there are the
Symbols if one needs quick comparison and can bear the cost of slow
interning.
--
best,
Eliot

Re: [Pharo-dev] 32 bits and 64 bits VM

[Andres Valloud]

* 64-bit aware hashing functions as the current ones are 32-bit centric.


This issue might not be the most urgent because the 32 bit hash 
functions should be sufficient to deal with at least moderately sized 64 
bit images.  But how much stuff would you have to allocate before seeing 
a problem?


Suppose object headers are 16 bytes and your hash functions are 
producing 28 bits' worth of data.  Clearly hashing 2^28 objects with 
zero instance variables and zero byte data is kind of silly, so we put 
in at least one instance variable making those objects at least 24 
bytes.  Then you also need a hashed collection with a single table large 
enough to hold all those objects (ignoring e.g. hash buckets), the load 
factor of which will be kept at 80%.  So...


24 * 2^28 * 4 // 5 + (2^28 * 8) = about 7gb

So, the current hash functions should be fine for images up to 7gb in 
size, and probably much larger than that because 7gb is the rock bottom 
low estimate.  Probably one might hash strings or other objects with 
more data than 8 bytes.  In that case, the low bound in that case will 
be higher.


Andres.

Re: [Pharo-dev] [Inspiration] Toward a better programming

[Andres Valloud]

Hey Doru, I was going to ask so is that a language or a library?...

On 4/1/14 23:12 , Tudor Girba wrote:

Indeed, I read this article several times over the last couple of days.
This work is impressive particularly when combined with the cloud part.

The language itself is less interesting for me, but what makes it stand
out is that it has a coherent and robust philosophy behind and
phenomenal goals to reach. In Pharo, we have the luxury of building on
top of coherent and robust philosophy (even if different from the
Wolfram one) and we should try as much as possible to keep our eyes on
phenomenal goals that seem unreachable.

Another thing I like in Wolfram's work is attention to details:
http://blog.wolfram.com/2008/01/10/ten-thousand-hours-of-design-reviews/

Details are crucial, and all the effort in Pharo around naming and
redesigning what already exists is incredibly important. But, it is
precisely at the moment when we are knee-deep in details that is crucial
to keep our eyes on the phenomenal long term goals.

There is so much to build. Let's be bold.

Doru



On Tue, Apr 1, 2014 at 7:22 PM, Sven Van Caekenberghe s...@stfx.eu
mailto:s...@stfx.eu wrote:


On 31 Mar 2014, at 06:21, S Krish krishnamachari.sudha...@gmail.com
mailto:krishnamachari.sudha...@gmail.com wrote:

  How about impact of this:
 
 

http://blog.stephenwolfram.com/2014/03/injecting-computation-everywhere-a-sxsw-update/
 
  I would agree it is quite complex for any beginner, but utility
of a programming language on these lines seems cut out for the future..

Wow, this is really powerful stuff, a long read, but well worth it.
By recombining and reusing all their technology they seem to be able
to move into more and more territory.

It is closed source and (very) expensive, and I don't like the
syntax, but we sure can get good ideas from them.

Thanks for sharing the link.

Sven





--
www.tudorgirba.com http://www.tudorgirba.com

Every thing has its own flow

Re: [Pharo-dev] [Bug] IdentitySetsize

[Andres Valloud]

So it seems the problem is with Fuel rather than IdentitySet, no?

On 3/2/14 10:47 , Max Leske wrote:

During serialization the IdentitySet size is stored and later its objects. 
During that step, #do:

Re: [Pharo-dev] [Bug] IdentitySetsize

[Andres Valloud]

So, just out of curiosity, how does the IdentitySet get damaged?

On 3/2/14 11:28 , Max Leske wrote:


On 02.03.2014, at 20:12, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


So it seems the problem is with Fuel rather than IdentitySet, no?


No. Asking an IdentitySet for its size is not reliable. That has nothing to do 
with Fuel. #size is especially important in hashed collections where the size 
is stored in a variable and thus allows fast access the size without the need 
to visit every element in the collection for its computation.



On 3/2/14 10:47 , Max Leske wrote:

During serialization the IdentitySet size is stored and later its objects. 
During that step, #do:

Re: [Pharo-dev] hashMultiply

[Andres Valloud]

Once the code is loaded, from the Tools menu use Hash Analysis Tool. 
There's a manual below, and also the Fundamentals book has a somewhat in 
depth discussion on how it works.


ftp://sqrmax.us.to/pub/Smalltalk/Papers/Hash%20Analysis%20Tool.pdf

On 3/2/14 15:06 , p...@highoctane.be wrote:

I have found the tools in the Cincom public repo.

Now, I have to see how this works in VW, I am not that proficient with it.

Phil



On Wed, Feb 26, 2014 at 8:55 AM, p...@highoctane.be
mailto:p...@highoctane.be p...@highoctane.be
mailto:p...@highoctane.be wrote:

Andres,

Thanks for the insights.

hash quality is indeed an key factor. At least, my mind is somewhat
grasping this hashing field a bit better.

I'll have a look at the tools, I haven't used them yet.

And a shot at the ASM version with NativeBoost in Pharo.

For what occurs in modern CPUs, well, no. Surprising to see that a
mul would be faster than a shr or shl. How comes?
I used to be ok with these things when I was writing demoscene code
a loong time ago but I'd need an extremely serious refresh.

As a side note, there is a huge uptake in the
BigData/mapreduce/hadoop environment where Smalltalk is sorely
absent. Scala seems to fill the void on the JVM.
There hashing is quite key, to remap all of the mapping phase
results to the reduce nodes. I am surprised to see that Smalltalk
vendors haven't jumped in that space.

Phil


On Wed, Feb 26, 2014 at 2:04 AM, Andres Valloud
avall...@smalltalk.comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net wrote:

Hello...

On 2/25/14 1:17 , p...@highoctane.be mailto:p...@highoctane.be
wrote:

I am currently reading through the Hashing in Smalltalk book

(http://www.lulu.com/shop/__andres-valloud/hashing-in-__smalltalk-theory-and-practice/__paperback/product-3788892.html

http://www.lulu.com/shop/andres-valloud/hashing-in-smalltalk-theory-and-practice/paperback/product-3788892.html__)
and, my head hurting notwithstanding, there are indeed a ton
of gems in
this system. As he mentions, doing the exercises brings a
lot of extra :-)


:) thank you.

When going to 64-bit, and with the new ObjectMemory scheme,
I guess a
couple of identity hashing functions will come under scrutiny.

e.g.

SmallIntegerhashMultiply
| low |

low := self bitAnd: 16383.
^(16r260D * low + ((16r260D * (self bitShift: -14) +
(16r0065 * low)
bitAnd: 16383) * 16384))
bitAnd: 16r0FFF


which will need some more bits.


IMO it's not clear that SmallIntegeridentityHash should be
implemented that way.  Finding a permutation of the small
integers that also behaves like a good quality hash function and
evaluates quickly (in significantly less time and complexity
than, say, Bob Jenkins' lookup3) would be a really interesting
research project.  I don't know if it's possible.  If no such
thing exists, then getting at least some rough idea of what's
the minimum necessary complexity for such hash functions would
be valuable.

Looking at hashMultiply as a non-identity hash function, one
would start having problems when significantly more than 2^28
objects are stored in a single hashed collection.  2^28 objects
with e.g. 12 bytes per header and a minimum of one instance
variable (so the hash value isn't a instance-constant) stored in
a hashed collection requires more than 4gb, so that is clearly a
64 bit image problem.  In 64 bits, 2^28 objects with e.g. 16
bytes per header and a minimum of one instance variable each is
already 6gb, and a minimum of 8gb with the hashed collection itself.

Because of these figures, I'd think improving the implementation
of hashed collections takes priority over adding more
non-identity hash function bits (as long as the existing hash
values are of good quality).

Did you look at the Hash Analysis Tool I wrote?  It's in the
Cincom public Store repository.  It comes in two bundles: Hash
Analysis Tool, and Hash Analysis Tool - Extensions.  With
everything loaded, the tool comes with 300+ hash functions and
100+ data sets out of the box.  The code is MIT.

I had a look at how it was done in VisualWorks;


The implementation of hashMultiply, yes.  Note however that
SmallIntegerhash is ^self.

hashMultiply
Multiply the receiver by 16r0019660D mod 2^28
without using large integer arithmetic for speed.
The constant is a generator of the multiplicative
subgroup of Z_2^30, see Knuth's TAOCP vol 2

Re: [Pharo-dev] hashMultiply

[Andres Valloud]

Hello...

On 2/25/14 18:32 , Martin McClure wrote:


Andres and I have an ongoing discussion on this topic. :-)


:)...


And #bitShift *should* be faster.


Ok, but how much faster?  64x64 bit multiplication on modern x86 is just 
8 cycles (!), and there will likely be parallelism done by the CPU anyway...



But test this. In VW, multiplication
is quite a lot faster than bitShift: for some reason. Unless Andres has
fixed this recently. :-)


I haven't measured to see what's going on.  In general terms though, 
unlike with multiplication, on top of the overflow check you also have 
the argument overflow check because on x86


shl eax, 65

is the same as

shl eax, 1

which is obviously not the behavior you want.  So that means extra cmp, 
extra jmp, and you also have to deal with positive / negative arguments 
to select shl or shr/sar so more cmp and jmp, etc...


Note though that IIRC the manual says that for SSE registers, doing 
something like


shl xmm0, 129

is the same as

xor xmm0, xmm0

(assembler for the sake of illustratio only) which is nice because then 
you can combine the overflow check with the argument overflow check.


The whole SSE thing though is much, much more than just bitShift.

Andres.

Re: [Pharo-dev] hashMultiply

[Andres Valloud]

Hello again...


For what occurs in modern CPUs, well, no. Surprising to see that a mul
would be faster than a shr or shl. How comes?


No, not faster.  But definitely way faster than 30 cycles not so long ago.


I used to be ok with these things when I was writing demoscene code a
loong time ago but I'd need an extremely serious refresh.


You said demoscene?!... do you have a handle???

Andres.


As a side note, there is a huge uptake in the BigData/mapreduce/hadoop
environment where Smalltalk is sorely absent. Scala seems to fill the
void on the JVM.
There hashing is quite key, to remap all of the mapping phase results to
the reduce nodes. I am surprised to see that Smalltalk vendors haven't
jumped in that space.

Phil


On Wed, Feb 26, 2014 at 2:04 AM, Andres Valloud
avall...@smalltalk.comcastbiz.net
mailto:avall...@smalltalk.comcastbiz.net wrote:

Hello...

On 2/25/14 1:17 , p...@highoctane.be mailto:p...@highoctane.be wrote:

I am currently reading through the Hashing in Smalltalk book

(http://www.lulu.com/shop/__andres-valloud/hashing-in-__smalltalk-theory-and-practice/__paperback/product-3788892.html

http://www.lulu.com/shop/andres-valloud/hashing-in-smalltalk-theory-and-practice/paperback/product-3788892.html__)
and, my head hurting notwithstanding, there are indeed a ton of
gems in
this system. As he mentions, doing the exercises brings a lot of
extra :-)


:) thank you.

When going to 64-bit, and with the new ObjectMemory scheme, I
guess a
couple of identity hashing functions will come under scrutiny.

e.g.

SmallIntegerhashMultiply
| low |

low := self bitAnd: 16383.
^(16r260D * low + ((16r260D * (self bitShift: -14) + (16r0065 * low)
bitAnd: 16383) * 16384))
bitAnd: 16r0FFF


which will need some more bits.


IMO it's not clear that SmallIntegeridentityHash should be
implemented that way.  Finding a permutation of the small integers
that also behaves like a good quality hash function and evaluates
quickly (in significantly less time and complexity than, say, Bob
Jenkins' lookup3) would be a really interesting research project.  I
don't know if it's possible.  If no such thing exists, then getting
at least some rough idea of what's the minimum necessary complexity
for such hash functions would be valuable.

Looking at hashMultiply as a non-identity hash function, one would
start having problems when significantly more than 2^28 objects are
stored in a single hashed collection.  2^28 objects with e.g. 12
bytes per header and a minimum of one instance variable (so the hash
value isn't a instance-constant) stored in a hashed collection
requires more than 4gb, so that is clearly a 64 bit image problem.
  In 64 bits, 2^28 objects with e.g. 16 bytes per header and a
minimum of one instance variable each is already 6gb, and a minimum
of 8gb with the hashed collection itself.

Because of these figures, I'd think improving the implementation of
hashed collections takes priority over adding more non-identity hash
function bits (as long as the existing hash values are of good quality).

Did you look at the Hash Analysis Tool I wrote?  It's in the Cincom
public Store repository.  It comes in two bundles: Hash Analysis
Tool, and Hash Analysis Tool - Extensions.  With everything loaded,
the tool comes with 300+ hash functions and 100+ data sets out of
the box.  The code is MIT.

I had a look at how it was done in VisualWorks;


The implementation of hashMultiply, yes.  Note however that
SmallIntegerhash is ^self.

hashMultiply
Multiply the receiver by 16r0019660D mod 2^28
without using large integer arithmetic for speed.
The constant is a generator of the multiplicative
subgroup of Z_2^30, see Knuth's TAOCP vol 2.
primitive: 1747
| low14Bits |
low14Bits := self bitAnd: 16r3FFF.
^16384
* (16r260D * (self bitShift: -14) + (16r0065 * low14Bits)
bitAnd: 16r3FFF)
+ (16r260D * low14Bits) bitAnd: 16rFFF

The hashing book version has:

multiplication
Computes self times 1664525 mod 2^38 while avoiding overflow into a
large integer by making the multiplication into two 14 bits
chunks. Do
not use any division or modulo operation.
| lowBits highBits|

lowBits := self bitAnd: 16r3FFF.
highBits := self bitShift: -14.
^(lowBits * 16r260D)
+ (((lowBits * 16r0065) bitAnd: 16r3FFF) bitShift: 14)
+ (((highBits * 16r260D) bitAnd: 16r3FFF) bitShift: 14)
bitAnd: 16rFFF

So, 16384 * is the same as bitShift: 14 and it looks like done once,
which may be better.


It should be a primitive (or otherwise optimized somehow

Re: [Pharo-dev] hashMultiply

[Andres Valloud]

Hello...

On 2/25/14 1:17 , p...@highoctane.be wrote:

I am currently reading through the Hashing in Smalltalk book
(http://www.lulu.com/shop/andres-valloud/hashing-in-smalltalk-theory-and-practice/paperback/product-3788892.html)
and, my head hurting notwithstanding, there are indeed a ton of gems in
this system. As he mentions, doing the exercises brings a lot of extra :-)


:) thank you.


When going to 64-bit, and with the new ObjectMemory scheme, I guess a
couple of identity hashing functions will come under scrutiny.

e.g.

SmallIntegerhashMultiply
| low |

low := self bitAnd: 16383.
^(16r260D * low + ((16r260D * (self bitShift: -14) + (16r0065 * low)
bitAnd: 16383) * 16384))
bitAnd: 16r0FFF


which will need some more bits.


IMO it's not clear that SmallIntegeridentityHash should be implemented 
that way.  Finding a permutation of the small integers that also behaves 
like a good quality hash function and evaluates quickly (in 
significantly less time and complexity than, say, Bob Jenkins' lookup3) 
would be a really interesting research project.  I don't know if it's 
possible.  If no such thing exists, then getting at least some rough 
idea of what's the minimum necessary complexity for such hash functions 
would be valuable.


Looking at hashMultiply as a non-identity hash function, one would start 
having problems when significantly more than 2^28 objects are stored in 
a single hashed collection.  2^28 objects with e.g. 12 bytes per header 
and a minimum of one instance variable (so the hash value isn't a 
instance-constant) stored in a hashed collection requires more than 4gb, 
so that is clearly a 64 bit image problem.  In 64 bits, 2^28 objects 
with e.g. 16 bytes per header and a minimum of one instance variable 
each is already 6gb, and a minimum of 8gb with the hashed collection itself.


Because of these figures, I'd think improving the implementation of 
hashed collections takes priority over adding more non-identity hash 
function bits (as long as the existing hash values are of good quality).


Did you look at the Hash Analysis Tool I wrote?  It's in the Cincom 
public Store repository.  It comes in two bundles: Hash Analysis Tool, 
and Hash Analysis Tool - Extensions.  With everything loaded, the tool 
comes with 300+ hash functions and 100+ data sets out of the box.  The 
code is MIT.



I had a look at how it was done in VisualWorks;


The implementation of hashMultiply, yes.  Note however that 
SmallIntegerhash is ^self.



hashMultiply
Multiply the receiver by 16r0019660D mod 2^28
without using large integer arithmetic for speed.
The constant is a generator of the multiplicative
subgroup of Z_2^30, see Knuth's TAOCP vol 2.
primitive: 1747
| low14Bits |
low14Bits := self bitAnd: 16r3FFF.
^16384
* (16r260D * (self bitShift: -14) + (16r0065 * low14Bits) bitAnd: 16r3FFF)
+ (16r260D * low14Bits) bitAnd: 16rFFF

The hashing book version has:

multiplication
Computes self times 1664525 mod 2^38 while avoiding overflow into a
large integer by making the multiplication into two 14 bits chunks. Do
not use any division or modulo operation.
| lowBits highBits|

lowBits := self bitAnd: 16r3FFF.
highBits := self bitShift: -14.
^(lowBits * 16r260D)
   + (((lowBits * 16r0065) bitAnd: 16r3FFF) bitShift: 14)
   + (((highBits * 16r260D) bitAnd: 16r3FFF) bitShift: 14)
bitAnd: 16rFFF

So, 16384 * is the same as bitShift: 14 and it looks like done once,
which may be better.


It should be a primitive (or otherwise optimized somehow).  At some 
point though that hash function was implemented for e.g. ByteArray in 
Squeak, I thought at that point the multiplication step was also 
implemented as a primitive?



Also VW marks it as a primitive, which Pharo does not.


In VW it is also a translated primitive, i.e. it's executed directly in 
the JIT without calling C.


Keep in mind that the speed at which hash values are calculated is only 
part of the story.  If the hash function quality is not great, or the 
hashed collection implementation is not efficient and induces collisions 
or other extra work, improving the efficiency of the hash functions 
won't do much.  I think it's mentioned in the hash book (I'd have to 
check), but once I made a hash function 5x times slower to get better 
quality and the result was that a report that was taking 30 minutes took 
90 seconds instead (and hashing was gone from the profiler output).



Would we gain
some speed doing that? hashMultiply is used a lof for identity hashes.

Bytecode has quite some work to do:

37 70 self
38 20 pushConstant: 16383
39 BE send: bitAnd:
40 68 popIntoTemp: 0
41 21 pushConstant: 9741
42 10 pushTemp: 0
43 B8 send: *
44 21 pushConstant: 9741
45 70 self
46 22 pushConstant: -14
47 BC send: bitShift:
48 B8 send: *
49 23 pushConstant: 101
50 10 pushTemp: 0
51 B8 send: *
52 B0 send: +
53 20 pushConstant: 16383
54 BE send: bitAnd:
55 24 pushConstant: 16384
56 B8 send: *
57 B0 send: +
58 25 pushConstant: 268435455
59 BE send: bitAnd

Re: [Pharo-dev] [GSoC ideas] Pharo - Derreferencer

[Andres Valloud]

On 2/16/14 3:38 , Levente Uzonyi wrote:

On Sat, 15 Feb 2014, Alejandro Infante wrote:


Well, I tried today using that to release some memory used by
Roassal. Let's say all the instances of ROElement. So I executed
the following code:

ROElement allInstances do: [:el | el becomeForward: nil ].

The next second the image crashed. Then I thought that was because
the visualization was open. So I did another little experiment,
closed the visualization, and then run the following code:

ROElement allInstances do: [:el | el becomeForward: nil ].
Everithing ok for now Smalltalk garbageCollect.

Again, same result, the image crashed. Also instead of doing
garbage collect, I tried open the browser, but the image crashed
just few seconds later.


It's a sign of a bug either in the VM, or the finalization mechanism
(if these objects are subject to finalization), or Roassal itself.
You better examine the crash dumps.


Yeah, who knows... maybe ROElement is dealing with FFI...

It's way better to figure out why you have objects that won't go away, 
then you fix the real problem.  Hopefully the scope for this GSoC 
project will be amended with those kinds of considerations in mind.


Andres.

Re: [Pharo-dev] [GSoC ideas] Pharo - Derreferencer

[Andres Valloud]

FYI writing any decent production quality tool takes way more than 1 
hour.  IME writing *two* reference finders, it takes a bunch of effort 
to make them work correctly, reliably, quickly, and in a useful way.


On 2/15/14 6:31 , Alexandre Bergel wrote:

Well... I do no think so. There is a fair amount of work regarding memory 
profiling. Dereferencing is probably only a small piece off.
Anyway, we will have time to refine the project description

Alexandre


Le 14-02-2014 à 18:17, Levente Uzonyi le...@elte.hu a écrit :


On Fri, 14 Feb 2014, Alexandre Bergel wrote:

Hi Paolo,

I can mentor the project below:

Project idea

Name: Instance Derreferencer for Pharo
Skill level: Intermediate
Possible Mentors:
Name of the Student: Alejandro Infante

Description:
A problem with image based environment is, when you detect a memory leak is 
removing those objects from the system. This tool would provide a handy UI to 
inspect the current instances of the classes that we think they have a memory 
leak and, in case there are objects that are not being garbage collected, offer 
to the user a Derreference button. This is going to iterate over all the 
objects that have a reference to my object and remove that reference, finally 
enabling the Garbage Collector to get rid of the leak.


It shouldn't take more than an hour to implement such tool.


Levente



Cheers,
Alexandre






2014-02-14 5:08 GMT-08:00 Nicolas Petton petton.nico...@gmail.com:

Paolo Bonzini writes:


Il 11/02/2014 10:42, Damien Cassou ha scritto:

Hi fellow Pharo hackers,

ESUG, the European Smalltalk User Group, is applying for this
year's Google Summer of Code.  As you probably know, the Summer
of Code provides the opportunity to fund students to work during
the summer on Pharo.  Please reply to this
email (be sure to use Reply to all) if you have ideas you
would like to propose.

Please include a summary of the project and links to web pages
that can help prospective students to write their application.
Please also include the following information:

- if applicable, other dialects that you would be willing to
  mentor this project for

- the skill level

- name of the mentor(s), email addresses, and possibly any IRC
  network/channel/nickname where they can be found.

Thanks for contributing to ESUG's Summer of Code application!


Just a note that I haven't yet processed this thread, but our ideas page
is already looking pretty well (and Pharo section is big) so I'll
probably do it in the weekend or next Monday.


I can do it if you want.

Cheers,
Nico


--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Re: [Pharo-dev] [GSoC ideas] Pharo - Derreferencer

[Andres Valloud]

All that I said was it takes way more than 1 hour.

The useful part comes in when e.g. inboundPointers reveals stuff 
that should also be gone.  So now you need a full blown reference 
finder, and getting one of those to work quickly and correctly is not 
trivial.  Also with some objects you can't really do becomeForward: (or 
it won't help), e.g. with a class (this would fall under the reliably 
category).


We are all capable of writing random workspace code, I don't think this 
is the idea for these projects.


On 2/15/14 16:34 , Levente Uzonyi wrote:

On Sat, 15 Feb 2014, Andres Valloud wrote:


FYI writing any decent production quality tool takes way more than 1 hour.
IME writing *two* reference finders, it takes a bunch of effort to make them
work correctly, reliably, quickly, and in a useful way.


I would agree with you if the building blocks weren't already in the
image.

Q: Which objects point to myObject?
A: myObject inboundPointers

Q: How can I dereference myObject?
A: myObject becomeForward: nil

Do you think it takes a summer to combine and extend these two lines into
a tool?


Levente



On 2/15/14 6:31 , Alexandre Bergel wrote:

Well... I do no think so. There is a fair amount of work regarding memory
profiling. Dereferencing is probably only a small piece off.
Anyway, we will have time to refine the project description

Alexandre


Le 14-02-2014 à 18:17, Levente Uzonyi le...@elte.hu a écrit :


On Fri, 14 Feb 2014, Alexandre Bergel wrote:

Hi Paolo,

I can mentor the project below:

Project idea

Name: Instance Derreferencer for Pharo
Skill level: Intermediate
Possible Mentors:
Name of the Student: Alejandro Infante

Description:
A problem with image based environment is, when you detect a memory leak
is removing those objects from the system. This tool would provide a
handy UI to inspect the current instances of the classes that we think
they have a memory leak and, in case there are objects that are not
being garbage collected, offer to the user a Derreference button. This
is going to iterate over all the objects that have a reference to my
object and remove that reference, finally enabling the Garbage Collector
to get rid of the leak.


It shouldn't take more than an hour to implement such tool.


Levente



Cheers,
Alexandre






2014-02-14 5:08 GMT-08:00 Nicolas Petton petton.nico...@gmail.com:

Paolo Bonzini writes:


Il 11/02/2014 10:42, Damien Cassou ha scritto:

Hi fellow Pharo hackers,

 ESUG, the European Smalltalk User Group, is applying for this
 year's Google Summer of Code.  As you probably know, the
Summer
 of Code provides the opportunity to fund students to work
during
 the summer on Pharo.  Please reply to this
 email (be sure to use Reply to all) if you have ideas you
 would like to propose.

 Please include a summary of the project and links to web pages
 that can help prospective students to write their application.
 Please also include the following information:

 - if applicable, other dialects that you would be willing to
   mentor this project for

 - the skill level

 - name of the mentor(s), email addresses, and possibly any IRC
   network/channel/nickname where they can be found.

 Thanks for contributing to ESUG's Summer of Code application!


Just a note that I haven't yet processed this thread, but our ideas
page
is already looking pretty well (and Pharo section is big) so I'll
probably do it in the weekend or next Monday.


I can do it if you want.

Cheers,
Nico


--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

Whoops, it was an old bug after all! :(

On 1/13/14 4:51 , Nicolai Hess wrote:

Yes, chaging the system time is a known issue for pharos linux-vm:

11324 https://pharo.fogbugz.com/default.asp?11324 Image freeze when
changing system time

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

Try the following on OS X...

1.  Launch Pharo (Cuis).

2.  Go to the System Settings time panel.

3.  Disable automatic time updates.

4.  Bump the clock forward by 1 minute.

5.  Check whether Pharo is still responsive (in the case of Cuis, it's 
frozen).


6.  Wait one minute.

7.  Check whether Pharo is still responsive (in the case of Cuis, it's 
responsive again).



As a variation, disable automatic time updates, then bump the clock 
backwards by 1 minute.  Things still work.  Then re-enable automatic 
time updates.  When the clock goes forward, the prediction is that Pharo 
(Cuis) will be frozen for 1 minute.


If this repeats for you, then also look at /var/log/system.log and see 
how many entries are there for NTP.  If you see that NTP periodically 
adjusts the clock forward, then you can also expect that Pharo (Cuis) 
will periodically be frozen for as much time as NTP (and, on OS X 10.9, 
pacemaker) bumps the clock forward.


I can reproduce this behavior on OS X 10.9 with a modern Cog VM and Cuis 
images.


On 1/13/14 0:22 , Sven Van Caekenberghe wrote:

Hi Andres,

Could you describe in some detail how to make something clearly fail related to 
this issue ?

There have been mysterious time(r) related problems in the past, it would be 
good to follow up on something concrete and repeatable.

Thx,

Sven

On 12 Jan 2014, at 04:41, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


Somewhat related only, but FYI something I've noticed with Cog VMs is that if 
you bump the computer clock forward by e.g. 30 seconds then Cuis remains frozen 
for 30 seconds.  Under the assumption that Cuis and Pharo are close enough, and 
considering that alt+. does not bring up a debugger while the image remains 
frozen, I can't help asking whether the VM is setting up OS alarms at a certain 
wall clock hour instead of setting up timers that expire after N time has 
elapsed (irrespective of the wall clock).  If the VM is setting up alarms at 
wall clock hours, then I'd strongly suggest changing that behavior because of 
NTP, time zone adjustments, etc.

BTW, I found this because apparently Mavericks isn't keeping time that great 
and NTP frequently changes the clock by ~2 seconds.

On 1/11/14 19:29 , Martin McClure wrote:

I'm beginning to suspect that delays are not expiring at the OS level


.

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

My friend, you need to believe :)...

Ok, so with Cuis the image hangs (unresponsive to input) when the clock 
goes forward.  With Pharo 2.0 (just checked), the image hangs when the 
clock goes backwards.  To verify, simply create a new process that 
periodically writes something to the Transcript, such as


p := [
[
(Delay forSeconds: 1) wait.
Transcript cr; show: Time now seconds printString
] repeat
] newProcess.
p resume

then go to the system settings panel, time section, uncheck the 
automatic time keeping, bump the clock up by one minute, save (note the 
skew in the Transcript output), then re-enable automatic time keeping, 
make sure the settings are saved... and now the image is unresponsive. 
Alt+. does nothing etc.  Wait one minute.  After that time, you will get 
a debugger and things will start happening again.


If you can't immediately reproduce it, I really suspect that if you play 
with the time for a bit you will get some weird behavior too.


On 1/13/14 1:18 , Esteban Lorenzano wrote:

Hi Andres,

Pharo (Cuis) does not exit… I suppose you are talking about the vm from Pharo, 
used with a Cuis image… can you confirm that same problem exists in pharo-vm 
with pharo-image?

Esteban

On 13 Jan 2014, at 09:37, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


Try the following on OS X...

1.  Launch Pharo (Cuis).

2.  Go to the System Settings time panel.

3.  Disable automatic time updates.

4.  Bump the clock forward by 1 minute.

5.  Check whether Pharo is still responsive (in the case of Cuis, it's frozen).

6.  Wait one minute.

7.  Check whether Pharo is still responsive (in the case of Cuis, it's 
responsive again).


As a variation, disable automatic time updates, then bump the clock backwards 
by 1 minute.  Things still work.  Then re-enable automatic time updates.  When 
the clock goes forward, the prediction is that Pharo (Cuis) will be frozen for 
1 minute.

If this repeats for you, then also look at /var/log/system.log and see how many 
entries are there for NTP.  If you see that NTP periodically adjusts the clock 
forward, then you can also expect that Pharo (Cuis) will periodically be frozen 
for as much time as NTP (and, on OS X 10.9, pacemaker) bumps the clock forward.

I can reproduce this behavior on OS X 10.9 with a modern Cog VM and Cuis images.

On 1/13/14 0:22 , Sven Van Caekenberghe wrote:

Hi Andres,

Could you describe in some detail how to make something clearly fail related to 
this issue ?

There have been mysterious time(r) related problems in the past, it would be 
good to follow up on something concrete and repeatable.

Thx,

Sven

On 12 Jan 2014, at 04:41, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


Somewhat related only, but FYI something I've noticed with Cog VMs is that if 
you bump the computer clock forward by e.g. 30 seconds then Cuis remains frozen 
for 30 seconds.  Under the assumption that Cuis and Pharo are close enough, and 
considering that alt+. does not bring up a debugger while the image remains 
frozen, I can't help asking whether the VM is setting up OS alarms at a certain 
wall clock hour instead of setting up timers that expire after N time has 
elapsed (irrespective of the wall clock).  If the VM is setting up alarms at 
wall clock hours, then I'd strongly suggest changing that behavior because of 
NTP, time zone adjustments, etc.

BTW, I found this because apparently Mavericks isn't keeping time that great 
and NTP frequently changes the clock by ~2 seconds.

On 1/11/14 19:29 , Martin McClure wrote:

I'm beginning to suspect that delays are not expiring at the OS level


.






.

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

(of course, make sure you're online when you re-enable automatic time 
keeping so that NTP bumps the clock backwards... if not, do it manually)


On 1/13/14 4:10 , Andres Valloud wrote:

My friend, you need to believe :)...

Ok, so with Cuis the image hangs (unresponsive to input) when the clock
goes forward.  With Pharo 2.0 (just checked), the image hangs when the
clock goes backwards.  To verify, simply create a new process that
periodically writes something to the Transcript, such as

p := [
[
(Delay forSeconds: 1) wait.
Transcript cr; show: Time now seconds printString
] repeat
] newProcess.
p resume

then go to the system settings panel, time section, uncheck the
automatic time keeping, bump the clock up by one minute, save (note the
skew in the Transcript output), then re-enable automatic time keeping,
make sure the settings are saved... and now the image is unresponsive.
Alt+. does nothing etc.  Wait one minute.  After that time, you will get
a debugger and things will start happening again.

If you can't immediately reproduce it, I really suspect that if you play
with the time for a bit you will get some weird behavior too.

On 1/13/14 1:18 , Esteban Lorenzano wrote:

Hi Andres,

Pharo (Cuis) does not exit… I suppose you are talking about the vm from Pharo, 
used with a Cuis image… can you confirm that same problem exists in pharo-vm 
with pharo-image?

Esteban

On 13 Jan 2014, at 09:37, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


Try the following on OS X...

1.  Launch Pharo (Cuis).

2.  Go to the System Settings time panel.

3.  Disable automatic time updates.

4.  Bump the clock forward by 1 minute.

5.  Check whether Pharo is still responsive (in the case of Cuis, it's frozen).

6.  Wait one minute.

7.  Check whether Pharo is still responsive (in the case of Cuis, it's 
responsive again).


As a variation, disable automatic time updates, then bump the clock backwards 
by 1 minute.  Things still work.  Then re-enable automatic time updates.  When 
the clock goes forward, the prediction is that Pharo (Cuis) will be frozen for 
1 minute.

If this repeats for you, then also look at /var/log/system.log and see how many 
entries are there for NTP.  If you see that NTP periodically adjusts the clock 
forward, then you can also expect that Pharo (Cuis) will periodically be frozen 
for as much time as NTP (and, on OS X 10.9, pacemaker) bumps the clock forward.

I can reproduce this behavior on OS X 10.9 with a modern Cog VM and Cuis images.

On 1/13/14 0:22 , Sven Van Caekenberghe wrote:

Hi Andres,

Could you describe in some detail how to make something clearly fail related to 
this issue ?

There have been mysterious time(r) related problems in the past, it would be 
good to follow up on something concrete and repeatable.

Thx,

Sven

On 12 Jan 2014, at 04:41, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


Somewhat related only, but FYI something I've noticed with Cog VMs is that if 
you bump the computer clock forward by e.g. 30 seconds then Cuis remains frozen 
for 30 seconds.  Under the assumption that Cuis and Pharo are close enough, and 
considering that alt+. does not bring up a debugger while the image remains 
frozen, I can't help asking whether the VM is setting up OS alarms at a certain 
wall clock hour instead of setting up timers that expire after N time has 
elapsed (irrespective of the wall clock).  If the VM is setting up alarms at 
wall clock hours, then I'd strongly suggest changing that behavior because of 
NTP, time zone adjustments, etc.

BTW, I found this because apparently Mavericks isn't keeping time that great 
and NTP frequently changes the clock by ~2 seconds.

On 1/11/14 19:29 , Martin McClure wrote:

I'm beginning to suspect that delays are not expiring at the OS level


.






.



.

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

My friend, you also need to believe :)...

Checked with Pharo 3.0, misbehavior reproduced.

I'm done hanging various versions of Pharo and Cuis on my machine.  It's 
someone else's turn now :D.


On 1/13/14 4:17 , Marcus Denker wrote:


On 13 Jan 2014, at 13:10, Andres Valloud avall...@smalltalk.comcastbiz.net 
wrote:


My friend, you need to believe :)...

Ok, so with Cuis the image hangs (unresponsive to input) when the clock goes 
forward.  With Pharo 2.0 (just checked),


It might be better to check in Pharo3… there where changes
e.g.

7444 Synchronizing DateAndTime with system OS date and time
https://pharo.fogbugz.com/f/cases/7444

Marcus

Re: [Pharo-dev] Pharo 3.0 not responding to mouse or keyboard after launch

[Andres Valloud]

Somewhat related only, but FYI something I've noticed with Cog VMs is 
that if you bump the computer clock forward by e.g. 30 seconds then Cuis 
remains frozen for 30 seconds.  Under the assumption that Cuis and Pharo 
are close enough, and considering that alt+. does not bring up a 
debugger while the image remains frozen, I can't help asking whether the 
VM is setting up OS alarms at a certain wall clock hour instead of 
setting up timers that expire after N time has elapsed (irrespective of 
the wall clock).  If the VM is setting up alarms at wall clock hours, 
then I'd strongly suggest changing that behavior because of NTP, time 
zone adjustments, etc.


BTW, I found this because apparently Mavericks isn't keeping time that 
great and NTP frequently changes the clock by ~2 seconds.


On 1/11/14 19:29 , Martin McClure wrote:

I'm beginning to suspect that delays are not expiring at the OS level

[Pharo-dev] Smalltalks 2013 video now available

[Andres Valloud]

Check out the playlist here:

http://www.youtube.com/playlist?list=PLCGAAdUizzH027lLWKXh_44cGuEsay7-R

Make sure you don't miss Smalltalks 2014!

Andres.

Re: [Pharo-dev] Efficient string concatenation - proposed new

[Andres Valloud]

Well, maybe, although it's interesting to consider how many strings 
streams need to beat string concatenation.  Moreover, streams aren't 
hyper efficient either...


On 11/16/13 10:09 , b...@openinworld.com wrote:


Code Critic rule Optimization  String concatenation instead of streams
says:

Check for string concatenation inside some iteration message. Since
string concatenation is O(n^2), it is better to use streaming since it
is O(n) - assuming that n is large enough. As a general principal avoid
, since the receiver is copied. Therefore chaining , messages will lead
to multiple useless copies of the receiver.

That is,
 String streamContents: [:s |
 #('abc' 'def' 'ghi')  do: [:each | s nextPutAll: each asString]]

should be used instead of...
 'abc' , 'def' , 'ghi'.

However the first clutters the code.  What about something like...
 { 'abc' . 'def' . 'ghi' } asStreamString
where
 CollectionasStreamString
 ^ String streamContents: [:s | self do: [:each | s nextPutAll:
each asString]]

cheers -ben

Re: [Pharo-dev] Efficient string concatenation - proposed new

[Andres Valloud]

The details really don't fit in an email, however the 2nd volume of my 
Fundamentals book already has an analysis of this issue and hence my 
previous comment.


On 11/17/13 8:45 , Stéphane Ducasse wrote:

this one is surprising to me


[ 'foo', 'bar', 'baz', 'foobar' ] bench. '2,030,000 per second.'
[ '' join: #( 'foo' 'bar' 'baz' 'foobar' ) ] bench. '665,000 per second.'
[ String streamContents: [ :out | out nextPutAll: 'foo'; nextPutAll:
'bar'; nextPutAll: 'baz'; nextPutAll: 'foobar' ] ] bench. '1,580,000
per second.'


.

Re: [Pharo-dev] Efficient string concatenation - proposed new

[Andres Valloud]

We’re only measuring execution speed, not memory allocation, which is important 
too.


Yes.  And besides having to collect the garbage at some point, 
allocation can be costly because the VM has to zero out the string bytes 
(or nil them if the string is not encoded) and then write on those 
zeroed (nilled) places with something else.  It takes time.



The length of the strings is a variable as well, of course.


The order also matters, e.g.

(String new: 7812681237643287423), 'a', 'b', 'c'

and the way you aggregate the result matters as well.


Conclusion, let’s be careful with a too simple advice.


Yes, it's much more complex than it seems when the inefficiency of naive 
concatenation methods is not flagrant.


Andres.

Re: [Pharo-dev] Pharo VM latest on the Debian 7

[Andres Valloud]

Are you saying the Pharo VM requires glibc version 2.15 of March 2012, 
and that consequently the Pharo VM as compiled won't run on any Linux 
distro older than 17 months?


On 8/8/13 5:52 , Igor Stasenko wrote:

Yes. It is known issue.
Now there's little we can do, except from building VM for debian
platform separately.

On 8 August 2013 11:43, Laszlo Zsolt Kiss lzsk...@dublin.ie wrote:

Hi.

The latest Pharo VM isnt working on Debian 7.

The error message:

/work/pharo3.0/bin/pharo: /lib/i386-linux-gnu/i686/cmov/libc.so.6: version
`GLIBC_2.15' not found

The Glibc version in Debian 7 is 2.13.

Best regards

Laszlo Zsolt KIss