/Usecase 1: Debug printing methods:/ In the VM you have
debug printing methods, for example, to print the call
stack. These methods are used from the VM simulator, to
output the string in the Transcript, and in gdb, to ouput
the string in the commandline. The commandline
(FileStream stdout in Pharo) and the Squeak Transcript
have the same behavior. In Pharo, the Transcript does not
inherit from Stream so you can't use the required stream
methods to print the debug printing method on the
Transcript. In addition, some printing methods print a
lot of things and it is important to show the stream as
it is printed.
For this use-case, we want to keep the smallest
difference between the gdb/commadline behavior and the VM
simulator/Transcript behavior. If you implement advanced
tooling in GT, you therefore need to implement gdb
extensions (and lldb extensions because some of us use
lldb instead of gdb) and maintain them. I don't think
this is a solution.
/Usecase 2: CCode generation debugging:/ The
CCodeGenerator or Slang translator translates Slang code
into C code. Sometimes there is a bug. To debug, instead
of generating the faulty C method into an external C
file, we print only the faulty C method in the
Transcript. Again, we want to keep the lowest difference
between the real usecase (printing on the C file) and the
debug usecase (printing on the Transcript). In Squeak the
FileStream and the Transcript are both Stream, everything
works as expected. In Pharo the Transcript has not the
expected behavior. Again the method can be long, you can
have to wait several seconds, so you'd like the
transcript to show the stream as you print it.
/Usecase 3: VM simulation:/ Simulating the VM is quite
slow, especially the machine code execution simulation.
During the simulation process, the UI is non interactive
and shows only every while what the simulator is doing in
the Transcript. It is important as sometimes when
debugging with a test at each machine code instruction it
could take several hours before the UI is interactive
again and you want to know what is going on. I don't
complain that it takes several hours because the
alternatives usually require days of debugging and we can
launch the VM simulator overnight. In Pharo this does not
work as expected.
/Usecase 4: In-image machine-code compilation:/ While
working in the JIT compiler, sometimes the machine code
generated for a bytecoded method is faulty. A common way
of debugging it is to print the machine code instructions
of the machine code version of the method in the
Transcript. It can take a while to print, so it is
important to have the Transcript showing the text as it
prints. Then, the easiest way of debugging is to look at
the machine code and understand what is wrong. For this
purpose, we add text decoration to color jump addresses
or the instructions where the instruction pointer was
when the VM crashed. Then, in squeak, we can easily copy
the decorated text to a workspace and generate a new
version of the machine code method and compare. In
machine code, it is very difficult to do analysis to have
more information than just the decompiled text. We add
some information while simulating because we know for
example the address of specific trampolines, therefore we
can print the name of the trampoline when we see that its
address is called. Again, sometimes we also have to debug
in gdb. In this case, we disassemble the machine code and
compare it to the one from in-image compilation, so both
printed strings have to be similar (similar text, same
chariot returns).
Another example is the complexity of the Pharo tools:
While developing the VM, I have sometimes a VM partially
working or with some plugins not working. In the Squeak
image, I can open a workspace on top of this half-working
VM and run do-its to see what is working and what is not.
In the Pharo image, I can't do anything. You can't open
the workspace without opening more advanced tools. I
tried to open the Playground, but the first time there
was a bug with Traits (Playground use Traits somehow and
they were not working due to the new bytecode set not
being finished), when that first bug was fixed I could
not open it because it crashed simply the VM (I believe
it tried to access an external file such as
playground-cache). Currently, the Pharo team is trying to
build a set of basic tools that have few dependencies to
debug a partially working system (that I think you will
use to debug glamour while editing it, because you cannot
use the glamour inspector if glamour is not working).
That would solve this issue.
But in no way this point is something that I can do alone
to be able to develop the VM in Pharo. This has to be a
community effort. And I am saying that because I can't be
blamed not to work on the VM in Pharo if to do so I need
to spend many months changing Pharo.
An example that I believe is a problem in term of the
community is the following:
I added with Eliot the support for the new bytecode set.
Currently, the Squeak image works with the new bytecode
set but not the Pharo image. This is because only the
Traits are broken, but this is something I could hardly
figure out in the Pharo image because nothing is working
as the GT tools use Traits. In Squeak I believe there are
very few users of Traits so everything worked, and the
test suite can reveal that the Traits are broken easily.
Currently, the VM process to me is to first make new
features work in Squeak, because it is simpler, and then
make it work with Pharo, which is more complex. In the
last section I discussed how Traits were a problem while
implementing the new bytecode set. So what is the long
term solution for this issue ?
- Will we have a bootstrap process that creates first a
Trait-free Kernel and then build the Pharo Kernel out of it ?
- Do we forbid people to use Traits in the Pharo Kernel
and does that make sense to have Traits in Pharo in this
case ?
- If we don't do anything, maybe the Traits are only a
slight difference with low impact in most cases and it's
fine. But maybe there are many small aspects like Traits,
such as the Slots the way they were used in GT recently
(I don't blame GT or anything, it was just using features
in the system that created issues for me), and maybe we
reached a point where the complexity between the Pharo
kernel and the Squeak kernel is big enough so that a VM
developer will first make Squeak works when introducing
new features and then deals with the complexity of Pharo ?
So, what do we do ? I don't see any simple solution for
this issue. And I believe there are people around that
see as the only solution for this issue not to have the
Pharo VM development process in Pharo because they will
see it as a threat to what they want to do with Pharo.
Best Doru !
PS: I am still using the GTInspector with additional
views on graphs created with Roassal everyday and I still
enjoy it.
PS2: I am on vacation currently because I was getting
crazy looking at machine code all day long, so I may not
answer as quick as usually during the next week.
Cheers,
Doru
On Sat, May 9, 2015 at 9:31 PM, Clément Bera
<[email protected]
<mailto:[email protected]>> wrote:
2015-05-09 20:25 GMT+02:00 stepharo
<[email protected] <mailto:[email protected]>>:
Le 9/5/15 20:16, Clément Bera a écrit :
This whole conversation here shows very well
the point that I tried to explain to Stef
last week. I'm sorry if the mail is a bit
long but I think this discussion has to be
done.
My whole Smalltalk development life, I have
used Pharo and was happy with it. Now I am
also working in Cog's JIT compiler and for
this specific project, I am working with
Squeak. I don't work with Squeak because I
don't like Pharo, I told you before, I have
worked with Pharo on all my project before,
enjoyed it and if it was possible I would
use Pharo. I work with Squeak because the VM
development tool and development process
simply does *not* work in Pharo. This is not
only because of VM tools working with the
old Morphic not working anymore in Pharo or
details like that, it is also due to deeper
changes in Pharo.
Stef believes it is important that Pharo is
able to host development for its own VM.
Therefore, I discussed with him and Esteban
about a first list of points that are
necessary for Pharo to support its VM
development in Pharo, which includes this
Transcript behavior.
As of today, and I am honest here, I believe
that what is required for Pharo to support
the development process of its VM includes
points which goes in the opposite direction
than a few points in the Pharo roadmap, that
people in the Pharo community will see as a
regression, as "an intrusion from the Squeak
philosophy into Pharo", or as forbidding the
integration of features that breaks the VM
development process. Therefore, I believe
the Pharo community would disapprove to make
such changes and I highly doubt that it is
possible to have the development process of the
Pharo VM in Pharo.
I was thinking that only a few points would
be a problem such as the increasing memory
footprint of the Pharo image that is going
to get worse with the sources that will be
included in the image in the future, whereas
a VM developer needs a small image (See
previous threads in this mailing list where
Hilaire complains about that for example).
clement can I ask a simple question?
why did I ask guille to work on minikernels
and bootstrap for his phd instead on a topic
where we can publish?
- choice A: lack of idea
- choice B: ....
I have already stated that you believe that it is
important that Pharo is able to host development
for its own VM.
I am not against what you did and I am very
excited with Guille's work.
Pharo is community-driven, so I am not asking the
question to you only, but to the community.
However, I didn't think that even simple points
like the Transcript behavior discussed here,
which looks like to me as a regression and is
required for VM development, would be seen as an
improvement by a non negligible part of the
community.
In this mailing-list, the whole Pharo community
is present and can see this discussion. So the
open questions are:
*Do you want to have the development of the
Pharo VM in Pharo, or do you want the
development of the Pharo VM to remain in Squeak ?*
*Do you think a system that is not good enough
to handle its own VM development is a good system ?*
I am not willing to go against the will of the
community because I enjoy community-driven
softwares. If the answer is that Pharo should be
able to support its own VM development then as I
started I will help Esteban and Stef to improve
Pharo so that it can support its own VM
development. Now, if the answer is that the
development of the Pharo VM should remain in
Squeak, I will continue developing the VM in
Squeak.
You are the Pharo community, you are the ones
that make Pharo alive and kicking, so you tell
me what you think we should do.
Clement
2015-05-09 18:23 GMT+02:00 Eliot Miranda
<[email protected]
<mailto:[email protected]>>:
Hi Ben,
On May 9, 2015, at 7:41 AM, Ben Coman
<[email protected]
<mailto:[email protected]>> wrote:
On Sat, May 9, 2015 at 10:09 PM, Ben Coman
<[email protected]
<mailto:[email protected]>> wrote:
From my limited experience bug hunting,
calling #changed: from a thread other
than the UI thread is a source of
evil. There are too many assumptions
throughout the system that the UI is
single threaded. Can anyone advise me
that is not a proper belief?
Then that implies that a Transcript
implementation where #nextPut: direct
calls #changed:
is not appropriate for use with
multi-threaded applications. In Pharo,
#changed: is only called from
#stepGlobal, which is called from
doOneCycle:. (This came about as a
last minute bug fix before Pharo 3
release and maybe could use some cleanup.
Separating the UI from Transcript into
its own viewer might be a good idea,
but actually it would not solve Stef's
case since his code would still be
running in the UI thread -- unless the
viewer ran in another thread, which
would have its own complexities.
I think the point about efficiency is
significant. The following example...
Time millisecondsToRun: [ 1000
timesRepeat: [ Transcript show: 'x' ] ]
on Squeak 4.5 --> 12749ms
on Pharo 50029 --> 2ms
As a point of comparison, on VW 8.0 -->
43817ms
and so you might guess, VW 8.0 outputs each
'x' immediately.
cheers -ben
Way to go, Squeak! Actually this is
disappointing. I'm rather frustrated with
Squeak's slow transcript, and was hoping
that VW would demonstrate it could be
faster. Looking at the Squeak implementation
I only see an obvious 30% or so improvement
via tuning. Looks like good performance will
take more work :-/
Eliot (phone)
--
www.tudorgirba.com <http://www.tudorgirba.com/>
"Every thing has its own flow"
