RE: [bigloo] Dataflow+ debuging
Hello all, Fix is working ! you can go with this :-) Thanks a lot for this ( like always ) very quick fix. Cheers, Pierre-Francois -Original Message- From: manuel.serr...@inria.fr [mailto:manuel.serr...@inria.fr] Sent: Thursday, March 15, 2018 7:30 AM To: manuel.serr...@inria.fr Cc: Ollagnon, Pierre-Francois ; bigloo@inria.fr Subject: RE: [bigloo] Dataflow+ debuging Hi Everyone, I think the problem is now solved. I have uploaded the patch into the github repository. When I have the confirmation that the problem is indeed solved, I will try my best to prepare the 4.3c final release. Then we will enter the 4.3d development cycle. Cheers, -- Manuel
RE: [bigloo] Dataflow+ debuging
Excellent! With this I should be able to fix the compiler problem. I will work on it as soon as possible. Thanks a lot for your effort and highly valuable feedback. -- Manuel
RE: [bigloo] Dataflow+ debuging
Hello Manuel, Like every time, it is really hard for us to make a testcase (more than 400 000 lines of code + C interface, TCL, other external C libraries . ) but I have worked all the day to finally find the issue and succeed to make a testcase: In fact it is the (procedure?) function that assume during the cfa that an object is not a function. Hence it doesn't go inside a part of the code where the "get-key" is used. It makes the Cfa thinking that the get-key function is not "lost?" and not used so it is replaced by the el-procedure. You have in attachment a simple testcase that shows the case. Note: I would be interested to know how to use the -t[24] +tpass function: I finally replace manually the "trace" function by a print but it would be usefully to use it manually without manual tweeks ! Best regards, Pierre-Francois -Original Message- From: bigloo-requ...@inria.fr [mailto:bigloo-requ...@inria.fr] On Behalf Of manuel.serr...@inria.fr Sent: Wednesday, March 14, 2018 1:50 PM To: Ollagnon, Pierre-Francois Cc: bigloo@inria.fr Subject: Re: [bigloo] Dataflow+ debuging Hi Pierre-François, I have bad time understanding what's going on. Could you provide a complete function definition because currently I'm not able to build something similar to the situation that raises the bug. You mentioned that the dataflow has erroneously removed some code. This is quite possible I will check what's going on as soon as possible but the problem is that in your excerpt I don't see what has been removed. What is missing according to you? I see no difference in between the dataflow and dataflow+. I don't suspect a GC problem. For now, it rather sounds like a mistake in the CFA analysis that incorrectly transforms a plain procedure into an el-procedure. It would really helped if you could send me an actual code (even privately if you have copyright issues). Thanks in advance. -- Manuel Bug_bigloo_cfa.tar.gz Description: Bug_bigloo_cfa.tar.gz
Re: [bigloo] Dataflow+ debuging
Hi Pierre-François, I have bad time understanding what's going on. Could you provide a complete function definition because currently I'm not able to build something similar to the situation that raises the bug. You mentioned that the dataflow has erroneously removed some code. This is quite possible I will check what's going on as soon as possible but the problem is that in your excerpt I don't see what has been removed. What is missing according to you? I see no difference in between the dataflow and dataflow+. I don't suspect a GC problem. For now, it rather sounds like a mistake in the CFA analysis that incorrectly transforms a plain procedure into an el-procedure. It would really helped if you could send me an actual code (even privately if you have copyright issues). Thanks in advance. -- Manuel
RE: [bigloo] Dataflow+ debuging
Hello Jospeh,
I have already dug the issue and when looking at expansion results it is CLEAR
that the problem comes from the expansion phase that suppress my code – please
look at my first mail that explain this –
I do not succeed to make a simple testcase showing the issue.
So my question is how I can debug ON MY SIDE the way Bigloo is doing dataflow
optimization.
The work of today (after patching bigloo) shows:
light?: (make-fx-procedure &get-key,#{approx _} 1 1) ... may be X and T (local)
It may be X and T because of this :
lost?: #f
X-T?: #t
arity: 2
import: local
Its seems that combination leads to my issue
Anyone can help to explain me what it means?
Best regards,
Pierre-Francois
From: Joseph Donaldson [mailto:donaldso...@yahoo.com]
Sent: Monday, March 12, 2018 7:32 PM
To: bigloo@inria.fr; Ollagnon, Pierre-Francois
Subject: Re: RE: [bigloo] Dataflow+ debuging
Hello, Pierre-Francois,
The runtime time error suggests the possibility of memory corruption. The
scenario I am envisioning is the following. The get-key binding is initially
correctly, but at some point in the execution flow, it is being corrupted,
eventually being reported as the error you are seeing in pvt-maker:dump. I have
seen similar errors before, and they were generally caused by c code that I was
interfacing with; the c code erroneously overwrote memory it should not have.
Do you happen to be interfacing with c as a part of your project? The memory
layout of a program is sensitive to not only changes in your code (such as the
addition of a print statement) but also to what optimizations are used. This
could explain why the code works when data-flow is disabled or when a print
statement is added.
If your program does not interface with c, compiling for the JVM backend may
help you diagnose the underlying issue. The runtime errors provided by the JVM
backend are different and sometimes can point you in the direction of the
underlying problem.
It would be great if you could provide the problematic code or, even better, a
small example demonstrating the issue. I know this is not always possible, but
it is very hard to provide assistance without such information.
Best Regards,
Joseph Donaldson
Re: [bigloo] Dataflow+ debuging
Hello, Pierre-Francois, The runtime time error suggests the possibility of memory corruption. The scenario I am envisioning is the following. The get-key binding is initially correctly, but at some point in the execution flow, it is being corrupted, eventually being reported as the error you are seeing in pvt-maker:dump. I have seen similar errors before, and they were generally caused by c code that I was interfacing with; the c code erroneously overwrote memory it should not have. Do you happen to be interfacing with c as a part of your project? The memory layout of a program is sensitive to not only changes in your code (such as the addition of a print statement) but also to what optimizations are used. This could explain why the code works when data-flow is disabled or when a print statement is added. If your program does not interface with c, compiling for the JVM backend may help you diagnose the underlying issue. The runtime errors provided by the JVM backend are different and sometimes can point you in the direction of the underlying problem. It would be great if you could provide the problematic code or, even better, a small example demonstrating the issue. I know this is not always possible, but it is very hard to provide assistance without such information. Best Regards,Joseph Donaldson
RE: [bigloo] Dataflow+ debuging
Hello, Like clearly said in my previous mail, it is the "data-flow-types / dataflow+" optimization step that fails. -O2 is not working -O2 -fno-datatype is working. But 1) I do not like to add specific compilation options for a certain module because of a bug. 2) If it arrive here it can arrive elsewhere so I prefer to fix bigloo... Yes it is working in released 4.3a (with -O6) . And not working in the current 4.3b from -O2. Thanks Pierre-Francois -Original Message- From: bigloo-requ...@inria.fr [mailto:bigloo-requ...@inria.fr] On Behalf Of Sven Hartrumpf Sent: Monday, March 12, 2018 12:37 PM To: bigloo@inria.fr Subject: Re: [bigloo] Dataflow+ debuging Hello. Pierre-Francois wrote, 2018-03-09: > I have an issue when compiling: The Dataflow+ fail and "remove" a part of the > code: > Unfortunatly I did not success to make a testcase Issue is probably > coming by the fact that several things are coming and used in several modules. > For information, it is working in 4.3a. Is it working in the released version 4.3a? In which versions do you get the problematic binary? If you suspect a too aggressive optimization, it is advisable to determine the optimization option(s) that turn from good to bad. You have not specified your bigloo compiler options, but in general you could try with: -O3 -O4 -O5 and proceed until your binary is incorrect. If you have identified the problematic step (e.g. from -O4 to -O5), you can add the -O5-specific options one by one to -O4. Greetings Sven
Re: [bigloo] Dataflow+ debuging
Hello. Pierre-Francois wrote, 2018-03-09: > I have an issue when compiling: The Dataflow+ fail and "remove" a part of the > code: > Unfortunatly I did not success to make a testcase Issue is probably > coming by the fact that several things are coming and used in several modules. > For information, it is working in 4.3a. Is it working in the released version 4.3a? In which versions do you get the problematic binary? If you suspect a too aggressive optimization, it is advisable to determine the optimization option(s) that turn from good to bad. You have not specified your bigloo compiler options, but in general you could try with: -O3 -O4 -O5 and proceed until your binary is incorrect. If you have identified the problematic step (e.g. from -O4 to -O5), you can add the -O5-specific options one by one to -O4. Greetings Sven
RE: [bigloo] Dataflow+ debuging
Hello Joseph.
I can provide more informations, but unfortunately it is like a ball of whool
that never ends.
That’s why I am more asking for a way to debug on my side …
There is no Error reported by the compiler, but it fails when running:
I get:
*** ERROR:pvt-maker:dump
Type "procedure" expected, "bint" provided – 11130932
%hashtable is a “custom hashtable” so it is a function not a bigloo hash.
To go futher:
The
“ (something that pass get-key in argument) “
Is something like:
(for-each
(lambda (the-pvt)
(print-pvt the-pvt get-key))
lpvts)
print-pvt is a function that will do:
(the-pvt :dump get-key)
Finaly the-pvt is a lambda that will also call another lambda with the
“get-key” function.
In this final function the get-key is an int instead of a function.
Note that if for example, just before the for-each I put a “(print get-key)”
Everything works.
It is like Bigloo assume that get-key is not used and remove the computation of
it.
If I add the print, get-key is correctly computed (ie not replaced by the
make-el-procedure 1) and the print-pvt function can use it correctly.
Thanks for your Help,
Pierre-Francois
From: Joseph Donaldson [mailto:donaldso...@yahoo.com]
Sent: Friday, March 09, 2018 5:44 PM
To: Ollagnon, Pierre-Francois ;
bigloo@inria.fr
Subject: Re: [bigloo] Dataflow+ debuging
Hello, Pierre-Francois,
Can you provide some more information? For example, what is the actual error
reported by the compiler and what is the definition of %hashtable? I assume
from its use that it is either a function or macro, but it's name suggests it
is a hashtable, which is problematic since they are not funcallable.
Best Regards,
Joseph Donaldson
On Fri, Mar 9, 2018 at 8:16 AM, Ollagnon, Pierre-Francois
mailto:pierre-francois_ollag...@mentor.com>>
==
(let* (((get-key(lambda (obj)
(let* ((key (obj :get-key))
(stuff (%hashtable :get key)))
(if stuff
stuff
"unknown"
(something that pass get-key in argument)
===
-Note that “stuff” is a lambda
After Dataflow it becomes:
==
(get-key
(obj)
(let ((key (funcall
obj
obj
:get-key)))
(let ((stuff (funcall
%hashtable
%hashtable
:get
key)))
(let ()
(if stuff
stuff
"unknown")
==
After the Dataflow+ it becomes:
==
Top of file:
-
;; &get-key
;; procedure-el x obj -> obj
;; [static side-effect: #t occ: 1 rm: now loc: #{location src/pvtsc_rdb.scm
46059 973} user?: #t removable: now]
(define (&get-key env obj)
(let ((%hashtable (procedure-el-ref env 0)))
(labels
((get-key
(obj)
(let ((key (funcall obj obj :get-key-pvtsc)))
(let ((stuff (funcall %hashtable %hashtable :get key)))
(let () (if stuff stuff "unknown"))
(get-key obj
In the “normal code”:
---
(let ((get-key
(make-el-procedure
1)))
(begin
(begin
(procedure-el-set!
get-key
0
%hashtable))
==
Do you have any advice on how I can debug this ???
Best regards,
Pierre-Francois
Re: [bigloo] Dataflow+ debuging
Hello, Pierre-Francois,
Can you provide some more information? For example, what is the actual error
reported by the compiler and what is the definition of %hashtable? I assume
from its use that it is either a function or macro, but it's name suggests it
is a hashtable, which is problematic since they are not funcallable.
Best Regards,Joseph Donaldson
On Fri, Mar 9, 2018 at 8:16 AM, Ollagnon,
Pierre-Francois
==
(let* (((get-key (lambda (obj)
(let* ((key (obj :get-key))
(stuff (%hashtable :get key)))
(if stuff
stuff
"unknown"
(something that pass get-key in argument)
===
-Note that “stuff” is a lambda
After Dataflow it becomes:
==
(get-key
(obj)
(let ((key (funcall
obj
obj
:get-key)))
(let ((stuff (funcall
%hashtable
%hashtable
:get
key)))
(let ()
(if stuff
stuff
"unknown")
==
After the Dataflow+ it becomes:
==
Top of file:
-
;; &get-key
;; procedure-el x obj -> obj
;; [static side-effect: #t occ: 1 rm: now loc: #{location src/pvtsc_rdb.scm
46059 973} user?: #t removable: now]
(define (&get-key env obj)
(let ((%hashtable (procedure-el-ref env 0)))
(labels
((get-key
(obj)
(let ((key (funcall obj obj :get-key-pvtsc)))
(let ((stuff (funcall %hashtable %hashtable :get key)))
(let () (if stuff stuff "unknown"))
(get-key obj
In the “normal code”:
---
(let ((get-key
(make-el-procedure
1)))
(begin
(begin
(procedure-el-set!
get-key
0
%hashtable))
==
Do you have any advice on how I can debug this ???
Best regards,
Pierre-Francois
