Re: [racket-users] errortrace requires 10x time and memory for the program to compile
Matthew, On 12/10/2016 06:22 PM, Matthew Flatt wrote: At Mon, 5 Dec 2016 10:19:03 -0700, Matthew Flatt wrote: There's a pending issue of making sure that `for` loops or other things are not needlessly instrumented, since they're only part of the expansion instead of the original code. We haven't gotten back to that, but I bet it would help with your program. We got back to that over the past week, so you could try a snapshot version of Racket to see if errortrace now works better for your application. It is better, however not to an extent that would make it usable for my project. It came down from ~10x to ~6x. Racket 6.7 or nightly, no errortrace: 17 seconds, ~260 MB of RAM Racket 6.7 + errortrace : 150 seconds, 2600+ MB of RAM Racket nightly + errortrace : 118 seconds, ~1600 MB of RAM As I wrote previously, I have no urgent need to have errortrace's demands lowered further, since I got around with hand-made continuation marks. However, I will surely provide any information about my program, in case such information is needed to debug errortrace's behavior. I will be happy to get back to errortrace in the future. Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
At Mon, 5 Dec 2016 10:19:03 -0700, Matthew Flatt wrote: > There's a pending issue of making sure that `for` loops or other things > are not needlessly instrumented, since they're only part of the > expansion instead of the original code. We haven't gotten back to that, > but I bet it would help with your program. We got back to that over the past week, so you could try a snapshot version of Racket to see if errortrace now works better for your application. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
Robby, On 12/06/2016 07:20 PM, Robby Findler wrote: I'm not sure if it helps, but errortrace fully expands your program and then traverses that and adds continuation marks (this is called "annotation" in the errortrace docs). There may be a bug in this process that causes information to be lost, I suppose, but the problem with the dynamically required module may also be that it didn't annotate it. It won't annotate it if there is a .zo file already, I believe. errortrace did add its continuation marks into the dynamically loaded module. It was the build-in Racket's (current-continuation-marks) who did not succeed in that. errortrace works great in principle, but too slow in some cases. Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
I'm not sure if it helps, but errortrace fully expands your program and then traverses that and adds continuation marks (this is called "annotation" in the errortrace docs). There may be a bug in this process that causes information to be lost, I suppose, but the problem with the dynamically required module may also be that it didn't annotate it. It won't annotate it if there is a .zo file already, I believe. Robby On Tue, Dec 6, 2016 at 10:15 AM, Dmitry Pavlov wrote: > >> I guess, under these circumstances, I should >> try and make my own continuation marks in the parser/compiler. > > > I managed to to that. I can not say that it is a beautiful implementation, > but it works. > > For "a+b", instead of a syntax object of '(plus a b), > the parser now generates a syntax object of > '(with-cont-mark (plus a b)), and my compiler replaces it with > (with-continuation-mark my-generated-mark-key scrloc (plus a b)) > where srcloc is calculated from the syntax object. > > So the same source location is specified twice: first in the custom > continuation mark, for my custom diagnostics, and second in the syntax > object itself, for Racket standard diagnostics. > > I wonder how errortrace manages to thread the continuation marks through all > the syntax transformers without them noticing. > > In general, I have a vague feeling that I am making inferior ad-hoc Racket > tools for tasks that are solved in Racket itself with better tools. An issue > that I find similar was reported by Konrad Hinsen earlier this year: > http://www.mail-archive.com/racket-users@googlegroups.com/msg32937.html > > > Best regards, > > Dmitry > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
I guess, under these circumstances, I should try and make my own continuation marks in the parser/compiler. I managed to to that. I can not say that it is a beautiful implementation, but it works. For "a+b", instead of a syntax object of '(plus a b), the parser now generates a syntax object of '(with-cont-mark (plus a b)), and my compiler replaces it with (with-continuation-mark my-generated-mark-key scrloc (plus a b)) where srcloc is calculated from the syntax object. So the same source location is specified twice: first in the custom continuation mark, for my custom diagnostics, and second in the syntax object itself, for Racket standard diagnostics. I wonder how errortrace manages to thread the continuation marks through all the syntax transformers without them noticing. In general, I have a vague feeling that I am making inferior ad-hoc Racket tools for tasks that are solved in Racket itself with better tools. An issue that I find similar was reported by Konrad Hinsen earlier this year: http://www.mail-archive.com/racket-users@googlegroups.com/msg32937.html Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
I just tried that: unfortunately, this stack trace does not seem to be able to cross the boundary of dynamically required module. What I see in (continuation-mark-set->context (current-continuation-marks)) are just lines in the "main" Racket module, and no lines that belong to the non-sexp module or deeper. And (continuation-mark-set->context (exn-continuation-marks e)) do cross the boundary, but show only some deeper lines in deeper modules, not in the "top" dynamically required module. Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
Matthew, Question 1: A factor of 10 is on the high side, but not unusual at the moment. There's a pending issue of making sure that `for` loops or other things are not needlessly instrumented, since they're only part of the expansion instead of the original code. We haven't gotten back to that, but I bet it would help with your program. Question 2: No, unfortunately. OK, got it. Question 3: If the built-in, approximate stack trace is good enough, you can get it with (continuation-mark-set->context (current-continuation-marks)) I just tried that: unfortunately, this stack trace does not seem to be able to cross the boundary of dynamically required module. Or, to be precise, the module that is required via (require-input-port) -- the function that you kindly wrote for me three years ago: https://lists.racket-lang.org/users/archive/2013-September/059449.html What I see in (continuation-mark-set->context (current-continuation-marks)) are just lines in the "main" Racket module, and no lines that belong to the non-sexp module or deeper. Actually, all I need is the line number of the non-sexp file where the error originated from. I guess, under these circumstances, I should try and make my own continuation marks in the parser/compiler. Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] errortrace requires 10x time and memory for the program to compile
Question 1: A factor of 10 is on the high side, but not unusual at the moment. There's a pending issue of making sure that `for` loops or other things are not needlessly instrumented, since they're only part of the expansion instead of the original code. We haven't gotten back to that, but I bet it would help with your program. Question 2: No, unfortunately. Question 3: If the built-in, approximate stack trace is good enough, you can get it with (continuation-mark-set->context (current-continuation-marks)) At Mon, 5 Dec 2016 19:56:07 +0300, Dmitry Pavlov wrote: > Hello, > > I have a program that takes 17 seconds and ~260 MB of memory. > If I use errortrace on it, the numbers grow about tenfold: 150 seconds and > 2600+ MB. > > That is just compilation; in the runtime the program does almost nothing and > terminates quickly. > I know little about how errortrace works and ask for help. > My understanding of the problem is shaped into three questions. > > Question 1. Is it normal for errortrace? > > Note 1: I am using errortrace just for getting the stack trace of a runtime > error: > > (with-handlers >((exn:fail? > (lambda (e) >(for ((stack-elem (continuation-mark-set->list > (exn-continuation-marks e) errortrace-key))) > .. > > > Note 2: The program which stack trace I am interested in is require-d > dynamically, and is actually written in non-lispy language. > errortrace does a great job in planting its continuation marks > into syntax objects that are generated by the parser and > the compiler for that language. > > > Question 2: Is there an option for errortrace to reduce its time and memory > consumption and still provide the continuation marks for stack trace? > I tried (profiling-record-enabled #f), it did not help. > > Question 3: Is there another way to programmatically get a stack trace for a > dynamically loaded program? Am I using a sledgehammer to crack a nut? > > > Best regards, > > Dmitry > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] errortrace requires 10x time and memory for the program to compile
Hello, I have a program that takes 17 seconds and ~260 MB of memory. If I use errortrace on it, the numbers grow about tenfold: 150 seconds and 2600+ MB. That is just compilation; in the runtime the program does almost nothing and terminates quickly. I know little about how errortrace works and ask for help. My understanding of the problem is shaped into three questions. Question 1. Is it normal for errortrace? Note 1: I am using errortrace just for getting the stack trace of a runtime error: (with-handlers ((exn:fail? (lambda (e) (for ((stack-elem (continuation-mark-set->list (exn-continuation-marks e) errortrace-key))) .. Note 2: The program which stack trace I am interested in is require-d dynamically, and is actually written in non-lispy language. errortrace does a great job in planting its continuation marks into syntax objects that are generated by the parser and the compiler for that language. Question 2: Is there an option for errortrace to reduce its time and memory consumption and still provide the continuation marks for stack trace? I tried (profiling-record-enabled #f), it did not help. Question 3: Is there another way to programmatically get a stack trace for a dynamically loaded program? Am I using a sledgehammer to crack a nut? Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
