Re: [racket-users] Correctly executing a source file from embedded Racket
On Thursday, July 27, 2017 at 9:01:11 AM UTC-4, Matthew Flatt wrote: > Declaring (as opposed to instantiating) a compiled module will normally > not raise an exception. Probably it's possible to construct a set of > embedded modules where there will be a declare-time error due to > conflicting or missing declarations, but I don't see how to make that > happen only sometimes. Good to know. > The escape-catching pattern is needed anywhere that you don't want to > just exit/crash. > > You can certainly call multiple `scheme_...` functions within a single > escape-handling block, including multiple calls to `scheme_eval_string`. Also good to know, thanks. On Wednesday, July 26, 2017 at 11:09:48 AM UTC-4, Matthew Flatt wrote: > At Wed, 26 Jul 2017 07:54:32 -0700 (PDT), Thomas Dickerson wrote: > > One more thing: in terms of repeatedly executing scripts, does it make > > sense > > to set up and tear down the interpreter every time? Or just swap in a fresh > > namespace? > > Between those two options, a fresh namespace is almost certainly > better. For posterity, it's worth noting that you can use the Boost Coroutine library to implement this in C++ in a nice object-oriented fashion to make a loop that executes Racket tasks in a way that doesn't require bouncing your `main` through `scheme_main_setup` or equivalent, which is great if you're embedding a racket interpreter to be reused as an API rather than a one-off program. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
At Wed, 26 Jul 2017 14:44:05 -0700 (PDT), Thomas Dickerson wrote: > Looking at this code specifically: > > | mz_jmp_buf * volatile save, fresh; > | save = scheme_current_thread->error_buf; > | scheme_current_thread->error_buf = &fresh; > | > | if (scheme_setjmp(scheme_error_buf)) { > |/* There was an error or continuation invocation */ > |if (scheme_jumping_to_continuation) { > | /* It was a continuation jump */ > | scheme_longjmp(*save, 1); > | > | /* To block the jump, instead: scheme_clear_escape(); */ > |} else { > | /* It was a primitive error escape */ > |} > | } else { > |scheme_eval_string("x", scheme_env); > | } > | scheme_current_thread->error_buf = save; > > I have questions about a few things: > (a) Why does the example check for an error *before* calling > `scheme_eval_string`, rather than afterward? Is this just something weird > with > setjmp/longjmp? Yes, that's how setjmp() and longjmp() work. The initial call to setjmp() always returns 0, etc. > (b) How do I handle the "primitive error escape" branch, assuming I > want to print a useful error message, and continue executing > gracefully? It looks like `scheme_clear_escape` may be useful for > preventing the exception from propagating, but I'm not sure about > that from the documentation (or how I'd be able to capture the error > message from the exception). That branch corresponds to `abort-current-continuation`. If it's due to an exception, then the exception was already printed and (probably) the default error escape handler called `abort-current-continuation`. Catching the value raised as an exception is not practical to do in C. It's better to wrap up the work like `scheme_eval_string("x", scheme_env)` in a thunk and call a Racket-implemented wrapper. Section 9.1 in "Inside Racket" shows one way to do that. > (c) Is this style of error handling only necessary around blocks that > are evaluating actual Racket code (via, e.g., `scheme_eval_string` or > `scheme_dynamic_require`)? What about the > `scheme_register_embedded_load` and `scheme_embedded_load` pair at > the bottom of `declare_modules`? I ran my binary through the Unix > `strings` command and the embedded data looks an awful lot like > expanded S-expressions in plain text. Additionally, at what level of > granularity should I structure the error handling? Does it need to be > for *every* evaluation, individually? Or could I surround a whole > block of evaluations with the same error handling code? Declaring (as opposed to instantiating) a compiled module will normally not raise an exception. Probably it's possible to construct a set of embedded modules where there will be a declare-time error due to conflicting or missing declarations, but I don't see how to make that happen only sometimes. The escape-catching pattern is needed anywhere that you don't want to just exit/crash. You can certainly call multiple `scheme_...` functions within a single escape-handling block, including multiple calls to `scheme_eval_string`. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
On Wednesday, July 26, 2017 at 11:09:48 AM UTC-4, Matthew Flatt wrote: > At Wed, 26 Jul 2017 07:54:32 -0700 (PDT), Thomas Dickerson wrote: > > One more thing: in terms of repeatedly executing scripts, does it make > > sense > > to set up and tear down the interpreter every time? Or just swap in a fresh > > namespace? > > Between those two options, a fresh namespace is almost certainly > better. Thanks. > Depending on how much you need to reset, you might be even better off > with `dynamic-rerequire` from `racket/rerequire`. Interesting. When would a rerequire be necessary, over a require? `dynamic-require` documentation says "Dynamically instantiates the module specified by mod in the current namespace’s registry at the namespace’s base phase, if it is not yet instantiated." Is a module instantiation local to the current namespace? If so, it sounds like wiping the namespace is sufficient. On Wednesday, July 26, 2017 at 12:27:09 PM UTC-4, Shu-Hung You wrote: > Hi Thomas, > > I tried following Matthew's suggestions to attach my-lang and use > dynamic-require. Turns out that works pretty well! And then there's > even no need to setup collection path since everything loads from > compiled byte-code. > https://gist.github.com/shhyou/aa2adaf7e1b7d548783cee352c3230a9 > > The code is basically calling (dynamic-require `(file ,argv-1) #f) to > run the external script. At line 35, the code builds (list 'file > argv-1) and at line 41 the code dynamic-requires the script. > > /* custom_dsl.cpp > > $ raco ctool --cgc --c-mods base.c ++lib racket/base ++lib > racket/base/lang/reader ++lib racket/runtime-config ++lib my-lang > ++lib my-lang/lang/reader > > $ g++ -framework Foundation -I/racket/include > -L/racket/src/build/racket > -L/racket/src/build/rktio custom_dsl.cpp -o > custom_dsl -lracket -lmzgc -lrktio -liconv > > $ ./custom_dsl ./a-module.rkt > > */ > #include > #include > > #include "base.c" > > static int run(Scheme_Env *e, int argc, char *argv[]) > { > Scheme_Object *curout; > int i; > Scheme_Thread *th; > mz_jmp_buf * volatile save, fresh; > > /* Declare embedded modules in "base.c": */ > declare_modules(e); > scheme_namespace_require(scheme_intern_symbol("my-lang")); > > curout = scheme_get_param(scheme_current_config(), > MZCONFIG_OUTPUT_PORT); > > th = scheme_get_current_thread(); > > save = th->error_buf; > th->error_buf = &fresh; > if (scheme_setjmp(*th->error_buf)) { > th->error_buf = save; > return -1; /* There was an error */ > } else { > std::cout << "Trying to load file from \"" << argv[1] << "\"\n"; > Scheme_Object *custom_dsl_path[2]; > custom_dsl_path[0] = scheme_make_pair( >scheme_intern_symbol("file"), >scheme_make_pair( > scheme_make_utf8_string(argv[1]), > scheme_make_null())); > custom_dsl_path[1] = scheme_make_false(); > Scheme_Object *custom_dsl = scheme_dynamic_require(2, custom_dsl_path); > th->error_buf = save; > } > return 0; > } Thanks for the continued help with this! I found that once I switched to using `dynamic-require` I didn't even need the initial `scheme_namespace_require` The core of my script execution code now looks like this: | ret = boost::none; | Scheme_Object *drArgs[2] = {scheme_make_path(filename), resultSym_}; | Scheme_Object *loadResult = scheme_dynamic_require(2, drArgs); | if(SCHEME_CPTRP(loadResult)){ | Scheme_Object *resT = SCHEME_CPTR_TYPE(loadResult); | if(acceptedTypeTag_ == resT){ | ret = (AcceptedType*)SCHEME_CPTR_VAL(loadResult); | } else { | std::cerr << "Wrong type-tag received" << std::endl; | Scheme_Object *curout = scheme_get_param(scheme_current_config(), MZCONFIG_ERROR_PORT); | std::cerr << "Got: "; | scheme_display(resT, curout); | scheme_display(scheme_make_char('\n'), curout); | std::cerr << "Wanted: "; | scheme_display(acceptedTypeTag_, curout); | scheme_display(scheme_make_char('\n'), curout); | } | } else { | std::cerr << "Result was not a cptr" << std::endl; | } I'm now working to understand the exception handling mechanism (reading through https://docs.racket-lang.org/inside/exceptions.html and the example code). The documentation appears to be mostly centered on what to do if your C code is being called from Racket (rather than vice-versa). Looking at this code specifically: | mz_jmp_buf * volatile save, fresh; | save = scheme_current_thread->error_buf; | sch
Re: [racket-users] Correctly executing a source file from embedded Racket
Hi Thomas, I tried following Matthew's suggestions to attach my-lang and use dynamic-require. Turns out that works pretty well! And then there's even no need to setup collection path since everything loads from compiled byte-code. https://gist.github.com/shhyou/aa2adaf7e1b7d548783cee352c3230a9 The code is basically calling (dynamic-require `(file ,argv-1) #f) to run the external script. At line 35, the code builds (list 'file argv-1) and at line 41 the code dynamic-requires the script. /* custom_dsl.cpp $ raco ctool --cgc --c-mods base.c ++lib racket/base ++lib racket/base/lang/reader ++lib racket/runtime-config ++lib my-lang ++lib my-lang/lang/reader $ g++ -framework Foundation -I/racket/include -L/racket/src/build/racket -L/racket/src/build/rktio custom_dsl.cpp -o custom_dsl -lracket -lmzgc -lrktio -liconv $ ./custom_dsl ./a-module.rkt */ #include #include #include "base.c" static int run(Scheme_Env *e, int argc, char *argv[]) { Scheme_Object *curout; int i; Scheme_Thread *th; mz_jmp_buf * volatile save, fresh; /* Declare embedded modules in "base.c": */ declare_modules(e); scheme_namespace_require(scheme_intern_symbol("my-lang")); curout = scheme_get_param(scheme_current_config(), MZCONFIG_OUTPUT_PORT); th = scheme_get_current_thread(); save = th->error_buf; th->error_buf = &fresh; if (scheme_setjmp(*th->error_buf)) { th->error_buf = save; return -1; /* There was an error */ } else { std::cout << "Trying to load file from \"" << argv[1] << "\"\n"; Scheme_Object *custom_dsl_path[2]; custom_dsl_path[0] = scheme_make_pair( scheme_intern_symbol("file"), scheme_make_pair( scheme_make_utf8_string(argv[1]), scheme_make_null())); custom_dsl_path[1] = scheme_make_false(); Scheme_Object *custom_dsl = scheme_dynamic_require(2, custom_dsl_path); th->error_buf = save; } return 0; } int main(int argc, char **argv) { std::cout << "In cpp main\n"; return scheme_main_setup(1, run, argc, argv); } ;; a-module #lang my-lang some 'DSL that turns everything into "quotes" Cheers, Shu-Hung On Wed, Jul 26, 2017 at 10:09 AM, Matthew Flatt wrote: > At Wed, 26 Jul 2017 07:54:32 -0700 (PDT), Thomas Dickerson wrote: >> One more thing: in terms of repeatedly executing scripts, does it make sense >> to set up and tear down the interpreter every time? Or just swap in a fresh >> namespace? > > Between those two options, a fresh namespace is almost certainly > better. > > Depending on how much you need to reset, you might be even better off > with `dynamic-rerequire` from `racket/rerequire`. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
At Wed, 26 Jul 2017 07:54:32 -0700 (PDT), Thomas Dickerson wrote: > One more thing: in terms of repeatedly executing scripts, does it make sense > to set up and tear down the interpreter every time? Or just swap in a fresh > namespace? Between those two options, a fresh namespace is almost certainly better. Depending on how much you need to reset, you might be even better off with `dynamic-rerequire` from `racket/rerequire`. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
Thanks for the help, a couple more quick questions: On Wednesday, July 26, 2017 at 9:15:12 AM UTC-4, Matthew Flatt wrote: > You don't have to populate the top-level environment with > `racket/base`. You could instead use > > scheme_namespace_require(scheme_intern_symbol("my-lang")) > > where "my-lang" is your language's main module to populate the > environment from there. Still, this isn't really the right direction. I actually tried this without much success, but I might still have been missing something. > Overall, I think you want to `scheme_dynamic_require` instead of > `scheme_load`, just as Shu-Hung suggests. Using > `scheme_dynamic_require` corresponds to what `racket` does where you > provide it a file on the command line, and it avoids the top-level > environment and `load`. Just tried this, and it worked great, thanks! I was having a little trouble getting the `scheme_eval` variant that constructed a `dynamic-require` form manually to work correctly, so knowing about `scheme_dynamic_require` is helpful. > I can see how the "Inside Racket" example lead you astray by showing a > traditional REPL, where `eval` and `load` come into play, and I'll look > at ways to improve the docs. But if your goal is to run modules, then > you can avoid all that trouble. Thanks! One more thing: in terms of repeatedly executing scripts, does it make sense to set up and tear down the interpreter every time? Or just swap in a fresh namespace? -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
At Tue, 25 Jul 2017 19:08:02 -0700 (PDT), Thomas Dickerson wrote: > On Tuesday, July 25, 2017 at 5:52:45 PM UTC-4, Shu-Hung You wrote:> > > As we can see, > `scheme_namespace_require(scheme_intern_symbol("racket/base"));` > > is actually missing from your setup program. This line requires racket > > base and the usual #%app bindings. It's probably needed even if you're > > loading your own DSL -- I suspect at some point racket/base will > > install the right `load` function to make things work, and that this > > will also affect loading custom #langs. > > You appear to be correct, but it strikes me as really odd that > (a) In a language which prides itself on metaprogramming, the interpreter > namespace needs to be manually bootstrapped with a specific module. > (b) That the form the error takes is not an error with the requires/provides > in the `main.rkt` for my #lang, but as an error in a module which is using it. I think the root problem is that you're thinking of `load` as `dynamic-require`, but those are different. The `load` function, including its C wrapper `scheme_load`, loads a top-level sequence of forms in the top-level environment. For that to work, the top-level environment needs to be populated with bindings. You don't have to populate the top-level environment with `racket/base`. You could instead use scheme_namespace_require(scheme_intern_symbol("my-lang")) where "my-lang" is your language's main module to populate the environment from there. Still, this isn't really the right direction. Meanwhile, since the top-level environment was empty, the message didn't come from the inside of a module. There was no `#%top-interaction` or `module` binding in the top-level environment, so nothing in "gpsm-test.gpsm" was parsed as a module. Instead, the error was attributed to a top-level form in "gpsm-test.gpsm" that could not be given any meaning in the current top-level environment. Overall, I think you want to `scheme_dynamic_require` instead of `scheme_load`, just as Shu-Hung suggests. Using `scheme_dynamic_require` corresponds to what `racket` does where you provide it a file on the command line, and it avoids the top-level environment and `load`. I can see how the "Inside Racket" example lead you astray by showing a traditional REPL, where `eval` and `load` come into play, and I'll look at ways to improve the docs. But if your goal is to run modules, then you can avoid all that trouble. Going back to the original message on `++lib`, to add a little to Shu-Hung's reply: > I noticed some oddities when using `raco ctool`. For example, the > documentation claims that a module specified for embedding with ++lib will > embed all of its dependencies, but my #lang's reader submodule needed to be > explicitly specified with its own ++lib argument. A language implementation normally would not depend on that language's reader. If you have #lang racket 'hello (module reader racket) then the enclosing module doesn't import the `reader` submodule, so it doesn't depend on it. (If you compile and load that module, then the `reader` submodule isn't event loaded.) So, if I understand the situation you're describing, that's why the reader submodule didn't get pulled along when embedding the outer module. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
On Tuesday, July 25, 2017 at 5:52:45 PM UTC-4, Shu-Hung You wrote:> > As we can see, > `scheme_namespace_require(scheme_intern_symbol("racket/base"));` > is actually missing from your setup program. This line requires racket > base and the usual #%app bindings. It's probably needed even if you're > loading your own DSL -- I suspect at some point racket/base will > install the right `load` function to make things work, and that this > will also affect loading custom #langs. You appear to be correct, but it strikes me as really odd that (a) In a language which prides itself on metaprogramming, the interpreter namespace needs to be manually bootstrapped with a specific module. (b) That the form the error takes is not an error with the requires/provides in the `main.rkt` for my #lang, but as an error in a module which is using it. This code: |#lang racket |(require "Value-Forms.rkt" | "Operator-Forms.rkt" | "Variable-Forms.rkt") | |(provide | #%module-begin | #%top-interaction | #%datum | #%app | (all-from-out "Value-Forms.rkt" | "Operator-Forms.rkt" | "Variable-Forms.rkt")) Should definitely complain if its imports aren't available. > * Dependencies for main.rkt and lang/reader.rkt: I don't think there's > a conflict. The dependencies seem to refer to `require`s, like > racket/base -> racket/list, racket/... Relatedly, even if `main.rkt` and `lang/reader.rkt` are independent, it's really odd that my module beginning with `#lang racket` is insufficient for raco to pick up the racket dependencies. > * load: instead of using load, I would probably try to implement a > function in Racket that use `dynamic-require` to load other DSL > scripts. Then in the C part, we can `dynamic-require` that function to > load argv[1]. Of course, remember to add that module in raco ctool. An > example is given in load-file.rkt and call_racket_from_load.cpp in the > link. > > I am not familiar with this; there could be a better way to > dynamically load a file. > Thanks, I'll think some more about this. > # Commands: > $ raco ctool --c-mods base.c ++lib racket/base ++lib > racket/base/lang/reader ++lib racket/runtime-config This appears to be precisely the necessary set of minimum ++libs. It *really* ought to be documented somewhere, in big bold letters. Thanks for the help! ~Thomas -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Correctly executing a source file from embedded Racket
Hi Thomas, I think the program is more of Racket programming setup rather than embedding Racket. When start a Racket program on our own, we have to setup the environment correctly. Here is my test program that can load an external program and the accompanying commands (also attached at the end). It has been tested the program on Mac OS X 10.11 with Racket head. https://gist.github.com/shhyou/e424424236d1bc685fa002192221c29b The program can also work for user-defined languages by adding ++lib some-lang ++lib some-lang/lang/reader as usual. Here are some thoughts: * That error message looks like trying to run a program from an empty namespace. For example, $ cat a-module.rkt #lang racket/base 123 $ cat run-from-empty-ns.rkt #lang racket (define host-ns (current-namespace)) (define ns (make-empty-namespace)) (parameterize ([current-namespace ns]) (namespace-attach-module host-ns ''#%builtin) (load "a-module.rkt")) $ racket run-from-empty-ns.rkt a-module.rkt:2:0: #%top-interaction: unbound identifier; also, no #%app syntax transformer is bound at: #%top-interaction in: As we can see, `scheme_namespace_require(scheme_intern_symbol("racket/base"));` is actually missing from your setup program. This line requires racket base and the usual #%app bindings. It's probably needed even if you're loading your own DSL -- I suspect at some point racket/base will install the right `load` function to make things work, and that this will also affect loading custom #langs. * Dependencies for main.rkt and lang/reader.rkt: I don't think there's a conflict. The dependencies seem to refer to `require`s, like racket/base -> racket/list, racket/... * load: instead of using load, I would probably try to implement a function in Racket that use `dynamic-require` to load other DSL scripts. Then in the C part, we can `dynamic-require` that function to load argv[1]. Of course, remember to add that module in raco ctool. An example is given in load-file.rkt and call_racket_from_load.cpp in the link. I am not familiar with this; there could be a better way to dynamically load a file. Cheers, Shu-Hung # Commands: $ raco ctool --c-mods base.c ++lib racket/base ++lib racket/base/lang/reader ++lib racket/runtime-config $ g++ -framework Foundation -I/racket/include -L/racket/src/build/racket -L/racket/src/build/rktio call_racket.cpp -o call_racket -lracket -lmzgc -lrktio -liconv ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame $ ./call_racket "/racket/collects" In cpp main Trying to load file from "a-module.rkt" # In cpp: Instantiating the loaded module, `a-module`: In Racket: module instantiated: ("this is in a function with" arg "being the first argument") 12354 In cpp: get the value of `a-value`: a-value is 12354 ;; a-module.rkt #lang racket/base (provide (all-defined-out)) (define (a-binding-to-a-function x) (list "this is in a function with" x "being the first argument")) (printf "In Racket: module instantiated: ~s\n" (a-binding-to-a-function 'arg)) (define a-value 12354) ;; By default, this should print 12354 a-value /* call_racket.cpp */ /* raco ctool --c-mods base.c ++lib racket/base ++lib racket/base/lang/reader ++lib racket/runtime-config g++ -framework Foundation -I/racket/include -L/racket/src/build/racket -L/racket/src/build/rktio call_racket.cpp -o call_racket -lracket -lmzgc -lrktio -liconv ./call_racket "/racket/collects" */ #include #include #include "base.c" static int run(Scheme_Env *e, int argc, char *argv[]) { Scheme_Object *curout; int i; Scheme_Thread *th; mz_jmp_buf * volatile save, fresh; /* Declare embedded modules in "base.c": */ declare_modules(e); scheme_namespace_require(scheme_intern_symbol("racket/base")); curout = scheme_get_param(scheme_current_config(), MZCONFIG_OUTPUT_PORT); th = scheme_get_current_thread(); Scheme_Object *path = scheme_make_path(argv[1]); { scheme_set_collects_path(path); /* // init collection paths with `((dynamic-require 'racket/base 'list) argv-1) Scheme_Object *rkt_list_sym[2], *args[1]; rkt_list_sym[0] = scheme_intern_symbol("racket/base"); rkt_list_sym[1] = scheme_intern_symbol("list"); args[0] = path; Scheme_Object *paths = scheme_apply(scheme_dynamic_require(2, rkt_list_sym), 1, args); scheme_init_collection_paths(e, paths); */ scheme_init_collection_paths(e, scheme_null); } save = th->error_buf; th->error_buf = &fresh; if (scheme_setjmp(*th->error_buf)) { th->error_buf = save; return -1; /* There was an error */ } else { std::cout << "Trying to load file from \"a-module.rkt\"\n"; Scheme_Object *load_result = scheme_load("a-module.rkt"); scheme_display(load_result, curout); scheme_display(scheme_make_char('\n'), curout); // Instantiate the loaded module std::cout << "\nIn cpp: Instantiating the loaded module, `a-module`:\n"; scheme_eval_string("(dynami