Re: [racket-users] Putting Racket into the database or, more sanely, storing continuations

[David Storrs]

Actually, what I've ended up doing is this:

1) There's a globally accessible jumptable hash that any function is
free to register with.  The hash maps a string key to a procedure.

2) Tasks get stored to the 'tasks' table in the DB as a JSONB blob of
the form (e.g.):  { "function": "release-space"   "args-list": [
"/foo/bar" ] }, along with a timestamp, priority, and a "claimed"
field.

3) There's a worker pool of threads which periodically grab one entry
out of the table and execute it with (apply (jumptable "function")
(json-blob "args-list")).  (Note that I'm using rackjure, so the
'hash-ref' isn't necessary.) The example above it would reduce to
(apply release-space '("/foo/bar"))



It's a little down at the heels, and using a global variable is nasty,
but it works.  Tony's suggestion about MFA is cleaner but
dynamic-require is problematic when use raco exe to build an
executable (which I'll need to do).  Still, I'm glad to see that I
independently made it most of the way to a solution that a lot of
smart people regard as good.

Thanks all -- the input is much appreciated.

Dave

Thanks for the input everyone
On Sat, Oct 29, 2016 at 11:05 AM, Philip McGrath
 wrote:
> If you want to go this way (and I suspect that there may be a better way),
> rather than using eval, I would look at serial-lambda from
> web-server/lang/serial-lambda, which lets you create closures (like the
> values produced by lambda) that could be put into a TEXT field using
> serialize and write. The benefit is that once you read and deserialize, you
> have a value that can be directly applied to arguments or called as a thunk,
> not source code that needs to be evaluated.
>
> For one approach to security, see web-server/stuffers/hmac-sha1. There are
> probably additional considerations in your case, but it would at least
> ensure that your serialized closures in the database are not forged.
>
> But I wonder why you need to save these "tasks" to disk. Unless they need to
> persist across runs of the application, it seems like you might be better
> served using threads (and possibly custodians and/or will executors, which I
> haven't had much cause to play with yet). For example, to do something in 5
> minutes, you could write:
> (thread
>  (λ ()
>(sleep (* 60 5))
>;; do work
>))
> Then you could leave all the work of making sure your "tasks" actually run
> at the appropriate moment to Racket. (In fact, even if some of your tasks do
> need to persist across runs of your application, you could still use this
> approach for actually running them if you include a step in your startup
> sequence to re-spawn any tasks that remain in your database.)
>
> On Sat, Oct 29, 2016 at 6:27 AM Tony Garnock-Jones 
> wrote:
>>
>> On 10/28/2016 08:21 PM, David Storrs wrote:
>> > Is it possible to take (e.g.) a procedure object and decompose it back
>> > into its original source code?
>>
>> I don't believe this is possible without murky unsafe programming, but...
>>
>> > One (bad) idea that came to mind was to simply shove some Racket code
>> > into a TEXT field in the database, then eval it when the time comes.
>>
>> ... this isn't actually so bad. From what you write, I think you're
>> already seeing the potential pitfalls: what should be in scope of the
>> code to be eval'd?
>>
>> > Now, this is horrible for a lot of reasons (security and error
>> > handling being two of them)
>>
>> Security will be a problem no matter what, but I don't see that error
>> handling gives undue difficulty! What am I missing?
>>
>> > suppose I already had a function that did what I needed and I
>> > wanted to use that
>>
>> Instead of storing a list-representing-code-to-eval, you could store
>>
>>  - the name of a module
>>  - the name of a function
>>  - a list of argument values
>>
>> and use `dynamic-require` in your task runner to find the given function:
>>
>>   > (dynamic-require 'racket/list 'filter-map)
>>   #
>>
>> and then `apply` with the arguments...
>>
>> Erlang uses essentially this approach in many places where actually
>> passing a closure around would be problematic (e.g.: code upgrades;
>> serialization to databases; etc). Erlang terminology is to call the
>> triple of module name, function name, and arguments an "MFA".
>>
>> Cheers,
>>   Tony
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Racket Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to racket-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Putting Racket into the database or, more sanely, storing continuations

[Philip McGrath]

If you want to go this way (and I suspect that there may be a better way),
rather than using eval, I would look at serial-lambda from
web-server/lang/serial-lambda, which lets you create closures (like the
values produced by lambda) that could be put into a TEXT field using
serialize and write. The benefit is that once you read and deserialize, you
have a value that can be directly applied to arguments or called as a
thunk, not source code that needs to be evaluated.

For one approach to security, see web-server/stuffers/hmac-sha1
.
There are probably additional considerations in your case, but it would at
least ensure that your serialized closures in the database are not forged.

But I wonder why you need to save these "tasks" to disk. Unless they need
to persist across runs of the application, it seems like you might be
better served using threads (and possibly custodians
 and/or will
executors , which
I haven't had much cause to play with yet). For example, to do something in
5 minutes, you could write:
(thread
 (λ ()
   (sleep (* 60 5))
   ;; do work
   ))
Then you could leave all the work of making sure your "tasks" actually run
at the appropriate moment to Racket. (In fact, even if some of your tasks
do need to persist across runs of your application, you could still use
this approach for actually running them if you include a step in your
startup sequence to re-spawn any tasks that remain in your database.)

On Sat, Oct 29, 2016 at 6:27 AM Tony Garnock-Jones 
wrote:

> On 10/28/2016 08:21 PM, David Storrs wrote:
> > Is it possible to take (e.g.) a procedure object and decompose it back
> > into its original source code?
>
> I don't believe this is possible without murky unsafe programming, but...
>
> > One (bad) idea that came to mind was to simply shove some Racket code
> > into a TEXT field in the database, then eval it when the time comes.
>
> ... this isn't actually so bad. From what you write, I think you're
> already seeing the potential pitfalls: what should be in scope of the
> code to be eval'd?
>
> > Now, this is horrible for a lot of reasons (security and error
> > handling being two of them)
>
> Security will be a problem no matter what, but I don't see that error
> handling gives undue difficulty! What am I missing?
>
> > suppose I already had a function that did what I needed and I
> > wanted to use that
>
> Instead of storing a list-representing-code-to-eval, you could store
>
>  - the name of a module
>  - the name of a function
>  - a list of argument values
>
> and use `dynamic-require` in your task runner to find the given function:
>
>   > (dynamic-require 'racket/list 'filter-map)
>   #
>
> and then `apply` with the arguments...
>
> Erlang uses essentially this approach in many places where actually
> passing a closure around would be problematic (e.g.: code upgrades;
> serialization to databases; etc). Erlang terminology is to call the
> triple of module name, function name, and arguments an "MFA".
>
> Cheers,
>   Tony
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Putting Racket into the database or, more sanely, storing continuations

[Tony Garnock-Jones]

On 10/28/2016 08:21 PM, David Storrs wrote:
> Is it possible to take (e.g.) a procedure object and decompose it back
> into its original source code?

I don't believe this is possible without murky unsafe programming, but...

> One (bad) idea that came to mind was to simply shove some Racket code
> into a TEXT field in the database, then eval it when the time comes.

... this isn't actually so bad. From what you write, I think you're
already seeing the potential pitfalls: what should be in scope of the
code to be eval'd?

> Now, this is horrible for a lot of reasons (security and error
> handling being two of them)

Security will be a problem no matter what, but I don't see that error
handling gives undue difficulty! What am I missing?

> suppose I already had a function that did what I needed and I
> wanted to use that

Instead of storing a list-representing-code-to-eval, you could store

 - the name of a module
 - the name of a function
 - a list of argument values

and use `dynamic-require` in your task runner to find the given function:

  > (dynamic-require 'racket/list 'filter-map)
  #

and then `apply` with the arguments...

Erlang uses essentially this approach in many places where actually
passing a closure around would be problematic (e.g.: code upgrades;
serialization to databases; etc). Erlang terminology is to call the
triple of module name, function name, and arguments an "MFA".

Cheers,
  Tony

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Putting Racket into the database or, more sanely, storing continuations

[David Storrs]

Is it possible to take (e.g.) a procedure object and decompose it back
into its original source code?

Background:

I need a very simple pure-Racket task manager for the app I'm working
on.  This is an application-internal TM, not a for-users TM -- the
sort of tasks it will be handling are "in 5 minutes, release this
block of disk space that we just reserved".  The app will eventually
be installed on end-user machines, so using something like ZeroMQ is
unappealing because it would mean installing additional software.

One (bad) idea that came to mind was to simply shove some Racket code
into a TEXT field in the database, then eval it when the time comes.
Now, this is horrible for a lot of reasons (security and error
handling being two of them), but it got me thinking:  how would I do
this?  I could build a Racket list that happens to be code, then eval
that but suppose I already had a function that did what I needed and I
wanted to use that -- how could I store that?  Some googling and
documentation-searching has not shown me a way to decompose a
procedure.  I played around with some syntax-related calls but made no
headway.

Another (saner) solution would be to serialize a continuation to disk,
the way the web server does, then put some sort of activation trigger
(e.g. a URL) in the database.  I'm not going that route because I
don't understand the process very well and based on the reading I've
done it's got some pitfalls.

So, with the caveat that I'm not actually going to put Racket in the
DB and this is mostly just intellectual curiosity, how would I do
this?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.