Thanks for responding Dan, that's a great trick for getting the right
stuff from ripple to try from curl. Unfortunately I had manually
constructed the same job and tried it from curl with the same result.
If I:
1. Manually do the map only, I get the right output.
2. Take that output (in JSON format) and manually use it as input to
my erlang module from riak_console, it works as expected (and logs).
3. Combine it all via Ripple or Curl and my module is never called (it
has catch-alls to error_logger and they never get evaluated.)
Here's an example of the output of my map (in json) :
>> sugg.to_json
=>
"[[\"10\"],[\"sylvain-interests\",\"soccer\"],[\"3\"],[\"sylvain-interests\",\"cats\"],[\"1\"],[\"sylvain-interests\",\"dogs\"]]"
And how it all chains together:
sugg =
Riak::MapReduce.new(Ripple.client).add('sylvain-interests').map("function(v){return
[[v.values[0].data], [v.bucket, v.key]];}", :keep =>
false).reduce(["get_erl","update_suggestion"], :arg => [], :keep =>
true).run
and my erlang module (get_erl) :
update_suggestion([Data | _], [Bucket, Key]) ->
error_logger:error_msg("I've been called!", []),
[];
update_suggestion(_, _) ->
error_logger:error_msg("I've been called, but did not match! ~p~n", []),
[].
Is there any open source code out there using erlang functions via
ripple or rest that I can look at to see a fully functional flow?
Thanks,
Sylvain
On Thu, Jun 2, 2011 at 1:11 PM, Dan Reverri <[email protected]> wrote:
> Have you tried submitting the map reduce job directly against the HTTP API
> using a tool like curl?
> You can get an idea of what the body should look like by outputting the JSON
> representation of the Ripple MapReduce object:
> puts
> Riak::MapReduce.new(Ripple.client).add(a-bucket').map("function(v){return
> [[v.values[0].data], [v.bucket, v.key]];}", :keep =>
> true).reduce(["my_module","my_function"], :keep => true).to_json
> Thanks,
> Dan
> Daniel Reverri
> Developer Advocate
> Basho Technologies, Inc.
> [email protected]
>
>
> On Thu, Jun 2, 2011 at 1:00 PM, Sylvain Niles <[email protected]>
> wrote:
>>
>> So I've had a few helpful emails, but unfortunately I'm about to write
>> my own erlang HTTP server to call my erlang M/R functions because even
>> examples from the contrib section of the wiki just don't work.
>>
>> My last plea for help before we look at writing a replacement for
>> Riak's REST API:
>>
>> a. Any documentation that shows how to properly deploy your erlang
>> code with the riak cluster (I'm currently writing a module, storing it
>> with riak_core, adding it to riak_core.app, and it's building fine,
>> can call it from riak console.)
>> b. The correct way to call an erlang function from the REST api or via
>> Ripple, specifically we are using the output of a Javascript Map to an
>> Erlang Reduce (If this is not supported, please let us know!)
>>
>>
>> Thanks in advance,
>> Sylvain Niles
>>
>>
>> On Fri, May 27, 2011 at 12:16 PM, Sylvain Niles <[email protected]>
>> wrote:
>> > Still looking for advice on this, additionally I'm looking for the
>> > correct way to call the erlang modules from Ripple. I've tried calling
>> > it from a reduce like so:
>> >
>> > reduced =
>> > Riak::MapReduce.new(Ripple.client).add(a-bucket').map("function(v){return
>> > [[v.values[0].data], [v.bucket, v.key]];}", :keep =>
>> > true).reduce(["my_module","my_function"], :keep => true).run
>> >
>> > and I can see that it never gets called, just returns [].
>> >
>> > I can call the module while attached to a Riak console in the cluster
>> > and it works exactly as expected given the exact output of the
>> > previous map: [["value"], ["bucket", "key"]]
>> >
>> > Any pointers would be greatly appreciated! If there's any open source
>> > code out there using ripple in this fashion I'd love a pointer!
>> >
>> > Thanks,
>> > -Sylvain
>> >
>> > On Tue, May 24, 2011 at 6:55 PM, Sylvain Niles <[email protected]>
>> > wrote:
>> >> So I've seen a few well written examples of erlang map or reduce
>> >> functions in the contrib section of the wiki/github but the missing
>> >> piece of glue for me is: Where do I compile from? I've done a lot of
>> >> ejabberd development and generally I just throw it in the src
>> >> directory, add a config param to the ejabberd.conf to load my new
>> >> module at startup, make install, and I'm done. Should I be deploying
>> >> my modules to /deps/riak_core/src/?
>> >>
>> >> The wiki has this note:
>> >>
>> >> Distributing Erlang MapReduce Code
>> >>
>> >> Any modules and functions you use in your Erlang MapReduce calls must
>> >> be available on all nodes in the cluster. You can add them in Erlang
>> >> applications by specifying the -pz option in vm.args or by adding the
>> >> path to the add_paths setting in app.config.
>> >>
>> >> But the vm.args page does not list the valid config format/options for
>> >> the -pz option. Is the add_paths behavior such that a valid beam in
>> >> that dir will automatically be loaded on startup and all exported
>> >> functions available? What about live code updates of internally
>> >> developed modules?
>> >>
>> >> Thanks in advance!
>> >>
>> >> -Sylvain
>> >>
>> >
>>
>> _______________________________________________
>> riak-users mailing list
>> [email protected]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
