Implementing the m/r functions in Erlang did the trick.
- Jeremy
On 5/10/2011 9:45 AM, Sean Cribbs wrote:
You might see an error in the log, but there are a lot of things about
the JS VM that are too opaque. It's on our list of things to improve,
for sure.
Sean Cribbs <[email protected] <mailto:[email protected]>>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/
On May 10, 2011, at 9:30 AM, Jeremy Raymond wrote:
Okay thanks. I figure something like this was going on. Besides the
timeout return value is there any way to tell what happened or at
what phase of some chained map reduce operations things failed? Would
I see errors in the logs that indicated a crash of the Javascript
engine or something along those lines?
- Jeremy
On Tue, May 10, 2011 at 9:27 AM, Sean Cribbs <[email protected]
<mailto:[email protected]>> wrote:
For large numbers of objects, your JavaScript functions will
become bottlenecks, largely because the data has to be converted
to and from JSON for every application of the function. If your
reduce function isn't actually coalescing a value (e.g. sort
doesn't), then you will run into this problem. See if you can
reimplement the functions in Erlang or decrease the quantity of
values being passed to and returned from the reduce functions.
Sean Cribbs <[email protected] <mailto:[email protected]>>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/
On May 10, 2011, at 9:20 AM, Jeremy Raymond wrote:
Hello,
In a MapReduce job I have filtering through about 6500 items if
I filter out enough items my m/r job completes successfully. If
I loosen the filtering such that a large number of items would
be included in the results the m/r fails with error
{error,javascript_reduce_timeout}. This error occurs after about
10 seconds of processing. Would Javascript errors such as the
Javascript process running out of memory and crashing result in
this timeout error? I also noticed that after this error is
returned, the riak node I made the request on chugs along at
~100% CPU for about another minute but the other two nodes quite
down almost immediately after the error is returned. How might I
go about debugging this?
- Jeremy
---------------------------------------------------------------------
This transmission (including any attachments) may contain
confidential information, privileged material (including
material protected by the solicitor-client or other applicable
privileges), or constitute non-public information. Any use of
this information by anyone other than the intended recipient is
prohibited. If you have received this transmission in error,
please immediately reply to the sender and delete this
information from your system. Use, dissemination, distribution,
or reproduction of this transmission by unintended recipients is
not authorized and may be
unlawful._______________________________________________
riak-users mailing list
[email protected] <mailto:[email protected]>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected] <mailto:[email protected]>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
