Re: Erlang reduce functions

[david martin]

On 28/05/12 10:54, Jeremy Smith wrote:

Thank you, that works perfectly.

I guess the only way to understand how this works is for me to understand more 
about Erlang, if you have any pointers/tips for learning Erlang I would be very 
grateful (I come from a Perl, C, Java background)

In my humble opinion, your map function shows you have a very good 
understanding of Erlang for having only started yesterday.

Your problem was not fully understanding the Couchdb reduce function.
From what I have gathered, it has two possible phases of operation, "normal" reduce and 
"re-reduce" reduce.
Whether it is in normal reduce or re-reduce reduce is indicated by the third 
parameter being true or false.

I have been experimenting in the area of Erlang view functions and am pleased 
to say that, so far,
all the Erlang function modules that Couchdb uses are available in the Erlang 
View function.
The function can be debugged using the built in Log("string") Function combined 
with formatting with  statements like

 Log(io_lib:format("X = ~p Val = ~p", [ Key,Value]))

Compile and run time errors are (very) fully  reported in the couch.log

I am using an even  balance of six cores without any specific attempts to use 
explicit parallel maps.

The use of parallel maps would however enable heavy processing  of things such 
as serious crypto functions
i.e. sha512  or complex Perl Compatible Regular Expressions (PCRE)
from the re module to be run in parallel over different Key/Values for each 
Document in the View.

Try that in Javascript!

Very long serial computations can cause the gen-server handling the view to
time out to stop runaway processes in the View code.

http://blog.vmoroz.com/2011/01/erlang-pmap.html


The problem of not seeming to allow recursive calls in the view function has to 
be
addressed by the use of the Y-combinator

http://blog.vmoroz.com/2010/10/recursive-anonymous-function-in-erlang.html

thanks to Victor Moroz for erudite explanations, well worth taking the time to 
understand (if that is possible!).

I believe that the true versatility and power of Couchdb can only be realised
using Erlang Views and there is very little in the way of examples on the web.

If there are any real specific interest cases I might even post some examples 
somewhere.

Many thanks
Jeremy


________________________________
  From: david martin<david.mar...@lymegreen.co.uk>
To: user@couchdb.apache.org
Sent: Monday, 28 May 2012, 9:40
Subject: Re: Erlang reduce functions

try this
fun(Keys,Values,ReReduce)->
      case ReReduce of
          true->lists:sum(Values);
          false->length(Values)
      end
end.


On 27/05/12 16:33, Jeremy Smith wrote:

Hi,

I am new to erlang, just started today, because we were advised to move some of 
our views to native erlang views, I am trying to write a reduce function:

The Values its using seem to be in the form:
[[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1]]


so I have tried to do the following to get a count of all the 1's, but it only 
gives me the count of the outermost lists (in the above case 3, instead of 
60(ish))

My Reduce funtion:
fun(Keys, Values, ReReduce) ->   lists:flatlength(Values) end.


My Map function looks like this:
fun({Doc}) ->
       Emitter = fun(Doc) ->
           Date           = proplists:get_value(<<"publish_date">>,   Doc),

           Year = string:substr(erlang:bitstring_to_list(Date), 1, 4),
           Emit(erlang:list_to_bitstring(Year), 1)
       end,

       HasRequiredFields = fun(Doc) ->
           case {proplists:is_defined(<<"a_number">>, Doc), proplists:is_defined(<<"publish">>, Doc), 
proplists:is_defined(<<"publish_date">>, Doc)} of
               {true, true, true} ->
                   Emitter(Doc);

               _->
                   false
           end
       end,

       HasRequiredFields(Doc)
end.

Cheers
Jeremy