Hi all,
In Field-based API, is there any way to group by a field and apply the same
reduce method on all other fields?
I'm thinking about something like:
pipe.groupBy(new Fields("fieldName"))(_.reduce(Fields.ALL -> Fields.ARGS){
(accum:TupleEntry, next:TupleEntry) =>
someMethod(accum, next)
})
the above code gets compiled but I believe it does not generate the output
I expect (in the output schema in the execution trace I only see the
groupby field name and nothing else).
As a concrete example, assume the following is your pipe:
field1, field2, field3
1 , 2 , 3
1 , 1 , 1
2, , 10 , 1
I want to group the pipe by field1 and sum up the values in the other
fields so that the output is:
field1, field2, field3
1 , 13 , 5
of course the logic that I want to implement in practice is more complex
than simple summation and yes I don't know how many fields I have in the
pipe, so, using typed api is not an option.
Thanks!
--
You received this message because you are subscribed to the Google Groups
"Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
