in the typed API, this would be something like:
val input: TypedPipe[(K, (T, T, T, T, T))] = ???
input.group.sum(Semigroup.semigroup5(Semigroup.from { (t1, t2) => fn(t1,
t2) }))
There is not convenient way to do that in the Fields API that I see at the
moment.
On Mon, Aug 21, 2017 at 2:25 PM, <[email protected]> wrote:
> Hi all,
>
> In Field-based API, is there any way to group by a field and apply the
> same reduce method on all other fields?
>
> I'm thinking about something like:
>
> pipe.groupBy(new Fields("fieldName"))(_.reduce(Fields.ALL ->
> Fields.ARGS){ (accum:TupleEntry, next:TupleEntry) =>
> someMethod(accum, next)
> })
>
> the above code gets compiled but I believe it does not generate the output
> I expect (in the output schema in the execution trace I only see the
> groupby field name and nothing else).
>
> As a concrete example, assume the following is your pipe:
>
> field1, field2, field3
> 1 , 2 , 3
> 1 , 1 , 1
> 2, , 10 , 1
>
> I want to group the pipe by field1 and sum up the values in the other
> fields so that the output is:
>
> field1, field2, field3
> 1 , 13 , 5
>
>
> of course the logic that I want to implement in practice is more complex
> than simple summation and yes I don't know how many fields I have in the
> pipe, so, using typed api is not an option.
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Scalding Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
