That's exactly what I was looking for. Many thanks Koert, Oscar, and Alex.
On Saturday, July 1, 2017 at 4:29:19 PM UTC-7, koert wrote:
>
> import cascading.tuple.{ Tuple => CTuple, TupleEntry}
>
> pipe.map(Fields.ALL -> Fields.ARGS){ (te: TupleEntry) => ... }
>
> You have access to the field names this way but cannot change then. Your
> function is TupleEntry => CTuple
>
> On Jul 1, 2017 6:36 PM, <[email protected] <javascript:>> wrote:
>
> Thanks Oscar! By function, do you mean a cascading function or I can still
> do this in pure scalding?
>
>
> On Saturday, July 1, 2017 at 12:23:46 PM UTC-7, Oscar Boykin wrote:
>
>> You can do Koert's trick except with a function accepting TupleEntry and
>> you will then have access to the field names.
>>
>> On Fri, Jun 30, 2017 at 14:31 <[email protected]> wrote:
>>
>>> Thanks very much for the suggestion koert. Yes, this does the trick for
>>> me. However, as you mentioned I won't have access to the field names which
>>> I guess is fine. For the scenario that I explained, I can just check the
>>> tuple elements and see if they are of type List or not, if they are I will
>>> invoke the method I want, otherwise I won't touch their values.
>>>
>>> Thanks again!
>>>
>>>
>>> On Friday, June 30, 2017 at 5:01:30 PM UTC-7, koert wrote:
>>>
>>>> you can do an operation on all fields along the lines of:
>>>>
>>>> import cascading.tuple.{ Tuple => CTuple }
>>>> pipe.map(Fields.ALL -> Fields.ARGS){ (ctuple: CTuple) => ... }
>>>>
>>>> however this does not give you access to the field names
>>>>
>>> On Fri, Jun 30, 2017 at 5:13 PM, <[email protected]> wrote:
>>>>
>>> Here is the question:
>>>>>
>>>>> Assume I have a pipe and I want to rename all the fields in the pipe
>>>>> programmatically, meaning that I don't want to hard code the field names
>>>>> in
>>>>> my code. Any idea how I can do this?
>>>>>
>>>>> As a concrete example, assume I have a pipe with two fields: "name"
>>>>> and "age" and I want to rename these fields to "employee_name" and
>>>>> "employee_age". Obviously the natural solution is to write a piece of
>>>>> code
>>>>> as below:
>>>>>
>>>>> pipe.rename(('name, 'age) -> ('employee_name, 'employee_age))
>>>>>
>>>>> or
>>>>>
>>>>> pipe.rename(new Fields("name", "age") -> new Fields("employee_name",
>>>>> "employee_age"))
>>>>>
>>>>> However, what I need is to be able to iterate through all fields in
>>>>> the pipe without knowing their names.
>>>>>
>>>>> There are a couple of methods (resolveIncomingOperationArgumentFields
>>>>> and resolveIncomingOperationPassThroughFields) callable on a pipe which
>>>>> look promising but the issue is that they both take and input argument of
>>>>> type cascading.flow.planner.Scope which I don't know where can I get it
>>>>> from in a scalding job.
>>>>>
>>>>> Another solution that comes to my mind is using "each" method on the
>>>>> pipe and implementing a cascading function and pass it to the each
>>>>> statement. But I was now able to find any sample code for that either.
>>>>>
>>>>> Thanks!
>>>>>
>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Scalding Development" group.
>>>>>
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>
>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Scalding Development" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Scalding Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
