import cascading.tuple.{ Tuple => CTuple, TupleEntry}
pipe.map(Fields.ALL -> Fields.ARGS){ (te: TupleEntry) => ... }
You have access to the field names this way but cannot change then. Your
function is TupleEntry => CTuple
On Jul 1, 2017 6:36 PM, <[email protected]> wrote:
Thanks Oscar! By function, do you mean a cascading function or I can still
do this in pure scalding?
On Saturday, July 1, 2017 at 12:23:46 PM UTC-7, Oscar Boykin wrote:
> You can do Koert's trick except with a function accepting TupleEntry and
> you will then have access to the field names.
>
> On Fri, Jun 30, 2017 at 14:31 <[email protected]> wrote:
>
>> Thanks very much for the suggestion koert. Yes, this does the trick for
>> me. However, as you mentioned I won't have access to the field names which
>> I guess is fine. For the scenario that I explained, I can just check the
>> tuple elements and see if they are of type List or not, if they are I will
>> invoke the method I want, otherwise I won't touch their values.
>>
>> Thanks again!
>>
>>
>> On Friday, June 30, 2017 at 5:01:30 PM UTC-7, koert wrote:
>>
>>> you can do an operation on all fields along the lines of:
>>>
>>> import cascading.tuple.{ Tuple => CTuple }
>>> pipe.map(Fields.ALL -> Fields.ARGS){ (ctuple: CTuple) => ... }
>>>
>>> however this does not give you access to the field names
>>>
>> On Fri, Jun 30, 2017 at 5:13 PM, <[email protected]> wrote:
>>>
>> Here is the question:
>>>>
>>>> Assume I have a pipe and I want to rename all the fields in the pipe
>>>> programmatically, meaning that I don't want to hard code the field names in
>>>> my code. Any idea how I can do this?
>>>>
>>>> As a concrete example, assume I have a pipe with two fields: "name" and
>>>> "age" and I want to rename these fields to "employee_name" and
>>>> "employee_age". Obviously the natural solution is to write a piece of code
>>>> as below:
>>>>
>>>> pipe.rename(('name, 'age) -> ('employee_name, 'employee_age))
>>>>
>>>> or
>>>>
>>>> pipe.rename(new Fields("name", "age") -> new Fields("employee_name",
>>>> "employee_age"))
>>>>
>>>> However, what I need is to be able to iterate through all fields in the
>>>> pipe without knowing their names.
>>>>
>>>> There are a couple of methods (resolveIncomingOperationArgumentFields
>>>> and resolveIncomingOperationPassThroughFields) callable on a pipe
>>>> which look promising but the issue is that they both take and input
>>>> argument of type cascading.flow.planner.Scope which I don't know where can
>>>> I get it from in a scalding job.
>>>>
>>>> Another solution that comes to my mind is using "each" method on the
>>>> pipe and implementing a cascading function and pass it to the each
>>>> statement. But I was now able to find any sample code for that either.
>>>>
>>>> Thanks!
>>>>
>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Scalding Development" group.
>>>>
>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>
>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Scalding Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
You received this message because you are subscribed to the Google Groups
"Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
