Thanks Oscar! By function, do you mean a cascading function or I can still do this in pure scalding?
On Saturday, July 1, 2017 at 12:23:46 PM UTC-7, Oscar Boykin wrote: > > You can do Koert's trick except with a function accepting TupleEntry and > you will then have access to the field names. > > On Fri, Jun 30, 2017 at 14:31 <[email protected] <javascript:>> wrote: > >> Thanks very much for the suggestion koert. Yes, this does the trick for >> me. However, as you mentioned I won't have access to the field names which >> I guess is fine. For the scenario that I explained, I can just check the >> tuple elements and see if they are of type List or not, if they are I will >> invoke the method I want, otherwise I won't touch their values. >> >> Thanks again! >> >> >> On Friday, June 30, 2017 at 5:01:30 PM UTC-7, koert wrote: >> >>> you can do an operation on all fields along the lines of: >>> >>> import cascading.tuple.{ Tuple => CTuple } >>> pipe.map(Fields.ALL -> Fields.ARGS){ (ctuple: CTuple) => ... } >>> >>> however this does not give you access to the field names >>> >> On Fri, Jun 30, 2017 at 5:13 PM, <[email protected]> wrote: >>> >> Here is the question: >>>> >>>> Assume I have a pipe and I want to rename all the fields in the pipe >>>> programmatically, meaning that I don't want to hard code the field names >>>> in >>>> my code. Any idea how I can do this? >>>> >>>> As a concrete example, assume I have a pipe with two fields: "name" and >>>> "age" and I want to rename these fields to "employee_name" and >>>> "employee_age". Obviously the natural solution is to write a piece of code >>>> as below: >>>> >>>> pipe.rename(('name, 'age) -> ('employee_name, 'employee_age)) >>>> >>>> or >>>> >>>> pipe.rename(new Fields("name", "age") -> new Fields("employee_name", >>>> "employee_age")) >>>> >>>> However, what I need is to be able to iterate through all fields in the >>>> pipe without knowing their names. >>>> >>>> There are a couple of methods (resolveIncomingOperationArgumentFields >>>> and resolveIncomingOperationPassThroughFields) callable on a pipe which >>>> look promising but the issue is that they both take and input argument of >>>> type cascading.flow.planner.Scope which I don't know where can I get it >>>> from in a scalding job. >>>> >>>> Another solution that comes to my mind is using "each" method on the >>>> pipe and implementing a cascading function and pass it to the each >>>> statement. But I was now able to find any sample code for that either. >>>> >>>> Thanks! >>>> >>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Scalding Development" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>> >>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Scalding Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Scalding Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
