Hi
I'm starting with this of hadoop and Pig, I have to pass a hadoop MapReduce
program that i made to Pig, in the hadoop program I have just a Map function
and on it I perform all the process
that consists to analize some text... to this 9 functions (operators) are
called, this functions run in a secuencial mode (when the first is done, the
second is started and so on), here is how map looks:
static class Map extends Mapper<LongWritable, Text, Text,
IntWritable>{
//declaration of operators or functions
Operator1 op1 = new Operator1();
Operator2 op2 = new Operator2();
Operator3 op3 = new Operator3();
...
...
/*map function
*/
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException{
//get a row from csv
String line = value.toString();
//some code to parse the line
...
...
//initialize all the operators if they are not
initialized
if( !op1.isInitialized() )
op1.initialize();
if( !op2.isInitialized() )
op2.initialize();
...
...//and so on with all operators
//process each operator
op1.process(line);
String[] resultOP1 = op1.getResults();
op2.process(resultOP1);
String[][] resultOP2 = op2.getResults();
...//and so on with all the operators
...
//finally collect results
String put = "";
for( int k = 0 ; k < resultOP9.length ; k++
){
for( int j = 0; j < resultOP9[k].length;
j++ ){
context.write...
}
}
}
}
}
My question is if its a good idea or if there is a way to pass this type of
program to Pig?
Thanks
--
*Cornelio*
