have a mapper that emit key/value pairs(composite keys and composite
values separated by comma).
e.g
*key:* a,b,c,d *Value:* 1,2,3,4,5
*key:* a1,b1,c1,d1 *Value:* 5,4,3,2,1
...
...
*key:* a,b,c,d *Value:* 5,4,3,2,1
I could easily SUM these values using reduceByKey.
e.g.
reduceByKey(new Function2<String, String, String>() {
@Override
public String call(String value1, String value2) {
String oldValue[] = value1.toString().split(",");
String newValue[] = value2.toString().split(",");
int iFirst = Integer.parseInt(oldValue[0]) +
Integer.parseInt(newValue[0]);
int iSecond = Integer.parseInt(oldValue[1]) +
Integer.parseInt(newValue[1]);
int iThird = Integer.parseInt(oldValue[2]) +
Integer.parseInt(newValue[2]);
int iFourth = Integer.parseInt(oldValue[3]) +
Integer.parseInt(newValue[3]);
int iFifth = Integer.parseInt(oldValue[4]) +
Integer.parseInt(newValue[4]);
return iFirst + "," + iSecond + ","
+ iThird+ "," + iFourth+ "," + iFifth;
}
});
But the problem is how do I find average of just one of these values. Lets
assume I want to SUM iFirst, iSecond, iThird and iFourth but I want to find
Average of iFifth. How do i do it? With a simple key/value pairs I could
use mapValues function but not sure how I could do it with my example.
Please advice.
