Am I able to get the values from writable of a previous job.
ie I have 2 MR jobs
*MR 1:*
I need to pass 3 element as values from reducer and the key is
NullWritable. So I created a custom writable class to achieve this.
* public class TreeInfoWritable implements Writable{*
* DoubleWritable entropy;*
* IntWritable sum;*
* IntWritable clsCount;*
* ..*
*}*
*MR 2:*
I need to access MR 1 result in MR2 mapper setup function. And I accessed
it as distributed cache (small file).
Is there a way to get those values using get method.
*while ((setupData = bf.readLine()) != null) {*
* System.out.println("Setup Line "+setupData);*
* TreeInfoWritable info = //something i can pass to TreeInfoWritable and
get values*
* DoubleWritable entropy = info.getEntropy();*
* System.out.println("entropy: "+entropy);*
*}*
Tried to convert writable to gson format.
*MR 1*
*Gson gson = new Gson();*
*String emitVal = gson.toJson(valEmit);*
*context.write(out, new Text(emitVal));*
But parsing canot be done in *MR2*.
*TreeInfoWritable info = gson.toJson(setupData, TreeInfoWritable.class);*
*Error: Type mismatch: cannot convert from String to TreeInfoWritable*
Once it is changed to string we cannot get values.
Am I able to get a workaround for the same. or to use just POJO classes
instaed of Writable. I'm afraid if that becomes slower as we are depending
on Java instaed of hadoop 's serializable classes
