this is my post from stackoverflow
but i am not getting any response.
I need to emit a 2D double array as key and value from mapper.There are
questions posted in stackoverflow. But they are not answered.
we have to create a custom datatype.but how?I am new to these custom
datatypes. i dnt have any idea where to start.I am doing some of the matrix
multiplication in a given dataset.and after that i need to emit the value of
A*Atrns which will be a matrix and Atrans*D which will also be a matrix.so
how to emit these matrices from mapper.And the value should be
corresponding to the key itself.I think for that we need to use
WritableComparable.
public class MatrixWritable implements WritableComparable<MatrixWritable>{
/**
* @param args
*/
private double[][] value;
public MatrixWritable() {
// TODO Auto-generated constructor stub
set(new double[0][0]);
}
public MatrixWritable(double[][] value) {
// TODO Auto-generated constructor stub
this.value = value;
}
public void set(double[][] value) {
this.value = value;
}
public double[][] getValue() {
return value;
}
@Override
public void write(DataOutput out) throws IOException {
System.out.println("write");
int row=0;
int col=0;
for(int i=0; i<value.length;i++){
row = value.length;
for(int j=0; j<value[i].length; j++){
col = value[i].length;
}
}
out.writeInt(row);
out.writeInt(col);
System.out.println("\nTotal no of observations: "+row+":"+col);
for(int i=0;i<row ; i++){
for(int j= 0 ; j< col;j++){
out.writeDouble(value[i][j]);
}
}
//priting array
for(int vali =0;vali< value.length ;vali ++){
for(int valj = 0;valj <value[0].length;valj++){
System.out.print(value[vali][valj]+ "\t");
}
System.out.println("");
}
}
@Override
public void readFields(DataInput in) throws IOException {
int row = in.readInt();
int col = in.readInt();
double[][] value = new double[row][col];
for(int i=0;i<row ; i++){
for(int j= 0 ; j< col;j++){
value[i][j] = in.readDouble();
}
}
}
@Override
public int hashCode() {
}
@Override
public boolean equals(Object o) {
}
@Override
public int compareTo(MatrixWritable o) {
// TODO Auto-generated method stub
return 0;
}
@Override
public String toString() {
return Arrays.toString(value);
}
}
I wrote matrix write,readfields and toString.
1.But my toString is not returning anything. why is it so?
2.How to print these values with in Reducer ?I tried doing(tried with
emiting only custom value- for checking)
public class MyReducer extends
Reducer<MatrixWritable, MatrixWritable, IntWritable, Text> {
public void reduce(Iterable<MatrixWritable> key,
Iterable<MatrixWritable> values, Context context){
for(MatrixWritable c : values){
System.out.println("print value "+c.toString());
}
}
but Nothing is printing.when i tried to print value[0].length in toString()
method it showsArrayIndexOutOfBoundExcep.Am i doing any thing wrong.and i
also needed to print my data asmatrix so i tried
public String toString() {
String separator = ", ";
StringBuffer result = new StringBuffer();
// iterate over the first dimension
for (int i = 0; i < value.length; i++) {
// iterate over the second dimension
for(int j = 0; j < value[i].length; j++){
result.append(value[i][j]);
System.out.print(value[i][j]);
result.append(separator);
}
// remove the last separator
result.setLength(result.length() - separator.length());
// add a line break.
result.append("\n");
}
return result.toString();
}
Again my output is empty. 3.Inorder to emit a key too as custom datatype
CompareTo is neccessary right .
4.so what should i include in that methods CompareTo,hashcode,equals and
what are these methods intended for.
Any Idea.Pls suggest a solution.
--
*Thanks & Regards*
*
*
Unmesha Sreeveni U.B*
*
*Junior Developer
*
*Amrita Center For Cyber Security
*
*
Amritapuri.
www.amrita.edu/cyber/
*
