Check this out:
http://developer.yahoo.com/hadoop/tutorial/module5.html#writable-notes
It shows how to create a customer writable. If you have "hadoop the
definitive guide" there is a really good explanation about custom data
types.
Happy Halloween
On Thu, Oct 31, 2013 at 3:03 PM, unmesha sreeveni <unmeshab...@gmail.com>wrote:
> this is my post from stackoverflow
> but i am not getting any response.
>
>
> I need to emit a 2D double array as key and value from mapper.There are
> questions posted in stackoverflow. But they are not answered.
> we have to create a custom datatype.but how?I am new to these custom
> datatypes. i dnt have any idea where to start.I am doing some of the matrix
> multiplication in a given dataset.and after that i need to emit the value
> of
> A*Atrns which will be a matrix and Atrans*D which will also be a matrix.so
> how to emit these matrices from mapper.And the value should be
> corresponding to the key itself.I think for that we need to use
> WritableComparable.
>
>
>
> public class MatrixWritable implements WritableComparable<MatrixWritable>{
>
> /**
> * @param args
> */
> private double[][] value;
>
> public MatrixWritable() {
> // TODO Auto-generated constructor stub
> set(new double[0][0]);
> }
>
> public MatrixWritable(double[][] value) {
> // TODO Auto-generated constructor stub
> this.value = value;
> }
>
> public void set(double[][] value) {
> this.value = value;
> }
>
> public double[][] getValue() {
> return value;
> }
>
> @Override
> public void write(DataOutput out) throws IOException {
> System.out.println("write");
> int row=0;
> int col=0;
> for(int i=0; i<value.length;i++){
> row = value.length;
> for(int j=0; j<value[i].length; j++){
> col = value[i].length;
> }
> }
> out.writeInt(row);
> out.writeInt(col);
>
> System.out.println("\nTotal no of observations: "+row+":"+col);
>
> for(int i=0;i<row ; i++){
> for(int j= 0 ; j< col;j++){
>
> out.writeDouble(value[i][j]);
> }
> }
> //priting array
> for(int vali =0;vali< value.length ;vali ++){
> for(int valj = 0;valj <value[0].length;valj++){
> System.out.print(value[vali][valj]+ "\t");
> }
> System.out.println("");
> }
>
> }
>
> @Override
> public void readFields(DataInput in) throws IOException {
> int row = in.readInt();
> int col = in.readInt();
>
> double[][] value = new double[row][col];
> for(int i=0;i<row ; i++){
> for(int j= 0 ; j< col;j++){
> value[i][j] = in.readDouble();
>
> }
> }
>
> }
>
> @Override
> public int hashCode() {
>
> }
>
> @Override
> public boolean equals(Object o) {
>
> }
>
>
> @Override
> public int compareTo(MatrixWritable o) {
> // TODO Auto-generated method stub
> return 0;
> }
> @Override
> public String toString() {
>
> return Arrays.toString(value);
>
> }
>
>
>
> }
>
> I wrote matrix write,readfields and toString.
>
> 1.But my toString is not returning anything. why is it so?
>
> 2.How to print these values with in Reducer ?I tried doing(tried with
> emiting only custom value- for checking)
>
> public class MyReducer extends
>
>
> Reducer<MatrixWritable, MatrixWritable, IntWritable, Text> {
>
> public void reduce(Iterable<MatrixWritable> key,
> Iterable<MatrixWritable> values, Context context){
> for(MatrixWritable c : values){
>
> System.out.println("print value "+c.toString());
>
> }
>
> }
>
> but Nothing is printing.when i tried to print value[0].length in toString()
> method it showsArrayIndexOutOfBoundExcep.Am i doing any thing wrong.and i
> also needed to print my data asmatrix so i tried
>
> public String toString() {
>
> String separator = ", ";
> StringBuffer result = new StringBuffer();
>
> // iterate over the first dimension
> for (int i = 0; i < value.length; i++) {
> // iterate over the second dimension
> for(int j = 0; j < value[i].length; j++){
> result.append(value[i][j]);
> System.out.print(value[i][j]);
> result.append(separator);
> }
> // remove the last separator
> result.setLength(result.length() - separator.length());
> // add a line break.
> result.append("\n");
> }
>
>
> return result.toString();
>
>
> }
>
> Again my output is empty. 3.Inorder to emit a key too as custom datatype
> CompareTo is neccessary right .
>
> 4.so what should i include in that methods CompareTo,hashcode,equals and
> what are these methods intended for.
> Any Idea.Pls suggest a solution.
>
> --
> *Thanks & Regards*
> *
> *
> Unmesha Sreeveni U.B*
> *
> *Junior Developer
> *
> *Amrita Center For Cyber Security
> *
> *
> Amritapuri.
>
> www.amrita.edu/cyber/
> *
>
