Thanks,
It worked :) . One more question. When I store Integer values as row keys or
any column values and run scan table from hbase shell, they come like this :
\000\000C| column=year:, timestamp=1227905036961,
value=1999
\000\000C~ column=name:, timestamp=1227905036962,
value=The 39 Steps
\000\000C~ column=yahoo_movie_id:,
timestamp=1227905036962, value=k{I\357\277\275
\000\000C~ column=year:, timestamp=1227905036962,
value=1935
\000\000C\200 column=name:, timestamp=1227905036962,
value=Prophecy
\000\000C\200 column=yahoo_movie_id:,
timestamp=1227905036962, value=k\357\277\275\n@
\000\000C\200 column=year:, timestamp=1227905036962,
value=1979
Notice the first column and value part both of which were integers. Is it
because they get converted to ImmutableBytesWritable that they look like
this ? Can I store them in readable form ?
Thanks
On Fri, Nov 28, 2008 at 3:08 PM, stack <[EMAIL PROTECTED]> wrote:
> How is job being setup? I'd suspect you are calling initTableReduceJob in
> job setup. Look at what it does. It sets the reduce key type. Maybe
> after
> calling it, reset the reduce key type to IntWritable.
> St.Ack
>
>
>
> On Fri, Nov 28, 2008 at 11:48 AM, Nishant Khurana <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> > I am trying to run a map reduce job which parses a text file and fills up
> a
> > Hbase Table. Following is the code :
> >
> >
> > public class UploadMoviesList extends Configured implements Tool
> > {
> > public static class MapClass extends MapReduceBase implements
> > Mapper<LongWritable, Text, IntWritable, MapWritable>
> > {
> > public void map(LongWritable key, Text value,
> > OutputCollector<IntWritable, MapWritable> output, Reporter reporter)
> throws
> > IOException
> > {
> > String line = value.toString();
> > String[] result = line.split("%");
> > MapWritable mw = new MapWritable();
> > mw.put(new Text("year:year"), new Text(result[1].toString()));
> > mw.put(new Text("name:name"), new Text(result[2].toString()));
> > int a = new Integer(result[3]).intValue();
> > mw.put(new Text("y_movie_id:y_movie_id"), new IntWritable(a));
> > int b = new Integer(result[0]).intValue();
> > output.collect(new IntWritable(b), mw);
> > }
> > }
> >
> > public static class ReduceClass extends TableReduce<IntWritable,
> > MapWritable>
> > {
> > @Override
> > public void reduce(IntWritable key, Iterator<MapWritable> values,
> > OutputCollector<ImmutableBytesWritable, BatchUpdate> output, Reporter
> > reporter) throws IOException
> > {
> > reporter.setStatus("Reducer committing " + key);
> > ImmutableBytesWritable ibw = new
> > ImmutableBytesWritable(Bytes.toBytes(key.get()));
> > BatchUpdate outval = new BatchUpdate(Bytes.toBytes(key.get()));
> > while (values.hasNext())
> > {
> > MapWritable hmw = new MapWritable(values.next());
> > outval.put("year:year",
> > Bytes.toBytes(hmw.get("year:year").toString()));
> > outval.put("name:name",
> > Bytes.toBytes(hmw.get("name:name").toString()));
> > IntWritable iw =
> (IntWritable)(hmw.get("y_movie_id:y_movie_id"));
> > outval.put("y_movie_id:y_movie_id", Bytes.toBytes(iw.get()));
> > output.collect(ibw,outval);
> > }
> > }
> > }
> >
> >
> > When I try to run it, I am getting following exceptions :
> > 08/11/28 14:42:27 INFO mapred.JobClient: Task Id :
> > attempt_200811281158_0005_m_000001_0, Status : FAILED
> > java.io.IOException: Type mismatch in key from map: expected
> > org.apache.hadoop.hbase.io.ImmutableBytesWritable, recieved
> > org.apache.hadoop.io.IntWritable
> > at
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
> > at dist_q_data.UploadMoviesList$MapClass.map(UploadMoviesList.java:45)
> > at dist_q_data.UploadMoviesList$MapClass.map(UploadMoviesList.java:1)
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> > at
> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >
> >
> > I don't know why it says it expects a ImmutableBytesWritable key. Any
> > suggestions ?
> > Thanks
> >
> > --
> > Nishant Khurana
> > Candidate for Masters in Engineering (Dec 2009)
> > Computer and Information Science
> > School of Engineering and Applied Science
> > University of Pennsylvania
> >
>
--
Nishant Khurana
Candidate for Masters in Engineering (Dec 2009)
Computer and Information Science
School of Engineering and Applied Science
University of Pennsylvania
