I guess it is using the mapred class 11/03/16 20:58:27 INFO mapred.JobClient: Task Id : attempt_201103161245_0005_m_000004_0, Status : FAILED java.io.IOException: Cannot create a record reader because of a previous error. Please look at the previous logs lines from the task's full log for more details. at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:98) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at org.apache.hadoop.mapred.Child$4.run(Child.java:240) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) at org.apache.hadoop.mapred.Child.main(Child.java:234)
How do I use mapreduce class? Viv On Wed, Mar 16, 2011 at 4:52 PM, Ted Yu <[email protected]> wrote: > Since we have lived so long without this information, I guess we can hold > for longer :-) > Another issue I am working on is to reduce memory footprint. See the > following discussion thread: > One of the regionserver aborted, then the master shut down itself > > We have to bear in mind that there would be around 10K regions or more in > production. > > Cheers > > On Wed, Mar 16, 2011 at 1:46 PM, Jeff Whiting <[email protected]> wrote: > > > Just a random thought. What about keeping a per region row count? Then > if > > you needed to get a row count for a table you'd just have to query each > > region once and sum. Seems like it wouldn't be too expensive because > you'd > > just have a row counter variable. It maybe more complicated than I'm > making > > it out to be though... > > > > ~Jeff > > > > > > On 3/16/2011 2:40 PM, Stack wrote: > > > >> On Wed, Mar 16, 2011 at 1:35 PM, Vivek Krishna<[email protected]> > >> wrote: > >> > >>> 1. How do I count rows fast in hbase? > >>> > >>> First I tired count 'test' , takes ages. > >>> > >>> Saw that I could use RowCounter, but looks like it is deprecated. > >>> > >> It is not. Make sure you are using the one from mapreduce package as > >> opposed to mapred package. > >> > >> > >> I just need to verify the total counts. Is it possible to see > somewhere > >>> in > >>> the web interface or ganglia or by any other means? > >>> > >>> We don't keep a current count on a table. Too expensive. Run the > >> rowcounter MR job. This page may be of help: > >> > >> > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#package_description > >> > >> Good luck, > >> St.Ack > >> > > > > -- > > Jeff Whiting > > Qualtrics Senior Software Engineer > > [email protected] > > > > >
