[
https://issues.apache.org/jira/browse/HBASE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214283#comment-13214283
]
[email protected] commented on HBASE-5166:
------------------------------------------------------
bq. On 2012-02-22 17:53:12, Ted Yu wrote:
bq. >
/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java,
line 114
bq. > <https://reviews.apache.org/r/3995/diff/2/?file=78619#file78619line114>
bq. >
bq. > Should read 'MultithreadedTableMapper instances'
done!
bq. On 2012-02-22 17:53:12, Ted Yu wrote:
bq. >
/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java,
line 155
bq. > <https://reviews.apache.org/r/3995/diff/2/?file=78619#file78619line155>
bq. >
bq. > Can we give better progress information here ?
I am not sure how to do it. It would be possible if I can access underlying
RecorderReaader/Writer passed to jobContext and simply calling there
getProgress. Could anybody help me here ?
bq. On 2012-02-22 17:53:12, Ted Yu wrote:
bq. >
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java,
line 223
bq. > <https://reviews.apache.org/r/3995/diff/2/?file=78620#file78620line223>
bq. >
bq. > This if block can be an else to the if block above.
done
- Jai
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3995/#review5268
-----------------------------------------------------------
On 2012-02-23 04:17:08, Jai Singh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3995/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-23 04:17:08)
bq.
bq.
bq. Review request for hbase, Ted Yu and Michael Stack.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. There is no MultiThreadedTableMapper in hbase currently just like we have
a MultiThreadedMapper in Hadoop for IO Bound Jobs.
bq. UseCase, webcrawler: take input (urls) from a hbase table and put the
content (urls, content) back into hbase.
bq. Running these kind of hbase mapreduce job with normal table mapper is
quite slow as we are not utilizing CPU fully (N/W IO Bound).
bq.
bq. Moreover, I want to know whether It would be a good/bad idea to use HBase
for these kind of usecases ?.
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java
PRE-CREATION
bq.
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3995/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Jai
bq.
bq.
> MultiThreaded Table Mapper analogous to MultiThreaded Mapper in hadoop
> ----------------------------------------------------------------------
>
> Key: HBASE-5166
> URL: https://issues.apache.org/jira/browse/HBASE-5166
> Project: HBase
> Issue Type: Improvement
> Reporter: Jai Kumar Singh
> Priority: Minor
> Labels: multithreaded, tablemapper
> Attachments: 0001-Added-MultithreadedTableMapper-HBASE-5166.patch,
> 0003-Added-MultithreadedTableMapper-HBASE-5166.patch,
> 0005-HBASE-5166-Added-MultithreadedTableMapper.patch,
> 0006-HBASE-5166-Added-MultithreadedTableMapper.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There is no MultiThreadedTableMapper in hbase currently just like we have a
> MultiThreadedMapper in Hadoop for IO Bound Jobs.
> UseCase, webcrawler: take input (urls) from a hbase table and put the content
> (urls, content) back into hbase.
> Running these kind of hbase mapreduce job with normal table mapper is quite
> slow as we are not utilizing CPU fully (N/W IO Bound).
> Moreover, I want to know whether It would be a good/bad idea to use HBase for
> these kind of usecases ?.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
