[
https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276341#comment-13276341
]
[email protected] commented on HBASE-6004:
------------------------------------------------------
bq. On 2012-05-15 23:11:04, Andrew Purtell wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line
164
bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line164>
bq. >
bq. > Maybe the client should just log this?
Client sometimes just gets a RetriesExhaustedException.
bq. On 2012-05-15 23:11:04, Andrew Purtell wrote:
bq. >
src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java,
line 51
bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109076#file109076line51>
bq. >
bq. > We have other configuration variables that start with
'hbase.mapreduce.*'
Will change to mapreduce.
bq. On 2012-05-15 23:11:04, Andrew Purtell wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line
54
bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line54>
bq. >
bq. > We control logging via log4j.properties files. Here also new
configuration in another file to switch some additional logging on and off.
Would it make more sense to create a logger class e.g.
ScannerCallable.ActivityLog that logs at TRACE level and update
log4j.properties with
log4j.logger.org.apache.hadoop.hbase.client.ScannerCallable.ActivityLog = TRACE
(default is INFO, i.e. disabled). Just a thought. We did something like this
for security audit logging.
It is a good idea. Because of other parameters introduced, it's better to put
them in the same place for now.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5131/#review7918
-----------------------------------------------------------
On 2012-05-15 22:24:40, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5131/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-15 22:24:40)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Added some logging for MR debugging in case scanner times out. The
logging is disabled by default.
bq. It will be helpful to know how much time spent in the scanner and, how
much in the mapper task.
bq. In case scanner issue, it is helpful to know the region server id, last
successful rows and so on.
bq.
bq.
bq. This addresses bug HBASE-6004.
bq. https://issues.apache.org/jira/browse/HBASE-6004
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
46b1c56
bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
42569fb
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
1c8a393
bq.
bq. Diff: https://reviews.apache.org/r/5131/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Adding more logging to help debugging MR job
> --------------------------------------------
>
> Key: HBASE-6004
> URL: https://issues.apache.org/jira/browse/HBASE-6004
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Fix For: 0.94.0, 0.96.0
>
> Attachments: hbase-6004.patch
>
>
> MR job sometime fails because scanner expired. In this case, it will be
> helpful to know the last successful row, the ip of the region sever, and so
> on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
