[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279407#comment-13279407 ] Hudson commented on HBASE-6004: --- Integrated in HBase-0.94-security #27 (See [https://builds.apache.org/job/HBase-0.94-security/27/]) HBASE-6004. Adding more logging to help debugging MR job (Jimmy Xiang) (Revision 1339809) Result = SUCCESS apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch, hbase-6004_v2-0.94.patch, hbase-6004_v2.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13278172#comment-13278172 ] Hudson commented on HBASE-6004: --- Integrated in HBase-0.94 #194 (See [https://builds.apache.org/job/HBase-0.94/194/]) HBASE-6004. Adding more logging to help debugging MR job (Jimmy Xiang) (Revision 1339809) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch, hbase-6004_v2-0.94.patch, hbase-6004_v2.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13278245#comment-13278245 ] Hudson commented on HBASE-6004: --- Integrated in HBase-TRUNK #2893 (See [https://builds.apache.org/job/HBase-TRUNK/2893/]) HBASE-6004. Adding more logging to help debugging MR job (Jimmy Xiang) (Revision 1339806) Result = SUCCESS apurtell : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch, hbase-6004_v2-0.94.patch, hbase-6004_v2.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13278418#comment-13278418 ] Hudson commented on HBASE-6004: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #8 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/8/]) HBASE-6004. Adding more logging to help debugging MR job (Jimmy Xiang) (Revision 1339806) Result = FAILURE apurtell : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch, hbase-6004_v2-0.94.patch, hbase-6004_v2.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13277140#comment-13277140 ] jirapos...@reviews.apache.org commented on HBASE-6004: -- bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 54 bq. https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line54 bq. bq. We control logging via log4j.properties files. Here also new configuration in another file to switch some additional logging on and off. Would it make more sense to create a logger class e.g. ScannerCallable.ActivityLog that logs at TRACE level and update log4j.properties with log4j.logger.org.apache.hadoop.hbase.client.ScannerCallable.ActivityLog = TRACE (default is INFO, i.e. disabled). Just a thought. We did something like this for security audit logging. bq. bq. Jimmy Xiang wrote: bq. It is a good idea. Because of other parameters introduced, it's better to put them in the same place for now. Fair enough, there isn't a way to get the value of properties in the log4j.properties file using either commons logging API nor java.util.logging that I can see. - Andrew --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/#review7918 --- On 2012-05-16 02:56:16, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/5131/ bq. --- bq. bq. (Updated 2012-05-16 02:56:16) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Added some logging for MR debugging in case scanner times out. The logging is disabled by default. bq. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. bq. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. bq. bq. bq. This addresses bug HBASE-6004. bq. https://issues.apache.org/jira/browse/HBASE-6004 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 bq.src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 bq. bq. Diff: https://reviews.apache.org/r/5131/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276287#comment-13276287 ] jirapos...@reviews.apache.org commented on HBASE-6004: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/ --- Review request for hbase. Summary --- Added some logging for MR debugging in case scanner times out. The logging is disabled by default. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. This addresses bug HBASE-6004. https://issues.apache.org/jira/browse/HBASE-6004 Diffs - src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 Diff: https://reviews.apache.org/r/5131/diff Testing --- Thanks, Jimmy Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276324#comment-13276324 ] jirapos...@reviews.apache.org commented on HBASE-6004: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/#review7918 --- src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java https://reviews.apache.org/r/5131/#comment17227 We control logging via log4j.properties files. Here also new configuration in another file to switch some additional logging on and off. Would it make more sense to create a logger class e.g. ScannerCallable.ActivityLog that logs at TRACE level and update log4j.properties with log4j.logger.org.apache.hadoop.hbase.client.ScannerCallable.ActivityLog = TRACE (default is INFO, i.e. disabled). Just a thought. We did something like this for security audit logging. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java https://reviews.apache.org/r/5131/#comment17224 Maybe the client should just log this? src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java https://reviews.apache.org/r/5131/#comment17225 The new message seems more helpful than the old. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java https://reviews.apache.org/r/5131/#comment17228 We have other configuration variables that start with 'hbase.mapreduce.*' - Andrew On 2012-05-15 22:24:40, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/5131/ bq. --- bq. bq. (Updated 2012-05-15 22:24:40) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Added some logging for MR debugging in case scanner times out. The logging is disabled by default. bq. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. bq. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. bq. bq. bq. This addresses bug HBASE-6004. bq. https://issues.apache.org/jira/browse/HBASE-6004 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 bq.src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 bq. bq. Diff: https://reviews.apache.org/r/5131/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276341#comment-13276341 ] jirapos...@reviews.apache.org commented on HBASE-6004: -- bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 164 bq. https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line164 bq. bq. Maybe the client should just log this? Client sometimes just gets a RetriesExhaustedException. bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java, line 51 bq. https://reviews.apache.org/r/5131/diff/1/?file=109076#file109076line51 bq. bq. We have other configuration variables that start with 'hbase.mapreduce.*' Will change to mapreduce. bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 54 bq. https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line54 bq. bq. We control logging via log4j.properties files. Here also new configuration in another file to switch some additional logging on and off. Would it make more sense to create a logger class e.g. ScannerCallable.ActivityLog that logs at TRACE level and update log4j.properties with log4j.logger.org.apache.hadoop.hbase.client.ScannerCallable.ActivityLog = TRACE (default is INFO, i.e. disabled). Just a thought. We did something like this for security audit logging. It is a good idea. Because of other parameters introduced, it's better to put them in the same place for now. - Jimmy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/#review7918 --- On 2012-05-15 22:24:40, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/5131/ bq. --- bq. bq. (Updated 2012-05-15 22:24:40) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Added some logging for MR debugging in case scanner times out. The logging is disabled by default. bq. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. bq. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. bq. bq. bq. This addresses bug HBASE-6004. bq. https://issues.apache.org/jira/browse/HBASE-6004 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 bq.src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 bq. bq. Diff: https://reviews.apache.org/r/5131/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6004) Adding more logging to help debugging MR job
[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276403#comment-13276403 ] jirapos...@reviews.apache.org commented on HBASE-6004: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/ --- (Updated 2012-05-16 02:56:16.913480) Review request for hbase. Summary --- Added some logging for MR debugging in case scanner times out. The logging is disabled by default. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. This addresses bug HBASE-6004. https://issues.apache.org/jira/browse/HBASE-6004 Diffs (updated) - src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 Diff: https://reviews.apache.org/r/5131/diff Testing --- Thanks, Jimmy Adding more logging to help debugging MR job Key: HBASE-6004 URL: https://issues.apache.org/jira/browse/HBASE-6004 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: hbase-6004.patch MR job sometime fails because scanner expired. In this case, it will be helpful to know the last successful row, the ip of the region sever, and so on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
