[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920989#comment-16920989
]
Pankaj Kumar commented on HBASE-22928:
--
Thanks [~reidchan]
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7
>
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920878#comment-16920878
]
Hudson commented on HBASE-22928:
Results for branch branch-2.1
[build #1547 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1547/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1547//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1547//console].
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1547//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7
>
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920875#comment-16920875
]
Hudson commented on HBASE-22928:
Results for branch branch-2.2
[build #562 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/562/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/562//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/562//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/562//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7
>
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920823#comment-16920823
]
Hudson commented on HBASE-22928:
Results for branch branch-2
[build #2214 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2214/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2214//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2214//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2214//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7
>
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920817#comment-16920817
]
Hudson commented on HBASE-22928:
Results for branch master
[build #1386 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1386/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1386//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1386//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1386//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7
>
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920581#comment-16920581
]
Reid Chan commented on HBASE-22928:
---
Applied to master, branch-2, branch-2.1, branch-2.2.
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919341#comment-16919341
]
Pankaj Kumar commented on HBASE-22928:
--
Thanks for reviewing [~reidchan]...
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919290#comment-16919290
]
Reid Chan commented on HBASE-22928:
---
Nice catch, LGTM.
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
