[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114814#comment-15114814 ] Yu Li commented on HBASE-11586: --- Have filed HBASE-15160 to put back HDFS op latency sampling and add metrics for monitoring. [~apurtell] could you help review? Thanks. > HFile's HDFS op latency sampling code is not used > - > > Key: HBASE-11586 > URL: https://issues.apache.org/jira/browse/HBASE-11586 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 0.98.5, 2.0.0 > > Attachments: HBASE-11586.patch, HBASE-11586.patch > > > HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls > HFile#offerWriteLatency but the samples are never retrieved. There are no > callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and > related. The three ArrayBlockingQueues we are using as sample buffers in > HFile will fill quickly and are never drained. > There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and > related, so we are incrementing a set of AtomicLong counters that will never > be read nor reset. > We are calling System.nanoTime in block read and write paths twice but not > utilizing the measurements. > We should hook this code back up to metrics or remove it. > We are also not using HFile#getChecksumFailuresCount anywhere but in some > unit test code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[
https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296593#comment-14296593
]
hongyu bi commented on HBASE-11586:
---
we just upgrade 0.98.6 from 0.94.15 , and I can't find the
fs{read|write}ops/latency metrics, what about the new jira? many thanks
HFile's HDFS op latency sampling code is not used
-
Key: HBASE-11586
URL: https://issues.apache.org/jira/browse/HBASE-11586
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11586.patch, HBASE-11586.patch
HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls
HFile#offerWriteLatency but the samples are never retrieved. There are no
callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and
related. The three ArrayBlockingQueues we are using as sample buffers in
HFile will fill quickly and are never drained.
There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and
related, so we are incrementing a set of AtomicLong counters that will never
be read nor reset.
We are calling System.nanoTime in block read and write paths twice but not
utilizing the measurements.
We should hook this code back up to metrics or remove it.
We are also not using HFile#getChecksumFailuresCount anywhere but in some
unit test code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14081420#comment-14081420 ] Andrew Purtell commented on HBASE-11586: bq. Let me fill in the story here Andrew. Will be back. Cool, we can put the metric back in a new JIRA I suppose. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076726#comment-14076726 ] stack commented on HBASE-11586: --- bq. I assume the removal of the metric was an intended and reviewed change though. Let me fill in the story here Andrew. Will be back. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074155#comment-14074155 ] Hudson commented on HBASE-11586: SUCCESS: Integrated in HBase-0.98 #419 (See [https://builds.apache.org/job/HBase-0.98/419/]) HBASE-11586 HFile's HDFS op latency sampling code is not used (apurtell: rev 27eef5f73960944f6cbaa3894fd58d5e5b3bfc28) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/HFileReadWriteTest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074174#comment-14074174 ] Hudson commented on HBASE-11586: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #398 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/398/]) HBASE-11586 HFile's HDFS op latency sampling code is not used (apurtell: rev 27eef5f73960944f6cbaa3894fd58d5e5b3bfc28) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/HFileReadWriteTest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074184#comment-14074184 ] Hudson commented on HBASE-11586: SUCCESS: Integrated in HBase-1.0 #70 (See [https://builds.apache.org/job/HBase-1.0/70/]) HBASE-11586 HFile's HDFS op latency sampling code is not used (apurtell: rev 13643807adffd7f5a798251594f275bc318d00eb) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/HFileReadWriteTest.java HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074210#comment-14074210 ] Hudson commented on HBASE-11586: FAILURE: Integrated in HBase-TRUNK #5342 (See [https://builds.apache.org/job/HBase-TRUNK/5342/]) HBASE-11586 HFile's HDFS op latency sampling code is not used (apurtell: rev 531eee003182647e9f944a5cbcb6117555c39e44) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/HFileReadWriteTest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074786#comment-14074786 ] Lars Hofhansl commented on HBASE-11586: --- Sorry I change the assignee to me... That was unintended. Lemme check the 0.94 code. Every bit of memory barriers we can remove is a win. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074868#comment-14074868 ] Andrew Purtell commented on HBASE-11586: Do we need the dynamic schema metrics that these block read and write path measurements feed? If not we could open a 0.94 specific JIRA to remove them. Then this change can be backported. We could do the work with two commits like that perhaps. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[
https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075010#comment-14075010
]
Lars Hofhansl commented on HBASE-11586:
---
Honestly I am confused where we use/display these metrics. Are these the global
fs{read|write}Latency histograms that are shown on the region server UI and via
JMX, or something else altogether. I'll check the 0.94 in a bit.
HFile's HDFS op latency sampling code is not used
-
Key: HBASE-11586
URL: https://issues.apache.org/jira/browse/HBASE-11586
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11586.patch, HBASE-11586.patch
HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls
HFile#offerWriteLatency but the samples are never retrieved. There are no
callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and
related. The three ArrayBlockingQueues we are using as sample buffers in
HFile will fill quickly and are never drained.
There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and
related, so we are incrementing a set of AtomicLong counters that will never
be read nor reset.
We are calling System.nanoTime in block read and write paths twice but not
utilizing the measurements.
We should hook this code back up to metrics or remove it.
We are also not using HFile#getChecksumFailuresCount anywhere but in some
unit test code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[
https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075043#comment-14075043
]
Andrew Purtell commented on HBASE-11586:
bq. Are these the global fs{read|write}Latency histograms that are shown on the
region server UI and via JMX
Yes. The same measurement is also used when updating one of the dynamic schema
metrics.
HFile's HDFS op latency sampling code is not used
-
Key: HBASE-11586
URL: https://issues.apache.org/jira/browse/HBASE-11586
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11586.patch, HBASE-11586.patch
HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls
HFile#offerWriteLatency but the samples are never retrieved. There are no
callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and
related. The three ArrayBlockingQueues we are using as sample buffers in
HFile will fill quickly and are never drained.
There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and
related, so we are incrementing a set of AtomicLong counters that will never
be read nor reset.
We are calling System.nanoTime in block read and write paths twice but not
utilizing the measurements.
We should hook this code back up to metrics or remove it.
We are also not using HFile#getChecksumFailuresCount anywhere but in some
unit test code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075081#comment-14075081 ] stack commented on HBASE-11586: --- Seem like useful metrics. How hard to hook them up? I could do it in new issue? HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075103#comment-14075103 ] Andrew Purtell commented on HBASE-11586: Sure, we can revert the current committed change and add back the regionserver metrics for this removed earlier. I assume the removal of the metric was an intended and reviewed change though. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never retrieved. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073743#comment-14073743 ] Srikanth Srungarapu commented on HBASE-11586: - +1 (non-binding vote). HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never drained. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[
https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073747#comment-14073747
]
Hadoop QA commented on HBASE-11586:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657668/HBASE-11586.patch
against trunk revision .
ATTACHMENT ID: 12657668
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10180//console
This message is automatically generated.
HFile's HDFS op latency sampling code is not used
-
Key: HBASE-11586
URL: https://issues.apache.org/jira/browse/HBASE-11586
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Andrew Purtell
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11586.patch
HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls
HFile#offerWriteLatency but the samples are never drained. There are no
callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and
related. The three ArrayBlockingQueues we are using as sample buffers in
HFile will fill quickly and are never drained.
There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and
related, so we are incrementing a set of AtomicLong counters that will never
be read nor reset.
We are calling System.nanoTime in block read and write paths twice but not
utilizing the measurements.
We should hook this code back up to metrics or remove it.
We are also not using HFile#getChecksumFailuresCount anywhere but in some
unit test code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073780#comment-14073780 ] Enis Soztutar commented on HBASE-11586: --- +1 for branch-1+. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never drained. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073929#comment-14073929 ] Lars Hofhansl commented on HBASE-11586: --- Nice. I'll check 0.94 as well. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Lars Hofhansl Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never drained. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[
https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073952#comment-14073952
]
Hadoop QA commented on HBASE-11586:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657725/HBASE-11586.patch
against trunk revision .
ATTACHMENT ID: 12657725
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10183//console
This message is automatically generated.
HFile's HDFS op latency sampling code is not used
-
Key: HBASE-11586
URL: https://issues.apache.org/jira/browse/HBASE-11586
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Andrew Purtell
Assignee: Lars Hofhansl
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11586.patch, HBASE-11586.patch
HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls
HFile#offerWriteLatency but the samples are never drained. There are no
callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and
related. The three ArrayBlockingQueues we are using as sample buffers in
HFile will fill quickly and are never drained.
There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and
related, so we are incrementing a set of AtomicLong counters that will never
be read nor reset.
We are calling System.nanoTime in block read and write paths twice but not
utilizing the measurements.
We should hook this code back up to metrics or remove it.
We are also not using HFile#getChecksumFailuresCount anywhere but in some
unit test code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11586) HFile's HDFS op latency sampling code is not used
[ https://issues.apache.org/jira/browse/HBASE-11586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074077#comment-14074077 ] Andrew Purtell commented on HBASE-11586: We still use the HDFS op latency measurements in 0.94 HFile, for regionserver and dynamic schema metrics. HFile's HDFS op latency sampling code is not used - Key: HBASE-11586 URL: https://issues.apache.org/jira/browse/HBASE-11586 Project: HBase Issue Type: Bug Affects Versions: 0.98.4 Reporter: Andrew Purtell Assignee: Lars Hofhansl Fix For: 0.99.0, 0.98.5, 2.0.0 Attachments: HBASE-11586.patch, HBASE-11586.patch HFileReaderV2 calls HFile#offerReadLatency and HFileWriterV2 calls HFile#offerWriteLatency but the samples are never drained. There are no callers of HFile#getReadLatenciesNanos, HFile#getWriteLatenciesNanos, and related. The three ArrayBlockingQueues we are using as sample buffers in HFile will fill quickly and are never drained. There are also no callers of HFile#getReadTimeMs or HFile#getWriteTimeMs, and related, so we are incrementing a set of AtomicLong counters that will never be read nor reset. We are calling System.nanoTime in block read and write paths twice but not utilizing the measurements. We should hook this code back up to metrics or remove it. We are also not using HFile#getChecksumFailuresCount anywhere but in some unit test code. -- This message was sent by Atlassian JIRA (v6.2#6252)
