[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4768: -- Resolution: Fixed Status: Resolved (was: Patch Available) Test fixed by HBASE-4795 Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4768.addendum, 4768.addendum2, D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4768: -- Attachment: 4768.addendum Proposed addendum removes duplicate assertion. Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4768.addendum, D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4768: - Status: Patch Available (was: Reopened) Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4768.addendum, D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4768: -- Status: Open (was: Patch Available) Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4768.addendum, D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4768: -- Attachment: 4768.addendum2 Second attempt to fixing assertion error: https://builds.apache.org/job/PreCommit-HBASE-Build/257//testReport/org.apache.hadoop.hbase.io.hfile/TestHFileBlock/testBlockHeapSize/ Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4768.addendum, 4768.addendum2, D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-4768: -- Release Note: Trying to get the most recent patch tested on Hudson. Status: Patch Available (was: Open) Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4768: --- Attachment: D363.5.patch mbautin updated the revision [jira] [HBASE-4768] Per-(table, columnFamily) metrics with configurable table name inclusion. Reviewers: jgray, nspiegelberg, stack, tedyu, todd, JIRA Rebasing on most recent change from trunk. REVISION DETAIL https://reviews.facebook.net/D363 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockInfo.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4768: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4768: --- Attachment: D363.3.patch mbautin updated the revision [jira] [HBASE-4768] Per-(table, columnFamily) metrics with configurable table name inclusion. Reviewers: jgray, nspiegelberg, stack, tedyu, todd, JIRA During cluster testing I found a mysterious column family name .tmp. It turns out that with cache-on-write the .tmp directory name was erroneously picked up as column family. Fixing this by setting table name and column family name from Store for temporary StoreFiles. REVISION DETAIL https://reviews.facebook.net/D363 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockInfo.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D363.1.patch, D363.2.patch, D363.3.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4768: --- Attachment: D363.4.patch mbautin updated the revision [jira] [HBASE-4768] Per-(table, columnFamily) metrics with configurable table name inclusion. Reviewers: jgray, nspiegelberg, stack, tedyu, todd, JIRA Addressing Ted's comments. All unit tests passed. REVISION DETAIL https://reviews.facebook.net/D363 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockInfo.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4768: --- Attachment: D363.2.patch mbautin updated the revision [jira] [HBASE-4768] Per-(table, columnFamily) metrics with configurable table name inclusion. Reviewers: jgray, nspiegelberg, stack, tedyu, todd, JIRA Addressing Nicolas's comments and removing two unused fields. REVISION DETAIL https://reviews.facebook.net/D363 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockInfo.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D363.1.patch, D363.2.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
