[jira] [Commented] (HBASE-18847) Remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176814#comment-16176814 ] Rich Howarth commented on HBASE-18847: -- No problem. Thanks to Sean for helping me get set up > Remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Patch Available (was: Open) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: HBASE-18847-branch-1.2-00.patch > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Open (was: Patch Available) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: (was: HBASE-18847.v1.patch) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Patch Available (was: Open) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847.v1.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: HBASE-18847.v1.patch > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847.v1.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18847) Huge performance bottleneck in HFileWriterV2 in 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171752#comment-16171752 ] Rich Howarth commented on HBASE-18847: -- Agreed, that's what I have in my local patch at the moment > Huge performance bottleneck in HFileWriterV2 in 1.2.6 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.2.6 >Reporter: Rich Howarth >Priority: Critical > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) Huge performance bottleneck in HFileWriterV2 in 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Description: The below code block starts at line 277 of HFileWriterV2.java. Class-level synchronization in a heavily used code path has a demonstrably significant negative effect on performance. I tested forcing a major compaction with 18 compaction threads per node; removing the synchronization resulted in an order of magnitude performance increase, with the bottleneck then being at the disks (where I want it to be). {code:java} synchronized (HFileWriterV2.class) { if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { LOG.warn("A minimum HFile version of " + HFile.MIN_FORMAT_VERSION_WITH_TAGS + " is required to support cell attributes/tags. Consider setting " + HFile.FORMAT_VERSION_KEY + " accordingly."); WARN_CELL_WITH_TAGS = false; } } {code} was: The below code block starts at line 277 of HFileWriterV2.java. Class-level synchronization in a heavily used code path has a demonstrably significant negative effect on performance. I tested forcing a major compaction with 18 compaction threads per node; removing the synchronization resulted in an order of magnitude performance increase, with the bottleneck then being at the disks (where I want it to be). synchronized (HFileWriterV2.class) { if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { LOG.warn("A minimum HFile version of " + HFile.MIN_FORMAT_VERSION_WITH_TAGS + " is required to support cell attributes/tags. Consider setting " + HFile.FORMAT_VERSION_KEY + " accordingly."); WARN_CELL_WITH_TAGS = false; } } > Huge performance bottleneck in HFileWriterV2 in 1.2.6 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.6 >Reporter: Rich Howarth > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18847) Huge performance bottleneck in HFileWriterV2 in 1.2.6
Rich Howarth created HBASE-18847: Summary: Huge performance bottleneck in HFileWriterV2 in 1.2.6 Key: HBASE-18847 URL: https://issues.apache.org/jira/browse/HBASE-18847 Project: HBase Issue Type: Bug Affects Versions: 1.2.6 Reporter: Rich Howarth The below code block starts at line 277 of HFileWriterV2.java. Class-level synchronization in a heavily used code path has a demonstrably significant negative effect on performance. I tested forcing a major compaction with 18 compaction threads per node; removing the synchronization resulted in an order of magnitude performance increase, with the bottleneck then being at the disks (where I want it to be). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) Huge performance bottleneck in HFileWriterV2 in 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Description: The below code block starts at line 277 of HFileWriterV2.java. Class-level synchronization in a heavily used code path has a demonstrably significant negative effect on performance. I tested forcing a major compaction with 18 compaction threads per node; removing the synchronization resulted in an order of magnitude performance increase, with the bottleneck then being at the disks (where I want it to be). synchronized (HFileWriterV2.class) { if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { LOG.warn("A minimum HFile version of " + HFile.MIN_FORMAT_VERSION_WITH_TAGS + " is required to support cell attributes/tags. Consider setting " + HFile.FORMAT_VERSION_KEY + " accordingly."); WARN_CELL_WITH_TAGS = false; } } was:The below code block starts at line 277 of HFileWriterV2.java. Class-level synchronization in a heavily used code path has a demonstrably significant negative effect on performance. I tested forcing a major compaction with 18 compaction threads per node; removing the synchronization resulted in an order of magnitude performance increase, with the bottleneck then being at the disks (where I want it to be). > Huge performance bottleneck in HFileWriterV2 in 1.2.6 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.6 >Reporter: Rich Howarth > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } -- This message was sent by Atlassian JIRA (v6.4.14#64029)