[jira] [Updated] (HBASE-18847) Remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18847: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Push to branch-1.2. Thanks for the patch. [~richhowarth] > Remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) Remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18847: --- Summary: Remove unneeded synchronized block from hfilev2 warning in branch-1.2 (was: remove unneeded synchronized block from hfilev2 warning in branch-1.2) > Remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Patch Available (was: Open) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: HBASE-18847-branch-1.2-00.patch > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847-branch-1.2-00.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Open (was: Patch Available) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: (was: HBASE-18847.v1.patch) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Status: Patch Available (was: Open) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847.v1.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Howarth updated HBASE-18847: - Attachment: HBASE-18847.v1.patch > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > Attachments: HBASE-18847.v1.patch > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-18847: --- Issue Type: Bug (was: Improvement) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18847: Summary: remove unneeded synchronized block from hfilev2 warning in branch-1.2 (was: Huge performance bottleneck in HFileWriterV2 in 1.2.6) > remove unneeded synchronized block from hfilev2 warning in branch-1.2 > - > > Key: HBASE-18847 > URL: https://issues.apache.org/jira/browse/HBASE-18847 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 >Reporter: Rich Howarth >Assignee: Rich Howarth >Priority: Critical > Fix For: 1.2.7 > > > The below code block starts at line 277 of HFileWriterV2.java. Class-level > synchronization in a heavily used code path has a demonstrably significant > negative effect on performance. I tested forcing a major compaction with 18 > compaction threads per node; removing the synchronization resulted in an > order of magnitude performance increase, with the bottleneck then being at > the disks (where I want it to be). > {code:java} > synchronized (HFileWriterV2.class) { > if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) { > LOG.warn("A minimum HFile version of " + > HFile.MIN_FORMAT_VERSION_WITH_TAGS > + " is required to support cell attributes/tags. Consider setting " > + HFile.FORMAT_VERSION_KEY + " accordingly."); > WARN_CELL_WITH_TAGS = false; > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)