[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718643#comment-15718643
]
Hudson commented on HBASE-17243:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2066 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2066/])
HBASE-17243 Reuse CompactionPartitionId and avoid creating MobFileName
(matteo.bertozzi: rev 03423fec6bbb16e049bd1294ec439cf56fd0e785)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17243-master-001.patch,
> HBASE-17243-master-002.patch, HBASE-17243-master-addendum.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718363#comment-15718363
]
Hudson commented on HBASE-17243:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2065 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2065/])
HBASE-17243 Reuse CompactionPartitionId and avoid creating MobFileName
(matteo.bertozzi: rev efcd15bf67f155e29b337886fe38eadfce75ffc6)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactionRequest.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileName.java
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17243-master-001.patch,
> HBASE-17243-master-002.patch, HBASE-17243-master-addendum.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718281#comment-15718281
]
huaxiang sun commented on HBASE-17243:
--
Thanks, I will create a jira to verify the select() logic.
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17243-master-001.patch,
> HBASE-17243-master-002.patch, HBASE-17243-master-addendum.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718270#comment-15718270
]
Matteo Bertozzi commented on HBASE-17243:
-
thanks for double checking. addendum is good, and committed. I did a run with
the addendum and it seems to pass.
but I guess we should add a test to verify the select(). open a new jira for
that.
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17243-master-001.patch,
> HBASE-17243-master-002.patch, HBASE-17243-master-addendum.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716764#comment-15716764
]
Hadoop QA commented on HBASE-17243:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 100m 34s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 143m 33s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12841551/HBASE-17243-master-002.patch
|
| JIRA Issue | HBASE-17243 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b2f219c713c8 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7775fed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4767/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4767/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4767/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
>
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716653#comment-15716653
]
Matteo Bertozzi commented on HBASE-17243:
-
+1
test failures seems unrelated
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17243-master-001.patch,
> HBASE-17243-master-002.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716591#comment-15716591
]
Hadoop QA commented on HBASE-17243:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 45s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 36s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestSerialReplication |
| Timed out junit tests |
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor |
| | org.apache.hadoop.hbase.TestRegionRebalancing |
| |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes |
| | org.apache.hadoop.hbase.quotas.TestQuotaThrottle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12841542/HBASE-17243-master-001.patch
|
| JIRA Issue | HBASE-17243 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux df0069310367 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4a20361 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4766/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4766/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-17243) Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
[
https://issues.apache.org/jira/browse/HBASE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716028#comment-15716028
]
Matteo Bertozzi commented on HBASE-17243:
-
do the _END_INDEX constants needs to be public? it seems something that only
the MobFileName should know about.
the constructor new CompactionPartitionId("", "") with the two empty string
looks a bit weird, maybe add an empty constructor since now the object is
mutable.
> Reuse CompactionPartitionId and avoid creating MobFileName in
> PartitionedMobCompactor to avoid unnecessary new objects
> --
>
> Key: HBASE-17243
> URL: https://issues.apache.org/jira/browse/HBASE-17243
> Project: HBase
> Issue Type: Improvement
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17243-master-001.patch
>
>
> In today's select() implementation, when it is an existing id, the new
> allocated object is discarded. It should be reused. fileName is created to
> getStartKey and getDate(), utility APIs can be created to directly get these
> fields from the string.
> {code}
> } else if (allFiles || linkedFile.getLen() < mergeableSize) {
> // add all files if allFiles is true,
> // otherwise add the small files to the merge pool
> MobFileName fileName =
> MobFileName.create(linkedFile.getPath().getName());
> CompactionPartitionId id = new
> CompactionPartitionId(fileName.getStartKey(),
> fileName.getDate());
> CompactionPartition compactionPartition = filesToCompact.get(id);
> if (compactionPartition == null) {
> compactionPartition = new CompactionPartition(id);
> compactionPartition.addFile(file);
> filesToCompact.put(id, compactionPartition);
> } else {
> compactionPartition.addFile(file);
> }
> selectedFileCount++;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
