[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581495#comment-13581495 ] Jesse Yates commented on HBASE-7725: Looks like the test failures are unrelated and it passes locally. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725-v5.patch, hbase-7725-v6.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581498#comment-13581498
]
Andrew Purtell commented on HBASE-7725:
---
The test failures are due to this:
{noformat}
java.net.BindException: Problem binding to localhost/127.0.0.1:42113 : Address
already in use
{noformat}
+1 for commit, thanks Jesse!
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725-v4.patch, hbase-7725-v5.patch, hbase-7725-v6.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581756#comment-13581756 ] Jesse Yates commented on HBASE-7725: Cool, thanks Andy! I'm planning to commmit this evening then, unless there are objections. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725-v5.patch, hbase-7725-v6.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13580770#comment-13580770 ] Andrew Purtell commented on HBASE-7725: --- This looks good. +1 if the new test passes Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725-v5.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13580806#comment-13580806
]
Hadoop QA commented on HBASE-7725:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569835/hbase-7725-v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
org.apache.hadoop.hbase.backup.TestHFileArchiving
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.regionserver.TestCompactionState
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.client.TestFromClientSide3
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4455//console
This message is automatically generated.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725-v4.patch, hbase-7725-v5.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13580816#comment-13580816
]
Andrew Purtell commented on HBASE-7725:
---
-1 Some NPEs out of compaction code:
{noformat}
Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompaction(CompactSplitThread.java:217)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.compactRegion(HRegionServer.java:3633)
{noformat}
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725-v4.patch, hbase-7725-v5.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13580950#comment-13580950
]
Hadoop QA commented on HBASE-7725:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569873/hbase-7725-v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4457//console
This message is automatically generated.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725-v4.patch, hbase-7725-v5.patch, hbase-7725-v6.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579283#comment-13579283 ] Andrew Purtell commented on HBASE-7725: --- bq. Therefore, for example, the CP that creates the CompactionRequest can check the attributes and make a decision as to whether or not it should modify the compaction (Oh, I already made this request! That means I need to ...). Or define a Comparator for CompactionRequest and have the CP extend that too as needed when extending CR? Should not need generic Object attributes as parameter to preCompactRequest and may not need generic Object attributes to pre/postCompactSelection if we pass in the nascent CompactionRequest reference there too, but no strong feeling about this either way. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579286#comment-13579286 ] Andrew Purtell commented on HBASE-7725: --- bq. The flip side is that we consider this a 'system' compaction separate from the user requested one and just pass in null attributes. I do think you should avoid having to make this distinction. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579415#comment-13579415 ] Jesse Yates commented on HBASE-7725: bq. may not need generic Object attributes to pre/postCompactSelection in if we pass in the nascent CompactionRequest reference there too I worry that we will start to pervert the compactionRequest creation logic for what is a relatively rare case - a user requested compaction via a CP (or at least that was the original intent, not sure what everyone will do with this). But an interesting idea (and another refactor :). So instead of passing in the attributes you pass in a compaction request to the CompactSplitThread? Something like, if the request is null we do the usual logic and build the request only after we get through all the selection logic in HStore. If not null, we apply the same logic, but set attributes on the compaction request (e.g. the store files to compact) on the passed in request? That certainly gets me what I need (controlled compactions) and hides the attribute business all the way down. Is that what you were getting at? I like it Andrew. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579417#comment-13579417 ] Jesse Yates commented on HBASE-7725: bq. I do think you should avoid having to make this distinction. Why? If we initiate a compaction and it does some stuff and then the system needs to go about doing its usual compaction business (its still has too many store files) then it ought to be able to. If you want to stop it, then you can hook into the usual compaction request creation mechansim. Am I missing something? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579512#comment-13579512
]
Andrew Purtell commented on HBASE-7725:
---
bq. That certainly gets me what I need (controlled compactions) and hides the
attribute business all the way down. Is that what you were getting at?
Yes.
{quote}
bq. I do think you should avoid having to make this distinction.
Why?
{quote}
I mean the (minor) extra cognitive load of figuring out if a CR is a system CR
or user CR or why the difference.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579599#comment-13579599 ] Sergey Shelukhin commented on HBASE-7725: - +1 Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579632#comment-13579632 ] Andrew Purtell commented on HBASE-7725: --- +1 patch looks good. Minor nit for on commit is the javadoc: custom compaction request - compaction request. May or may not be custom, right? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579642#comment-13579642 ] Jesse Yates commented on HBASE-7725: bq. Minor nit for on commit is the javadoc: custom compaction request - compaction request. May or may not be custom, right? Yeah, could also be null for pre/postCompactSelection. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579644#comment-13579644 ] Jesse Yates commented on HBASE-7725: I'd like to commit soon (this weekend) to get Sergey rolling on his compaction work, unless there are objections...? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579700#comment-13579700 ] Lars Hofhansl commented on HBASE-7725: -- +1 Wanna a simple (hopefully) test that tracks a CompactionRequest through the queue? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725-v4.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578934#comment-13578934 ] Sergey Shelukhin commented on HBASE-7725: - I don't think I understand why we have to make attributes visible to HBase code. For example, when we call this: server.compactSplitThread.requestCompaction(region, store, Recursive enqueue, attributes); we are inside old CompactionRequest, so we could just pass this instead of attributes all the way down, and coproc would get attributes if it needs to. That way HBase code doesn't even have to know attributes exist inside coproc-subclasses CompactionRequest. And coproc has to cast in this case anyway, either CR to subclass, or Object to specific attribute type. But, this shouldn't be blocking, so +1 Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579000#comment-13579000
]
Jesse Yates commented on HBASE-7725:
{quote}
I don't think I understand why we have to make attributes visible to HBase code.
For example, when we call this:
server.compactSplitThread.requestCompaction(region, store, Recursive enqueue,
attributes);
we are inside old CompactionRequest, so we could just pass this instead of
attributes all the way down, and coproc would get attributes if it needs to.
That way HBase code doesn't even have to know attributes exist inside
coproc-subclasses CompactionRequest.
{quote}
The question to me came down to be whether the re-enque is supposed to be a
'new' compaction request or a re-attempt at the same compaction. In the
existing code, its a brand new compaction request that just happens to be on
the same region and store.
So, do we pass along the original attributes or do we make it a 'system
created' compaction, separate from the original request?
My inclination was to pass the same attributes back along again. Therefore, for
example, the CP that creates the CompactionRequest can check the attributes and
make a decision as to whether or not it should modify the compaction (Oh, I
already made this request! That means I need to ...). Likely, you aren't going
to want to do anything again, but this is a bit more logic and knowledge of the
compaction architecture than most people are going to want to deal with (What,
why am I getting this request again! The one I ran is done!). But this is CPs
and you are supposed to have the rope to hang yourself :)
The flip side is that we consider this a 'system' compaction separate from the
user requested one and just pass in null attributes. This is nice in that its
cleaner (no leaked attributes) and logically is a different operation from the
original one that that client started. For instance, suppose the client starts
a compaction, but their compaction doesn't do actually change the number of
files. If we have a blocking number of store files we could re-enque that
compaction as a regular system operation.
Thoughts? I could go either way.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725-v3.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578094#comment-13578094 ] Sergey Shelukhin commented on HBASE-7725: - [~jesse_yates] do you want to commit soonish? I already made some cleanup of compaction request that can cause minor merge, bigger changes are afoot, just want to avoid big conflicts. Thanks! Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578103#comment-13578103 ] Jesse Yates commented on HBASE-7725: Yeah, I'll get on it. got a little sidetracked at the office, but I'll try to post the updated version on RB tonight/tomorrow am. Thanks for the heads up. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574738#comment-13574738 ] Sergey Shelukhin commented on HBASE-7725: - So, is there consensus on letting coproc to subclass CompactionRequest? I'm curious to look at the code change involved, for some other purpose :) Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574742#comment-13574742 ] Jesse Yates commented on HBASE-7725: That's what I was thinking - posted an update to RB with what the refactor looks like. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574969#comment-13574969 ] Andrew Purtell commented on HBASE-7725: --- bq. So, is there consensus on letting coproc to subclass CompactionRequest +1 Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574997#comment-13574997 ] Sergey Shelukhin commented on HBASE-7725: - posted responses on /r/, btw Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573767#comment-13573767 ] Sergey Shelukhin commented on HBASE-7725: - left some non-blocking comments in /r/. I am assuming Stack meant [~apurtell] in the last comment? :) Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573792#comment-13573792 ] Andrew Purtell commented on HBASE-7725: --- I think downcasting is a code smell and should be avoided unless necessary. This is not necessarily better: What about passing the CompactionRequest through the CP hooks and making it so the CompactionRequest type can be extended? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573795#comment-13573795 ] Jesse Yates commented on HBASE-7725: Thanks [~sershe]! Since I've got +1s from stack and Ted (and it looks like [~sershe] blessing too?), I'm planning on committing v1 to trunk and 0.94 tonight. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573796#comment-13573796 ] Andrew Purtell commented on HBASE-7725: --- Fine, -1. Consider the above. Then let's talk. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573833#comment-13573833
]
Jesse Yates commented on HBASE-7725:
I'd need to control a compaction request from inception to execution. Looking
at the way we make compactions, there are the pre/postCompactSelection which
would need to get the attributes. Then we could definitely add a hook at the
end where we can override the compactionRequest:
{code}
...
// everything went better than expected. create a compaction request
int pri = getCompactPriority(priority);
ret = new CompactionRequest(region, this, filesToCompact, isMajor, pri,
compactionAttributes);
...
{code}
to something like:
{code}
...
// everything went better than expected. create a compaction request
int pri = getCompactPriority(priority);
if (region.getCoprocessorHost() != null) {
ret = region.getCoprocessorHost().preCompactionRequest(region, this,
filesToCompact, isMajor, pri, compactionAttributes)
}
if(ret == null){
ret = new CompactionRequest(region, this, filesToCompact, isMajor, pri,
compactionAttributes);
}
{code}
Then we'd just have to pass in the compaction request to all the compaction CP
hooks, rather than the compactionAttributes object so we can handle special
case compaction setup (for methods like preCompactScannerOpen and preCompact).
So the same overall usage and layout, but we don't need to modify
CompactionRequest and let people add their own finish hooks and store
attributes if the want.
I dig it Andrew. Sound reasonable to everyone else?
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573837#comment-13573837 ] Andrew Purtell commented on HBASE-7725: --- I do like it better, thanks for listening, +0. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573840#comment-13573840 ] Sergey Shelukhin commented on HBASE-7725: - hmm, allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes. The latter is a common pattern when calling plugins that cannot preserve information across calls. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573844#comment-13573844 ] Sergey Shelukhin commented on HBASE-7725: - Another alternative, uglier but with more general applicability, is to create some ID for the operation and pass it to coprocessor methods if they belong to the same logical operation. If coprocessor has its own context, it can manage it, for example in a Map. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573860#comment-13573860
]
Jesse Yates commented on HBASE-7725:
{quote}
Another alternative, uglier but with more general applicability, is to create
some ID for the operation and pass it to coprocessor methods if they belong to
the same logical operation. If coprocessor has its own context, it can manage
it, for example in a Map.
{quote}
This would be far better managed as a set of attributes attached to a
CompactionRequest. Generally, you aren't going to start a compaction from one
of the generic RegionObserver methods, so its going to be an Endpoint, which
mean another class, so you then need to keep a static Map shared across the
classes.
{quote}
allowing the user to create compaction requests seems more dangerous to me than
having an opaque context/attributes.
{quote}
Sure, but CPs are inherently more dangerous. They are there for you people able
to futz around with the internals and come with no warranty ;) In our use case,
we will probably just subclass the exsiting CompactionRequest and add the
things that I added in the attached patches (v0, v1). I think adding this
subclass would be a reasonable inclusion, at the least in the hbase-examples
module, if not in the standard directories.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573861#comment-13573861 ] Andrew Purtell commented on HBASE-7725: --- bq. allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes I see it as being more powerful. This is Coprocessors, dangerous is implied. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573866#comment-13573866 ] Sergey Shelukhin commented on HBASE-7725: - Hmm, sure. +1. CompactionRequest then also needs InterfaceAudience, and ctor cannot be removed later. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573870#comment-13573870 ] Jesse Yates commented on HBASE-7725: Not necessarily - we already have a couple cases where we override an InterfaceAudience.Private. See RegionCoprocessorHost# postScannerOpen(final Scan scan, RegionScanner s), which returns a RegionScanner. Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[
https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573871#comment-13573871
]
Andrew Purtell commented on HBASE-7725:
---
We don't have consensus on if interface annotations are applicable to
coprocessors. I'd be cool with LimitedPrivate({coprocessors}) and Evolving.
Add generic attributes to CP initiated compaction request AND latch on
compaction completion
Key: HBASE-7725
URL: https://issues.apache.org/jira/browse/HBASE-7725
Project: HBase
Issue Type: Bug
Components: Compaction, Coprocessors, regionserver
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0, 0.94.6
Attachments: example.java, hbase-7725_0.94-v0.patch,
hbase-7725-v0.patch, hbase-7725-v1.patch,
hbase-7725_with-attributes-0.94-v0.patch,
hbase-7725_with-attributes-0.94-v1.patch
You can request that a compaction be started, but you can't be sure when that
compaction request completes. This is a simple update to the
CompactionRequest interface and the compact-split thread on the RS that
doesn't actually impact the RS exposed interface.
This is particularly useful for CPs so they can control starting/running a
compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7725) Add generic attributes to CP initiated compaction request AND latch on compaction completion
[ https://issues.apache.org/jira/browse/HBASE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13572225#comment-13572225 ] stack commented on HBASE-7725: -- Patch looks good to me. Mr Component Chief [~adp], what you reckon sir? Add generic attributes to CP initiated compaction request AND latch on compaction completion Key: HBASE-7725 URL: https://issues.apache.org/jira/browse/HBASE-7725 Project: HBase Issue Type: Bug Components: Compaction, Coprocessors, regionserver Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0, 0.94.6 Attachments: example.java, hbase-7725_0.94-v0.patch, hbase-7725-v0.patch, hbase-7725-v1.patch, hbase-7725_with-attributes-0.94-v0.patch, hbase-7725_with-attributes-0.94-v1.patch You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface. This is particularly useful for CPs so they can control starting/running a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
