[jira] [Updated] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update
[ https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6499: - Resolution: Fixed Status: Resolved (was: Patch Available) Was committed a while back. Resolving. StoreScanner's QueryMatcher not reset on store update - Key: HBASE-6499 URL: https://issues.apache.org/jira/browse/HBASE-6499 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.96.0 Reporter: Max Lapan Assignee: Max Lapan Fix For: 0.96.0 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch When underlying store changed (due compact, bulk load, etc), we destroy current KeyValueHeap and recreate it using checkReseek call. Besides heap recreation, it resets underlying QueryMatcher instance. The problem is that checkReseek not called by seek() and reseek(), only by next(). If someone calls seek() just after store changed, it gets wrong scanner results. Call to reseek may end up with NPE. AFAIK, current codebase don't call seek and reseek, but it is quite possible in future. Personally, I spent lots of time to find source of wrong scanner results in HBASE-5416. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6365) Deprecate classes that depend on (old) metrics framework
[ https://issues.apache.org/jira/browse/HBASE-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550957#comment-13550957 ] stack commented on HBASE-6365: -- [~eclark] This issue is now invalid or we should commit what is in here? Deprecate classes that depend on (old) metrics framework Key: HBASE-6365 URL: https://issues.apache.org/jira/browse/HBASE-6365 Project: HBase Issue Type: Task Reporter: Ted Yu Fix For: 0.96.0 Attachments: 6365-94.txt, 6365-94-v2.txt org.apache.hadoop.metrics.* classes have been deprecated. We should deprecate HBase classes that depend on them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7506) Judgment of carrying ROOT/META will become wrong when expiring server
[ https://issues.apache.org/jira/browse/HBASE-7506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550960#comment-13550960 ] Hudson commented on HBASE-7506: --- Integrated in HBase-0.94 #723 (See [https://builds.apache.org/job/HBase-0.94/723/]) HBASE-7506 Judgment of carrying ROOT/META will become wrong when expiring server (Chunhui) (Revision 1431903) Result = SUCCESS zjushch : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java Judgment of carrying ROOT/META will become wrong when expiring server - Key: HBASE-7506 URL: https://issues.apache.org/jira/browse/HBASE-7506 Project: HBase Issue Type: Bug Affects Versions: 0.94.3 Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.96.0, 0.94.5 Attachments: 7506-94.patch, 7506-trunk v1.patch, 7506-trunkv1.patch, 7506-trunkv2.patch We will check whether server carrying ROOT/META when expiring the server. See ServerManager#expireServer. If the dead server carrying META, we assign meta directly in the process of ServerShutdownHandler. If the dead server carrying ROOT, we will offline ROOT and then verifyAndAssignRootWithRetries() How judgement of carrtying ROOT/META become wrong? If region is in RIT, and isCarryingRegion() return true after addressing from zk. However, once RIT time out(could be caused by this.allRegionServersOffline !noRSAvailable, see AssignmentManager#TimeoutMonitor) and we assign it to otherwhere, this judgement become wrong. See AssignmentManager#isCarryingRegion for details With the wrong judgement of carrtying ROOT/META, we would assign ROOT/META twice. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6307) Fix hbase unit tests running on hadoop 2.0
[ https://issues.apache.org/jira/browse/HBASE-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6307: - Priority: Critical (was: Major) Fix hbase unit tests running on hadoop 2.0 -- Key: HBASE-6307 URL: https://issues.apache.org/jira/browse/HBASE-6307 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Priority: Critical Fix For: 0.96.0 This is an umbrella issue for fixing unit tests and hbase builds form 0.92+ on top of hadoop 0.23 (currently 0.92/0.94) and hadoop 2.0.x (trunk/0.96). Once these are up and passing properly, we'll close out the umbrella issue by adding hbase-trunk-on-hadoop-2 build to the hadoopqa bot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7486) master pid file is not getting removed if we stop hbase from stop-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550969#comment-13550969 ] rajeshbabu commented on HBASE-7486: --- ok. Thanks. master pid file is not getting removed if we stop hbase from stop-hbase.sh -- Key: HBASE-7486 URL: https://issues.apache.org/jira/browse/HBASE-7486 Project: HBase Issue Type: Bug Components: shell Reporter: rajeshbabu Assignee: rajeshbabu Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7486_92.patch, HBASE-7486_94.patch, HBASE-7486_trunk.patch in stop-hbase.sh script we are not removing master pid file after master termination. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4451) Improve zk node naming (/hbase/shutdown)
[
https://issues.apache.org/jira/browse/HBASE-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4451:
-
Priority: Critical (was: Major)
Improve zk node naming (/hbase/shutdown)
Key: HBASE-4451
URL: https://issues.apache.org/jira/browse/HBASE-4451
Project: HBase
Issue Type: Improvement
Components: master
Affects Versions: 0.94.0
Reporter: Harsh J
Priority: Critical
Labels: noob
Fix For: 0.96.0
Right now the node {{/hbase/shutdown}} is used to indicate cluster status
(cluster up, cluster down).
However, upon a chat with Lars George today, we feel that having a name
{{/hbase/shutdown}} is possibly bad. The {{/hbase/shutdown}} zknode contains
a date when the cluster was _started_. Now that is difficult to understand
and digest, given that a person may connect to zk and try to look at what it
is about (they may think it 'shutdown' at that date.).
I feel a better name may simply be: {{/hbase/running}}. Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7365) Safer table creation and deletion using .tmp dir
[
https://issues.apache.org/jira/browse/HBASE-7365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550973#comment-13550973
]
Hadoop QA commented on HBASE-7365:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12564368/HBASE-7365-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestLocalHBaseCluster
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup.testBalancerWithRackLocality(TestBalancerWithNodeGroup.java:220)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3978//console
This message is automatically generated.
Safer table creation and deletion using .tmp dir
Key: HBASE-7365
URL: https://issues.apache.org/jira/browse/HBASE-7365
Project: HBase
Issue Type: Improvement
Components: master
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Fix For: 0.96.0
Attachments: HBASE-7365-v0.patch, HBASE-7365-v1.patch,
HBASE-7365-v2.patch, HBASE-7365-v3.patch
Currently tables are created in the root directory, and the removal works on
the root directory.
Change the code to use a /hbase/.tmp directory to make the creation and
removal a bit safer
Table Creation steps
* Create the table descriptor (table folder, in /hbase/.tmp/)
* Create the table regions (always in temp)
* Move the table from temp to the root folder
* Add the regions to meta
* Trigger assignment
* Set enable flag in ZooKeeper
Table Deletion steps
* Wait for regions in transition
* Remove regions from meta (use bulk delete)
* Move the table in /hbase/.tmp
* Remove the table from the descriptor cache
* Remove table from zookeeper
* Archive the table
The main changes in the current code are:
* Writing to /hbase/.tmp and then rename
* using bulk delete in DeletionTableHandler
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4231) Flush regions in multiple threads
[ https://issues.apache.org/jira/browse/HBASE-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-4231. -- Resolution: Duplicate Marking duplicate of a later issue that has more energy behind it, HBASE-6466 Flush regions in multiple threads - Key: HBASE-4231 URL: https://issues.apache.org/jira/browse/HBASE-4231 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.94.0 Reporter: Todd Lipcon Fix For: 0.96.0 As servers get beefier, it makes less and less sense to have only a single thread do flushes at a time. In watching an insert-only benchmark run, HBase isn't currently disk, network, or CPU-bound -- it seems it's bound only by the rate at which flushes and compactions get processed. Increasing the threading a bit here should increase total disk/network throughput utilization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3996: - Priority: Critical (was: Major) Marking critical so gets review. This is popular request. Lets try get it in. Support multiple tables and scanners as input to the mapper in map/reduce jobs -- Key: HBASE-3996 URL: https://issues.apache.org/jira/browse/HBASE-3996 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Eran Kutner Assignee: Lars Hofhansl Priority: Critical Fix For: 0.96.0, 0.94.5 Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, HBase-3996.patch It seems that in many cases feeding data from multiple tables or multiple scanners on a single table can save a lot of time when running map/reduce jobs. I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7213: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for the patch DD. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7213: - Release Note: The regionserver carrying .META. region will now write two WALs; the usual one w/ all edits and then a special one with a .meta. suffix into which all edits for .META. region go. These files will be recovered first on server crash. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7293) [replication] Remove dead sinks from ReplicationSource.currentPeers, it's spammy
[
https://issues.apache.org/jira/browse/HBASE-7293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550982#comment-13550982
]
stack commented on HBASE-7293:
--
Want me to commit?
[replication] Remove dead sinks from ReplicationSource.currentPeers, it's
spammy
Key: HBASE-7293
URL: https://issues.apache.org/jira/browse/HBASE-7293
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Jean-Daniel Cryans
Assignee: Lars Hofhansl
Fix For: 0.96.0, 0.94.5
Attachments: 7293-0.94.txt, 7293-0.94-v2.txt, 7293-0.96.txt
I happened to look at a log today where I saw a lot lines like this:
{noformat}
2012-12-06 23:29:08,318 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Slave
cluster looks down: This server is in the failed servers list:
sv4r20s49/10.4.20.49:10304
2012-12-06 23:29:15,987 WARN
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
replicate because of a local or network error:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:519)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:484)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:416)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:462)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1150)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1000)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy14.replicateLogEntries(Unknown Source)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:627)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:365)
2012-12-06 23:29:15,988 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Slave
cluster looks down: Connection refused
{noformat}
What struck me as weird is this had been going on for some days, I would
expect the RS to find new servers if it wasn't able to replicate. But the
reality is that only a few of the chosen sink RS were down so eventually the
source hits one that's good and is never able to refresh its list of servers.
We should remove the dead servers, it's spammy and probably adds some slave
lag.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3093) DeleteColumns deletes alphabetical next columns when using timestamp 0
[ https://issues.apache.org/jira/browse/HBASE-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550992#comment-13550992 ] Anoop Sam John commented on HBASE-3093: --- This test case is tested against 0.94 branch and it passed.! May be issue with 89 branch alone? !! DeleteColumns deletes alphabetical next columns when using timestamp 0 -- Key: HBASE-3093 URL: https://issues.apache.org/jira/browse/HBASE-3093 Project: HBase Issue Type: Bug Reporter: Evert Arckens Attachments: DeleteColumnsTest.java When calling Delete.deleteColumns to delete (all versions of) a column, the columns with a qualifier that is alphabetical higher than the deleted column are deleted as well when the cells have timestamp 0. When the cells have a timestamp higher than 0 this is not the case. The test case DeleteColumnsTest (tested against HBase 0.89.0-r1004203-3076) contains a scenario showing this behaviour : - put values in 3 columns (A, B and C) of the same column family at timestamp 0. - delete all versions of column B (using Delete.deleteColumns() ) - read columns A and C - result : for column A a result is given, for column C not (the alphabetical order is in play here!) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5148) Compaction property at the server level are not propagated at the table level.
[ https://issues.apache.org/jira/browse/HBASE-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550996#comment-13550996 ] Anoop Sam John commented on HBASE-5148: --- Mikael Sitruk Where the config param hbase.hregion.majorcompaction is used, we are 1st checking whether this is added at the table level. If not we take the server level value configured in the xml file. HTableDescriptor#getValue() is just a getter on the POJO. It should return the value which is set on it (if set something). IMO it no need to fetch the data from the config xml. When this value is used we need to consider the value from HTableDescriptor or config.. I think this is not a valid bug and can be closed. Compaction property at the server level are not propagated at the table level. -- Key: HBASE-5148 URL: https://issues.apache.org/jira/browse/HBASE-5148 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.90.1 Reporter: Mikael Sitruk In case you do not override compaction parameter on the table/cf level, the values returned by the table descriptor will not reflect the value configured in the cluster. For example - let assume you disabled major compaction by setting hbase.hregion.majorcompaction in the config to 0, prior starting the cluster. Let also assume that you have a table that in which you didn't set at all this parameter. Then invoking HTableDescriptor hTableDescriptor = conn.getHTableDescriptor(Bytes.toBytes(my table)); hTableDescriptor.getValue(hbase.hregion.majorcompaction) should return the cluster property (currently returns the default, ignoring the cluster prop.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6656) Cannot call a Coprocessor Endpoint from a RegionObserver
[
https://issues.apache.org/jira/browse/HBASE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551013#comment-13551013
]
Anoop Sam John commented on HBASE-6656:
---
Is there a jar (which includes all the classes for your Endpoint) there in the
Region Server process's classpath?
Cannot call a Coprocessor Endpoint from a RegionObserver
Key: HBASE-6656
URL: https://issues.apache.org/jira/browse/HBASE-6656
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.92.1
Environment: CentOS5
Reporter: Mauricio Morales
I'm trying to call a Coprocessor Endpoint from within the preGet handler of a
RegionObserver, and it's throwing Class Loader issues.
The exact same Coprocessor Endpoint works perfectly from remote Java client,
however, fails from within the same Region Server.
For our particular test environment, only 1 Region Server is available, so I
guess it's a local call that fails, and perhaps a remote RegionServer
wouldn't fail, but that doesn't justify the issue :).
The Code within the RegionObserver is roughly (way reduced) as follows:
---
@Override
public void preGet(ObserverContextRegionCoprocessorEnvironment e, Get
get, ListKeyValue results)
throws IOException {
Mapbyte[], Setbyte[] results;
// scan: for all regions
try {
Batch.CallPlatformStatsIndexEndpointProtocol,Setbyte[] batchCall = new
Batch.CallPlatformStatsIndexEndpointProtocol,Setbyte[]() {
public Setbyte[]
call(PlatformStatsIndexEndpointProtocol instance) throws IOException{
return instance.getKeyTokenByPrefix(index,
match, additionalMatches);
}
};
results =
indexTable.coprocessorExec(PlatformStatsIndexEndpointProtocol.class, null,
null, batchCall);
} catch (Throwable e1) {
e1.printStackTrace();
throw new IOException(e1);
}
Setbyte[] finalResultSet = new HashSetbyte[]();
for (Map.Entrybyte[], Setbyte[] e : results.entrySet()) {
finalResultSet.addAll(e.getValue());
}
}
---
The Code for the Coprocessor Endpoint is irrelevant, as it never gets
executed.
This is the Exception I get on the Client side (Server side logged exception
below).
---
Thu Aug 23 17:37:45 CST 2012,
org.apache.hadoop.hbase.client.HTable$5@26659db7,java.io.IOException:
java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException:
interface com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol
is not visible from class loader
at
com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:100)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:553)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3737)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3639)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
Caused by: java.io.IOException: java.lang.IllegalArgumentException: interface
com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol is not
visible from class loader
at
com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokens(PlatformStatsIndexer.java:390)
at
com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokensBySubstring(PlatformStatsIndexer.java:348)
at
com.company.hbase.platformstats.IndexQueryRegionObserver.searchTokens(IndexQueryRegionObserver.java:148)
at
com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:97)
... 9 more
Caused by: java.lang.IllegalArgumentException: interface
com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol is not
visible from class loader
at java.lang.reflect.Proxy.getProxyClass(Proxy.java:353)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581)
at
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551018#comment-13551018 ] Hudson commented on HBASE-7213: --- Integrated in HBase-TRUNK #3730 (See [https://builds.apache.org/job/HBase-TRUNK/3730/]) HBASE-7213 Have HLog files for .META. edits only (Revision 1431935) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetaLogRoller.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogFactory.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7542) SCVF: Avoid sending two unwanted boolean values from client to RS
Anoop Sam John created HBASE-7542: - Summary: SCVF: Avoid sending two unwanted boolean values from client to RS Key: HBASE-7542 URL: https://issues.apache.org/jira/browse/HBASE-7542 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.94.0, 0.96.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0 In SingleColumnValueFilter the 2 boolean fields foundColumn and matchedColumn needed only at the server side during the filtering. There is no need to serialize these fields from client to server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7542) SCVF: Avoid sending two unwanted boolean values from client to RS
[ https://issues.apache.org/jira/browse/HBASE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551070#comment-13551070 ] Anoop Sam John commented on HBASE-7542: --- In 94 (older) version we can not change this code as it may break the compatibility. If some one running a client with this change and server with out(vice versa).!! SCVF: Avoid sending two unwanted boolean values from client to RS - Key: HBASE-7542 URL: https://issues.apache.org/jira/browse/HBASE-7542 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.94.0, 0.96.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0 In SingleColumnValueFilter the 2 boolean fields foundColumn and matchedColumn needed only at the server side during the filtering. There is no need to serialize these fields from client to server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7542) SCVF: Avoid sending two unwanted boolean values from client to RS
[ https://issues.apache.org/jira/browse/HBASE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-7542: -- Status: Patch Available (was: Open) SCVF: Avoid sending two unwanted boolean values from client to RS - Key: HBASE-7542 URL: https://issues.apache.org/jira/browse/HBASE-7542 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.94.0, 0.96.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7542_Trunk.patch In SingleColumnValueFilter the 2 boolean fields foundColumn and matchedColumn needed only at the server side during the filtering. There is no need to serialize these fields from client to server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7542) SCVF: Avoid sending two unwanted boolean values from client to RS
[ https://issues.apache.org/jira/browse/HBASE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-7542: -- Attachment: HBASE-7542_Trunk.patch Simple patch. Just removal of some code :) SCVF: Avoid sending two unwanted boolean values from client to RS - Key: HBASE-7542 URL: https://issues.apache.org/jira/browse/HBASE-7542 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.94.0, 0.96.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7542_Trunk.patch In SingleColumnValueFilter the 2 boolean fields foundColumn and matchedColumn needed only at the server side during the filtering. There is no need to serialize these fields from client to server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6330) TestImportExport has been failing against hadoop 0.23/2.0 profile [Part2]
[ https://issues.apache.org/jira/browse/HBASE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-6330: - Assignee: (was: Jonathan Hsieh) Removing myself from this for now. TestImportExport has been failing against hadoop 0.23/2.0 profile [Part2] - Key: HBASE-6330 URL: https://issues.apache.org/jira/browse/HBASE-6330 Project: HBase Issue Type: Sub-task Components: test Affects Versions: 0.94.1, 0.96.0 Reporter: Jonathan Hsieh Labels: hadoop-2.0 Fix For: 0.96.0 Attachments: hbase-6330-94.patch, hbase-6330-trunk.patch, hbase-6330-v2.patch See HBASE-5876. I'm going to commit the v3 patches under this name since there has been two months (my bad) since the first half was committed and found to be incomplte. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7542) SCVF: Avoid sending two unwanted boolean values from client to RS
[
https://issues.apache.org/jira/browse/HBASE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551121#comment-13551121
]
Hadoop QA commented on HBASE-7542:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12564403/HBASE-7542_Trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.replication.TestReplicationWithCompression
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.TestLocalHBaseCluster
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup.testBalancerWithRackLocality(TestBalancerWithNodeGroup.java:220)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3980//console
This message is automatically generated.
SCVF: Avoid sending two unwanted boolean values from client to RS
-
Key: HBASE-7542
URL: https://issues.apache.org/jira/browse/HBASE-7542
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.94.0, 0.96.0
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Priority: Minor
Fix For: 0.96.0
Attachments: HBASE-7542_Trunk.patch
In SingleColumnValueFilter the 2 boolean fields foundColumn and
matchedColumn needed only at the server side during the filtering. There is
no need to serialize these fields from client to server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6656) Cannot call a Coprocessor Endpoint from a RegionObserver
[
https://issues.apache.org/jira/browse/HBASE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551134#comment-13551134
]
Mauricio Morales commented on HBASE-6656:
-
The jar is stored in HDFS and the Coprocessor setup refers to the HDFS path for
the Endpoint's class location.
The RegionServer is able to load that very same jar (from HDFS) on remote
calls... it's only the local call from another Coprocessor (say a
RegionObserver) that fails.
Do you think having the jar on the CLASSPATH for the JVM should do the trick?
(this implies, however, manual deployment to all nodes instead of the
convenience of using HDFS for jar storage of Coprocessor jars.
Cannot call a Coprocessor Endpoint from a RegionObserver
Key: HBASE-6656
URL: https://issues.apache.org/jira/browse/HBASE-6656
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.92.1
Environment: CentOS5
Reporter: Mauricio Morales
I'm trying to call a Coprocessor Endpoint from within the preGet handler of a
RegionObserver, and it's throwing Class Loader issues.
The exact same Coprocessor Endpoint works perfectly from remote Java client,
however, fails from within the same Region Server.
For our particular test environment, only 1 Region Server is available, so I
guess it's a local call that fails, and perhaps a remote RegionServer
wouldn't fail, but that doesn't justify the issue :).
The Code within the RegionObserver is roughly (way reduced) as follows:
---
@Override
public void preGet(ObserverContextRegionCoprocessorEnvironment e, Get
get, ListKeyValue results)
throws IOException {
Mapbyte[], Setbyte[] results;
// scan: for all regions
try {
Batch.CallPlatformStatsIndexEndpointProtocol,Setbyte[] batchCall = new
Batch.CallPlatformStatsIndexEndpointProtocol,Setbyte[]() {
public Setbyte[]
call(PlatformStatsIndexEndpointProtocol instance) throws IOException{
return instance.getKeyTokenByPrefix(index,
match, additionalMatches);
}
};
results =
indexTable.coprocessorExec(PlatformStatsIndexEndpointProtocol.class, null,
null, batchCall);
} catch (Throwable e1) {
e1.printStackTrace();
throw new IOException(e1);
}
Setbyte[] finalResultSet = new HashSetbyte[]();
for (Map.Entrybyte[], Setbyte[] e : results.entrySet()) {
finalResultSet.addAll(e.getValue());
}
}
---
The Code for the Coprocessor Endpoint is irrelevant, as it never gets
executed.
This is the Exception I get on the Client side (Server side logged exception
below).
---
Thu Aug 23 17:37:45 CST 2012,
org.apache.hadoop.hbase.client.HTable$5@26659db7,java.io.IOException:
java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException:
interface com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol
is not visible from class loader
at
com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:100)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:553)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3737)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3639)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
Caused by: java.io.IOException: java.lang.IllegalArgumentException: interface
com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol is not
visible from class loader
at
com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokens(PlatformStatsIndexer.java:390)
at
com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokensBySubstring(PlatformStatsIndexer.java:348)
at
com.company.hbase.platformstats.IndexQueryRegionObserver.searchTokens(IndexQueryRegionObserver.java:148)
at
com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:97)
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551150#comment-13551150 ] Ted Yu commented on HBASE-7213: --- The test failure in TestSplitTransaction of QA run prevented large tests from running. I guess the failed tests (TestWALReplay, TestScannerTimeout) in trunk build #3730 were related to this patch. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551202#comment-13551202 ] Devaraj Das commented on HBASE-7213: I will look at the test failures shortly. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7477) Remove Proxy instance from HBase RPC
[ https://issues.apache.org/jira/browse/HBASE-7477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551205#comment-13551205 ] Karthik Ranganathan commented on HBASE-7477: [~saint@gmail.com] Totally, feel free to open a new one for trunk. Will definitely check out HBASE-7460. One other change that has happened in the past (which makes this easier) is that we have done away with proxy objects per conf on the client side - it used to be a singleton. Now with this patch, its just a straight up object instance. Remove Proxy instance from HBase RPC Key: HBASE-7477 URL: https://issues.apache.org/jira/browse/HBASE-7477 Project: HBase Issue Type: Sub-task Reporter: Karthik Ranganathan Attachments: 7477experiment.txt, HBASE-7477.patch Currently, we use HBaseRPC.getProxy() to get an Invoker object to serialize the RPC parameters. This is pretty inefficient as it uses reflection to lookup the current method name. The aim is to break up the proxy into an actual proxy implementation so that: 1. we can make it more efficient by eliminating reflection 2. can re-write some parts of the protocol to make it even better -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7538) Improve snapshot related error and exception messages
[ https://issues.apache.org/jira/browse/HBASE-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551288#comment-13551288 ] Matteo Bertozzi commented on HBASE-7538: TakeSnapshotHandler, DisabledTableSnapshotHandler, RestoreSnapshotHandler, and other don't seems to be covered, other jira? SnapshotDescriptionUtils.toString() doesn't have a doc string, useful just to explain why we don't use directly the toString() on the protobuf object. Improve snapshot related error and exception messages - Key: HBASE-7538 URL: https://issues.apache.org/jira/browse/HBASE-7538 Project: HBase Issue Type: Sub-task Affects Versions: hbase-6055, hbase-7290 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-7290 Attachments: hbase-7538.patch, pre-hbase-7538.patch Several of the error messages don't include the snapshot request that triggered the exception. In other places we use the toString of the protobuf SnapshotDescription which spans more than a single line. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7492) add new online-snapshot properties to hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551291#comment-13551291
]
Matteo Bertozzi commented on HBASE-7492:
+1 to expose them, nice to have in the configuration description how
increasing/decreasing them impact memory or other performance/resource related
things
add new online-snapshot properties to hbase-default.xml
---
Key: HBASE-7492
URL: https://issues.apache.org/jira/browse/HBASE-7492
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Suggested by jesse on the HBASE-6864 review.
{code}
76
/** Maximum number of concurrent snapshot region tasks that can run
concurrently */
77
private static final String CONCURENT_SNAPSHOT_TASKS_KEY =
hbase.snapshot.region.concurrentTasks;
78
private static final int DEFAULT_CONCURRENT_SNAPSHOT_TASKS = 3;
79
80
/** Conf key for number of request threads to start snapshots on
regionservers */
81
public static final String SNAPSHOT_REQUEST_THREADS_KEY =
hbase.snapshot.region.pool.threads;
82
/** # of threads for snapshotting regions on the rs. */
83
public static final int SNAPSHOT_REQUEST_THREADS_DEFAULT = 10;
84
85
/** Conf key for max time to keep threads in snapshot request pool waiting
*/
86
public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
hbase.snapshot.region.timeout;
87
/** Keep threads alive in request pool for max of 60 seconds */
88
public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
89
90
/** Conf key for millis between checks to see if snapshot completed or if
there are errors*/
91
public static final String SNAPSHOT_REQUEST_WAKE_MILLIS_KEY =
hbase.snapshot.region.wakefrequency;
92
/** Default amount of time to check for errors while regions finish
snapshotting */
93
private static final long SNAPSHOT_REQUEST_WAKE_MILLIS_DEFAULT = 500;
{code}
nit: add these to hbase-default.xml?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7477) Remove Proxy instance from HBase RPC
[ https://issues.apache.org/jira/browse/HBASE-7477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7477: - Affects Version/s: 0.89-fb Remove Proxy instance from HBase RPC Key: HBASE-7477 URL: https://issues.apache.org/jira/browse/HBASE-7477 Project: HBase Issue Type: Sub-task Affects Versions: 0.89-fb Reporter: Karthik Ranganathan Attachments: 7477experiment.txt, HBASE-7477.patch Currently, we use HBaseRPC.getProxy() to get an Invoker object to serialize the RPC parameters. This is pretty inefficient as it uses reflection to lookup the current method name. The aim is to break up the proxy into an actual proxy implementation so that: 1. we can make it more efficient by eliminating reflection 2. can re-write some parts of the protocol to make it even better -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7536) Add test that confirms that multiple concurrent snapshot requests are rejected.
[ https://issues.apache.org/jira/browse/HBASE-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551300#comment-13551300 ] Matteo Bertozzi commented on HBASE-7536: why the test has a timeout? are you expecting things hanging? are 60sec enough to fill a table and take 5 snapshot, when jenkins is busy with other large stuff? why are you using a thread instead of using the taskSnapshotAsync()? (no particular opinion on that, I like the thread) Add test that confirms that multiple concurrent snapshot requests are rejected. --- Key: HBASE-7536 URL: https://issues.apache.org/jira/browse/HBASE-7536 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-7536.patch, pre-hbase-7536.patch Currently the rule is that we can only have online snapshot running at a time. This test tries to prove this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551302#comment-13551302 ] stack commented on HBASE-7213: -- [~ted_yu] Thanks Ted. Let me back out the patch till we get green light from DD. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-7213: -- Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551306#comment-13551306 ] stack commented on HBASE-7213: -- Or, I'll let it for a few hours... if you get a chance to look see and make an addendum, that'd be sweet [~devaraj] Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7213: - Attachment: 7213v13.txt Here is what I committed last night. I fixed a long line IIRC. Putting this up in case we have to rollback. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7543) TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails
[ https://issues.apache.org/jira/browse/HBASE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-7543: - Assignee: Ted Yu TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails --- Key: HBASE-7543 URL: https://issues.apache.org/jira/browse/HBASE-7543 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu testValidateGlobalSnapshotDescriptor(org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils): Message missing required fields: name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7543) TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails
Ted Yu created HBASE-7543: - Summary: TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails Key: HBASE-7543 URL: https://issues.apache.org/jira/browse/HBASE-7543 Project: HBase Issue Type: Sub-task Reporter: Ted Yu testValidateGlobalSnapshotDescriptor(org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils): Message missing required fields: name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7543) TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails
[ https://issues.apache.org/jira/browse/HBASE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7543: -- Attachment: 7543.txt With the patch, TestSnapshotDescriptionUtils passes. TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails --- Key: HBASE-7543 URL: https://issues.apache.org/jira/browse/HBASE-7543 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Attachments: 7543.txt testValidateGlobalSnapshotDescriptor(org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils): Message missing required fields: name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7536) Add test that confirms that multiple concurrent snapshot requests are rejected.
[ https://issues.apache.org/jira/browse/HBASE-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551318#comment-13551318 ] Jonathan Hsieh commented on HBASE-7536: --- bq. why the test has a timeout? are you expecting things hanging? are 60sec enough to fill a table and take 5 snapshot, when jenkins is busy with other large stuff? In general if I have code that has a while(true) loop, I make it have a timeout. The test finishes in about 10 seconds so giving a timeout of 5-10x seems reasonable. If it isn't enough, we can bump it up. bq. why are you using a thread instead of using the taskSnapshotAsync()? (no particular opinion on that, I like the thread) No good reason, I'll change it to the async version on the next rev. Add test that confirms that multiple concurrent snapshot requests are rejected. --- Key: HBASE-7536 URL: https://issues.apache.org/jira/browse/HBASE-7536 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-7536.patch, pre-hbase-7536.patch Currently the rule is that we can only have online snapshot running at a time. This test tries to prove this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7536) Add test that confirms that multiple concurrent snapshot requests are rejected.
[ https://issues.apache.org/jira/browse/HBASE-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551318#comment-13551318 ] Jonathan Hsieh edited comment on HBASE-7536 at 1/11/13 5:48 PM: bq. why the test has a timeout? are you expecting things hanging? are 60sec enough to fill a table and take 5 snapshot, when jenkins is busy with other large stuff? In general if I have code that has a while(true) loop, I make it have a timeout. The test finishes in about 10 seconds so giving a timeout of 5-10x seems reasonable. If it isn't enough, we can bump it up. bq. why are you using a thread instead of using the taskSnapshotAsync()? (no particular opinion on that, I like the thread) No good reason, I'll change it to the async version on the next rev. was (Author: jmhsieh): bq. why the test has a timeout? are you expecting things hanging? are 60sec enough to fill a table and take 5 snapshot, when jenkins is busy with other large stuff? In general if I have code that has a while(true) loop, I make it have a timeout. The test finishes in about 10 seconds so giving a timeout of 5-10x seems reasonable. If it isn't enough, we can bump it up. bq. why are you using a thread instead of using the taskSnapshotAsync()? (no particular opinion on that, I like the thread) No good reason, I'll change it to the async version on the next rev. Add test that confirms that multiple concurrent snapshot requests are rejected. --- Key: HBASE-7536 URL: https://issues.apache.org/jira/browse/HBASE-7536 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-7536.patch, pre-hbase-7536.patch Currently the rule is that we can only have online snapshot running at a time. This test tries to prove this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7543) TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails
[ https://issues.apache.org/jira/browse/HBASE-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551328#comment-13551328 ] Jonathan Hsieh commented on HBASE-7543: --- Hey Ted, I think you maybe working off of one on of the personal dev branches that hasn't been committed yet. The official branch with committed code is is currently here: https://github.com/jmhsieh/hbase/tree/snapshots Can you keep this around for when we get to that? (this is likely part of HBASE-6867). Also, I've unassigned myself from some of the issues that I'm not currently focused on. If you are interested in the global snapshot that Jesse's assigned to, ping him to see if he is still working on them. TestSnapshotDescriptionUtils#testValidateGlobalSnapshotDescriptor fails --- Key: HBASE-7543 URL: https://issues.apache.org/jira/browse/HBASE-7543 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Attachments: 7543.txt testValidateGlobalSnapshotDescriptor(org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils): Message missing required fields: name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7538) Improve snapshot related error and exception messages
[ https://issues.apache.org/jira/browse/HBASE-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551329#comment-13551329 ] Jonathan Hsieh commented on HBASE-7538: --- bq. TakeSnapshotHandler, DisabledTableSnapshotHandler, RestoreSnapshotHandler, and other don't seems to be covered, other jira? Probably because these came from when I was debugging one problem. I'll go through and do the others as well and submit a new patch. bq. SnapshotDescriptionUtils.toString() doesn't have a doc string, useful just to explain why we don't use directly the toString() on the protobuf object. will add better docs explaining why we added this. Improve snapshot related error and exception messages - Key: HBASE-7538 URL: https://issues.apache.org/jira/browse/HBASE-7538 Project: HBase Issue Type: Sub-task Affects Versions: hbase-6055, hbase-7290 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-7290 Attachments: hbase-7538.patch, pre-hbase-7538.patch Several of the error messages don't include the snapshot request that triggered the exception. In other places we use the toString of the protobuf SnapshotDescription which spans more than a single line. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7539) when region is closed, the closing server should null out location in meta in process
[ https://issues.apache.org/jira/browse/HBASE-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551331#comment-13551331 ] Sergey Shelukhin commented on HBASE-7539: - [~stack] it retries with wait, which is probably the desired behavior if region is being opened somewhere when region is closed, the closing server should null out location in meta in process - Key: HBASE-7539 URL: https://issues.apache.org/jira/browse/HBASE-7539 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin No point in having stale META record when the server that put it has already closed the region. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7480) Explicit message for not allowed snapshot on meta tables
[
https://issues.apache.org/jira/browse/HBASE-7480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-7480:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
committed to the snapshots branch
Explicit message for not allowed snapshot on meta tables
Key: HBASE-7480
URL: https://issues.apache.org/jira/browse/HBASE-7480
Project: HBase
Issue Type: Sub-task
Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: hbase-6055, 0.96.0
Attachments: HBASE-7480-v0.patch
taking a snapshot of -ROOT- or .META. now results in something like this:
{code}
Illegal first character 46 at 0. User-space table names can only start with
'word characters': i.e. [a-zA-Z_0-9]
{code}
changing the message in something more human readable to inform that meta
table are not supported
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7484) Fix Restore with schema changes
[ https://issues.apache.org/jira/browse/HBASE-7484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7484: --- Resolution: Fixed Status: Resolved (was: Patch Available) committed to the snapshots branch Fix Restore with schema changes --- Key: HBASE-7484 URL: https://issues.apache.org/jira/browse/HBASE-7484 Project: HBase Issue Type: Sub-task Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Blocker Fix For: hbase-6055, hbase-7290 Attachments: HBASE-7484-v0.patch, HBASE-7484-v1.patch The restore code in the offline branch doesn't handle the schema change. I think that I've lost it during the various rebase, the Handler restore the schema, but the restoreRegion() method in the helper handle just the same schema case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7453) HBASE-7423 snapshot followup
[ https://issues.apache.org/jira/browse/HBASE-7453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7453: --- Resolution: Fixed Status: Resolved (was: Patch Available) committed to the snapshots branch HBASE-7423 snapshot followup Key: HBASE-7453 URL: https://issues.apache.org/jira/browse/HBASE-7453 Project: HBase Issue Type: Sub-task Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Fix For: hbase-6055, hbase-7290 Attachments: HBASE-7453-v0.patch, HBASE-7453-v1.patch HBASE-7423 change the arguments for one method used by restore code -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7521) fix HBASE-6060 (regions stuck in opening state) in 0.94
[ https://issues.apache.org/jira/browse/HBASE-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551350#comment-13551350 ] Sergey Shelukhin commented on HBASE-7521: - Hmm, I see. For the former, it seems like hijack flag should take care of that as in case with timeout. I'll look into the 2nd one. fix HBASE-6060 (regions stuck in opening state) in 0.94 --- Key: HBASE-7521 URL: https://issues.apache.org/jira/browse/HBASE-7521 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7521-v0.patch, HBASE-7521-v1.patch Discussion in HBASE-6060 implies that the fix there does not work on 0.94. Still, we may want to fix the issue in 0.94 (via some different fix) because the regions stuck in opening for ridiculous amounts of time is not a good thing to have. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7540: --- Attachment: Screen Shot 2013-01-11 at 10.23.50 AM.png Screenshot showing replication related znodes in the zkDump now. (on the master UI) Make znode dump to print a dump of replciation znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6988) single-node HBASE (or at least shell) went into inconsistent state
[
https://issues.apache.org/jira/browse/HBASE-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin resolved HBASE-6988.
-
Resolution: Cannot Reproduce
no repro
single-node HBASE (or at least shell) went into inconsistent state
--
Key: HBASE-6988
URL: https://issues.apache.org/jira/browse/HBASE-6988
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Priority: Minor
Thru the cycle of restarting HBase and shell, I was somehow able to get into
this state (on trunk):
{noformat}
hbase(main):020:0 describe 'test'
DESCRIPTION
ENABLED
'test', {METHOD = 'table_att', CONFIG = {'Key' = 'Value2', 'Key2' =
'Value2'}}, {NAME true
= 'cf', DATA_BLOCK_ENCODING = 'NONE', BLOOMFILTER = 'NONE',
REPLICATION_SCOPE = '0',
COMPRESSION = 'NONE', VERSIONS = '3', TTL = '2147483647', MIN_VERSIONS
= '0', KEEP_D
ELETED_CELLS = 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'true',
IN_MEMORY = 'fa
lse', BLOCKCACHE = 'true'}
1 row(s) in 0.0420 seconds
hbase(main):021:0 disable 'test'
ERROR: Table test does not exist.'
Here is some help for this command:
Start disable of named table: e.g. hbase disable 't1'
hbase(main):022:0 create 'test', 'cf2'
0 row(s) in 1.0900 seconds
= Hbase::Table - test
hbase(main):023:0 describe 'test'
DESCRIPTION
ENABLED
'test', {METHOD = 'table_att', CONFIG = {'Key' = 'Value2', 'Key2' =
'Value2'}}, {NAME true
= 'cf', DATA_BLOCK_ENCODING = 'NONE', BLOOMFILTER = 'NONE',
REPLICATION_SCOPE = '0',
COMPRESSION = 'NONE', VERSIONS = '3', TTL = '2147483647', MIN_VERSIONS
= '0', KEEP_D
ELETED_CELLS = 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'true',
IN_MEMORY = 'fa
lse', BLOCKCACHE = 'true'}
1 row(s) in 0.0440 seconds
hbase(main):024:0 disable 'test'
0 row(s) in 7.0590 seconds
hbase(main):026:0 alter 'test', METHOD = 'table_att', CONFIG = { 'Key' =
'Value3' }, MAX_FILESIZE = 1234567
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.1210 seconds
hbase(main):027:0 describe 'test'
DESCRIPTION
ENABLED
'test', {METHOD = 'table_att', MAX_FILESIZE = '1234567', CONFIG = {'Key'
= 'Value3', false
'Key2' = 'Value2'}}, {NAME = 'cf', DATA_BLOCK_ENCODING = 'NONE',
BLOOMFILTER = 'NONE'
, REPLICATION_SCOPE = '0', COMPRESSION = 'NONE', VERSIONS = '3', TTL =
'2147483647',
MIN_VERSIONS = '0', KEEP_DELETED_CELLS = 'false', BLOCKSIZE = '65536',
ENCODE_ON_DISK
= 'true', IN_MEMORY = 'false', BLOCKCACHE = 'true'}
1 row(s) in 0.0400 seconds
hbase(main):029:0 enable 'test'
0 row(s) in 7.0710 seconds
hbase(main):030:0 put 'test', 'row1', 'cf:foo', 'bar'
0 row(s) in 0.0240 seconds
hbase(main):031:0 put 'test', 'row1', 'cf2:foo', 'bar'
ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family cf2 does not exist in region
test,,1350063293064.7867533297035efe96611b5bcc54f332. in table 'test',
{METHOD = 'table_att', MAX_FILESIZE = '1234567', CONFIG = {'Key' =
'Value3', 'Key2' = 'Value2'}}, {NAME = 'cf', DATA_BLOCK_ENCODING = 'NONE',
BLOOMFILTER = 'NONE', REPLICATION_SCOPE = '0', COMPRESSION = 'NONE',
VERSIONS = '3', TTL = '2147483647', MIN_VERSIONS = '0', KEEP_DELETED_CELLS
= 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'true', IN_MEMORY =
'false', BLOCKCACHE = 'true'}
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent
[jira] [Commented] (HBASE-6324) Direct API calls from embedded Thrift server to regionserver
[ https://issues.apache.org/jira/browse/HBASE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551359#comment-13551359 ] Sergey Shelukhin commented on HBASE-6324: - Should this be resolved won't fix then? Direct API calls from embedded Thrift server to regionserver Key: HBASE-6324 URL: https://issues.apache.org/jira/browse/HBASE-6324 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin When handling Thrift calls in the regionserver we should not go through RPC to talk to the local regionserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7502) TestScannerTimeout fails on snapshot branch
[
https://issues.apache.org/jira/browse/HBASE-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551358#comment-13551358
]
Ted Yu commented on HBASE-7502:
---
Integrated patch v2 to trunk.
TestScannerTimeout fails on snapshot branch
---
Key: HBASE-7502
URL: https://issues.apache.org/jira/browse/HBASE-7502
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: hbase-7290
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.96.0, 0.94.5, hbase-7290
Attachments: HBASE-7502-v1.patch, HBASE-7502-v2.patch
TestScannerTimeout#test3686a fails consistently on snapshot branch. This is
because there is an increase in the number of watches on the rs znode and its
deletion takes more time now. The repercussion is that when test3686a starts,
it ensures that there are two regionservers and it counts the aborted
regionserver as a live one. While processing, it kills one of its server, and
also the znode of the previously aborted server expires. Overall effect is
there are no regionservers now, and client hangs.
{code}
Error Message
test timed out after 30 milliseconds
Stacktrace
java.lang.Exception: test timed out after 30 milliseconds
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.close(HConnectionManager.java:1769)
at org.apache.hadoop.hbase.client.HTable.close(HTable.java:961)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:180)
at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
at
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551360#comment-13551360 ] Sergey Shelukhin commented on HBASE-3909: - this appears to have stalled Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: Bug Reporter: stack Assignee: Subbu M Iyer Fix For: 0.96.0 Attachments: 3909_090712-2.patch, 3909-102812.patch, 3909-102912.patch, 3909.v1, 3909-v1.patch, HBase Cluster Config Details.xlsx, patch-v2.patch, testMasterNoCluster.stack I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6371) [89-fb] Tier based compaction
[ https://issues.apache.org/jira/browse/HBASE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551362#comment-13551362 ] Sergey Shelukhin commented on HBASE-6371: - Should this issue be resolved? Backport is handled in HBASE-7055 and assorted spin-off issues. [89-fb] Tier based compaction - Key: HBASE-6371 URL: https://issues.apache.org/jira/browse/HBASE-6371 Project: HBase Issue Type: Improvement Reporter: Akashnil Assignee: Liyin Tang Labels: noob Attachments: HBASE-6371-089fb-commit.patch, HBase_Tier_Base_Compaction.pdf Currently, the compaction selection is not very flexible and is not sensitive to the hotness of the data. Very old data is likely to be accessed less, and very recent data is likely to be in the block cache. Both of these considerations make it inefficient to compact these files as aggressively as other files. In some use-cases, the access-pattern is particularly obvious even though there is no way to control the compaction algorithm in those cases. In the new compaction selection algorithm, we plan to divide the candidate files into different levels according to oldness of the data that is present in those files. For each level, parameters like compaction ratio, minimum number of store-files in each compaction may be different. Number of levels, time-ranges, and parameters for each level will be configurable online on a per-column family basis. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7039) Port HBASE-5914 Bulk assign regions in the process of ServerShutdownHandler (and bugfix part of HBASE-6012) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7039: Resolution: Won't Fix Status: Resolved (was: Patch Available) Decided not to port, due to protocol change considerations in the bugfix. Port HBASE-5914 Bulk assign regions in the process of ServerShutdownHandler (and bugfix part of HBASE-6012) to 0.94 --- Key: HBASE-7039 URL: https://issues.apache.org/jira/browse/HBASE-7039 Project: HBase Issue Type: Task Affects Versions: 0.94.2 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7039-squashed.patch, test-am.log, test.log, test-rerun.log This is a major feature, so please -1 if you think it's too dangerous to port. However, it's also a perf improvement for recovery. The 2nd thing that HBASE-6012 addresses cannot be included without a breaking interface change (HRegionInterface openRegions doesn't return region states which are relied upon by the trunk code that is using protocol buffers API); or a non-breaking interface change with version-checking hackery to take advantage of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7316) HBase region server retries are not always smart. Make smarter/replace with request timeouts?
[ https://issues.apache.org/jira/browse/HBASE-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin resolved HBASE-7316. - Resolution: Invalid bad jira on my part, no done criteria HBase region server retries are not always smart. Make smarter/replace with request timeouts? - Key: HBASE-7316 URL: https://issues.apache.org/jira/browse/HBASE-7316 Project: HBase Issue Type: Brainstorming Components: Client Reporter: Sergey Shelukhin Priority: Minor Example can be seen in HBASE-7250. Server A has region, region is moved to B, B dies, region starts moving to C. Client goes to A, A sends him to B, B is dead so he refreshes meta (which has A), it goes to A, which sends it to B, ... These retries are unnecessary. Overall, I wonder if retry policy should be replaced by timeout policy, preferably configurable on per-request basis by client? That sounds like a reasonable feature to me, so if the timeout is set to high values retries will not be exhausted due to situations like this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7317) consider full-featured support for HTrace for HBase debugging
[ https://issues.apache.org/jira/browse/HBASE-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7317: Summary: consider full-featured support for HTrace for HBase debugging (was: server-side request problems are hard to debug) consider full-featured support for HTrace for HBase debugging --- Key: HBASE-7317 URL: https://issues.apache.org/jira/browse/HBASE-7317 Project: HBase Issue Type: Brainstorming Components: IPC/RPC, regionserver Reporter: Sergey Shelukhin Priority: Minor I've seen cases during integration tests where the write or read request took an unexpectedly large amount of time (that, after the client went to the region server that is reported alive and well, which I know from temporary debug logging :)), and it's impossible to understand what is going on on the server side, short of catching the moment with jstack. Some solutions (off by default) could be - a facility for tests (especially integration tests) that would trace Server/Master calls into some log or file (won't help with internals but at least one could see what was actually received); - logging the progress of requests between components inside master/server (e.g. request id=N received, request id=N is being processed in MyClass, N being drawn on client from local sequence - no guarantees of uniqueness are necessary). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7317) consider full-featured support for HTrace for HBase debugging
[ https://issues.apache.org/jira/browse/HBASE-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551365#comment-13551365 ] Sergey Shelukhin commented on HBASE-7317: - I will probably take a look into this, but not soon... consider full-featured support for HTrace for HBase debugging --- Key: HBASE-7317 URL: https://issues.apache.org/jira/browse/HBASE-7317 Project: HBase Issue Type: Brainstorming Components: IPC/RPC, regionserver Reporter: Sergey Shelukhin Priority: Minor I've seen cases during integration tests where the write or read request took an unexpectedly large amount of time (that, after the client went to the region server that is reported alive and well, which I know from temporary debug logging :)), and it's impossible to understand what is going on on the server side, short of catching the moment with jstack. Some solutions (off by default) could be - a facility for tests (especially integration tests) that would trace Server/Master calls into some log or file (won't help with internals but at least one could see what was actually received); - logging the progress of requests between components inside master/server (e.g. request id=N received, request id=N is being processed in MyClass, N being drawn on client from local sequence - no guarantees of uniqueness are necessary). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7535) Fix restore reference files
[
https://issues.apache.org/jira/browse/HBASE-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551367#comment-13551367
]
Jonathan Hsieh commented on HBASE-7535:
---
I've been testing this at the system test level and this has fixed a commonly
occurring problem, so functionally I'm +1.
I'm not sure I get the OPEN part of this name. Can we just be more specific
with the comment and pick a better name for LINK_NAME_OPEN_PATTERN? Maybe
REF_OR_HFILE_LINK_PATTERN?
{code}
+ /** The link should be used for everything that can be found in
/hbase/table/region/family/ */
+ private static final Pattern LINK_NAME_OPEN_PATTERN =
+Pattern.compile(String.format(^(%s)=(%s)-(.+)$,
HTableDescriptor.VALID_USER_TABLE_REGEX,
+ HRegionInfo.ENCODED_REGION_NAME_REGEX));
+
{code}
Add an example in comments of where we start from and what we get?
{code}
private void restoreReferenceFile(final Path familyDir, final HRegionInfo
regionInfo,
final String hfileName) throws IOException {
-Path inPath = new Path(new Path(new Path(snapshotDesc.getTable(),
-regionInfo.getEncodedName()), familyDir.getName()), hfileName);
-Path outPath = new Path(familyDir,
StoreFile.getReferredToFile(inPath).getName());
-InputStream in = new HFileLink(conf, inPath).open(fs);
+// Extract the referred information (hfile name and parent region)
+
{code}
Fix restore reference files
---
Key: HBASE-7535
URL: https://issues.apache.org/jira/browse/HBASE-7535
Project: HBase
Issue Type: Sub-task
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Blocker
Attachments: HBASE-7535-v0.patch, HBASE-7535-v1.patch
After HBASE-7419 the HFileLink regex became stricter, to have the proper
isHFileLink() check.
but HFileLink should open both reference and hfiles
since the main idea behind it is open stuff in /table/region/family/XYZ
This patch fix the reference (split files) restore problem and open the
hfilelink regex for HFileLink(/table/region/family/xyz).open()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7535) Fix restore reference files
[ https://issues.apache.org/jira/browse/HBASE-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551369#comment-13551369 ] Ted Yu commented on HBASE-7535: --- I had similar thinking about LINK_NAME_OPEN_PATTERN when I looked at the patch. Fix restore reference files --- Key: HBASE-7535 URL: https://issues.apache.org/jira/browse/HBASE-7535 Project: HBase Issue Type: Sub-task Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Blocker Attachments: HBASE-7535-v0.patch, HBASE-7535-v1.patch After HBASE-7419 the HFileLink regex became stricter, to have the proper isHFileLink() check. but HFileLink should open both reference and hfiles since the main idea behind it is open stuff in /table/region/family/XYZ This patch fix the reference (split files) restore problem and open the hfilelink regex for HFileLink(/table/region/family/xyz).open() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7535) Fix restore reference files
[ https://issues.apache.org/jira/browse/HBASE-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551369#comment-13551369 ] Ted Yu edited comment on HBASE-7535 at 1/11/13 6:41 PM: I had similar thought about LINK_NAME_OPEN_PATTERN when I looked at the patch. was (Author: yuzhih...@gmail.com): I had similar thinking about LINK_NAME_OPEN_PATTERN when I looked at the patch. Fix restore reference files --- Key: HBASE-7535 URL: https://issues.apache.org/jira/browse/HBASE-7535 Project: HBase Issue Type: Sub-task Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Blocker Attachments: HBASE-7535-v0.patch, HBASE-7535-v1.patch After HBASE-7419 the HFileLink regex became stricter, to have the proper isHFileLink() check. but HFileLink should open both reference and hfiles since the main idea behind it is open stuff in /table/region/family/XYZ This patch fix the reference (split files) restore problem and open the hfilelink regex for HFileLink(/table/region/family/xyz).open() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7540: --- Attachment: HBASE-7540-trunk.patch trunk patch; ran the master UI to see the replication znodes dump. Make znode dump to print a dump of replciation znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7540: --- Status: Patch Available (was: Open) Make znode dump to print a dump of replciation znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[
https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551386#comment-13551386
]
Andrew Purtell commented on HBASE-4210:
---
An option here would be to allocate an internal identifier for every operation,
corresponding to the user RPC, and then no matter how many units of work that
is broken down internally the RPC ID would be carried through. Then there could
be a CP hook at step 3 which announces processing on a given RPC (batch). The
other hooks will get the ID somehow so the CP can correlate. Then finally at
step 8 an upcall notifying that processing the given RPC ID is completed (or
errored out).
Allow coprocessor to interact with batches per region sent from a client(?)
---
Key: HBASE-4210
URL: https://issues.apache.org/jira/browse/HBASE-4210
Project: HBase
Issue Type: New Feature
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Assignee: Anoop Sam John
Priority: Minor
Fix For: 0.96.0, 0.94.5
Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly
one row|cell operations.
It might be a good idea to allow a coprocessor to deal with batches of puts
and deletes as they arrive from the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[
https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551387#comment-13551387
]
Ted Yu commented on HBASE-7213:
---
TestCatalogTrackerOnCluster failed in trunk builds #3730 and #3731. It fails
locally as well.
I saw the following in test output:
{code}
2013-01-11 10:42:33,443 ERROR [Shutdown of
org.apache.hadoop.hbase.fs.HFileSystem@2dc4df0b] hdfs.DFSClient(416): Failed to
close file
/user/tyu/hbase/.logs/10.10.8.161,51511,1357929747103/10.10.8.161%2C51511%2C1357929747103.1357929752224.meta
org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on
/user/tyu/hbase/.logs/10.10.8.161,51511,1357929747103/10.10.8.161%2C51511%2C1357929747103.1357929752224.meta
File does not exist. [Lease. Holder:
DFSClient_hb_rs_10.10.8.161,51511,1357929747103_891897357_107, pendingcreates:
1]
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1720)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1711)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1619)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:729)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
{code}
Will attach test output.
Have HLog files for .META. edits only
-
Key: HBASE-7213
URL: https://issues.apache.org/jira/browse/HBASE-7213
Project: HBase
Issue Type: Improvement
Components: master, regionserver
Reporter: Devaraj Das
Assignee: Devaraj Das
Priority: Critical
Fix For: 0.96.0
Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch,
7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch,
7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt
Over on HBASE-6774, there is a discussion on separating out the edits for
.META. regions from the other regions' edits w.r.t where the edits are
written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7213: -- Attachment: TEST-org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster.xml Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt, TEST-org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster.xml Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7329) remove flush-related records from WAL
[ https://issues.apache.org/jira/browse/HBASE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551390#comment-13551390 ] Sergey Shelukhin commented on HBASE-7329: - It looks like the patch doesn't contain the test for new class, which in all likelihood is nuked because I never committed it to local git :( I am going to rebase the patch, recover or rewrite the test, write up some explanation and post it on /r/, for now please continue disregarding this JIRA :) remove flush-related records from WAL - Key: HBASE-7329 URL: https://issues.apache.org/jira/browse/HBASE-7329 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7329-v0.patch, HBASE-7329-v0.patch, HBASE-7329-v0-tmp.patch, HBASE-7329-v1.patch, HBASE-7329-v1.patch Comments from many people in HBASE-6466 and HBASE-6980 indicate that flush records in WAL are not useful. If so, they should be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551412#comment-13551412 ] stack commented on HBASE-7213: -- I reverted for now. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt, TEST-org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster.xml Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7540: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Resolving. [~lhofhansl] You want this? Make znode dump to print a dump of replciation znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551427#comment-13551427 ] Devaraj Das commented on HBASE-7213: Yeah the revert makes sense. Things may have changed in the codebase in the last couple of weeks leading to some test failures. I'm investigating more deeply now. Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt, TEST-org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster.xml Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6324) Direct API calls from embedded Thrift server to regionserver
[ https://issues.apache.org/jira/browse/HBASE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6324. -- Resolution: Invalid Resolving as no longer valid given running thrift server in regionserver context an abandoned experiment. Direct API calls from embedded Thrift server to regionserver Key: HBASE-6324 URL: https://issues.apache.org/jira/browse/HBASE-6324 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin When handling Thrift calls in the regionserver we should not go through RPC to talk to the local regionserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7502) TestScannerTimeout fails on snapshot branch
[
https://issues.apache.org/jira/browse/HBASE-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551431#comment-13551431
]
Hudson commented on HBASE-7502:
---
Integrated in HBase-TRUNK #3732 (See
[https://builds.apache.org/job/HBase-TRUNK/3732/])
HBASE-7502 TestScannerTimeout fails on snapshot branch (Himanshu) (Revision
1432211)
Result = FAILURE
tedyu :
Files :
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannerTimeout.java
TestScannerTimeout fails on snapshot branch
---
Key: HBASE-7502
URL: https://issues.apache.org/jira/browse/HBASE-7502
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: hbase-7290
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.96.0, 0.94.5, hbase-7290
Attachments: HBASE-7502-v1.patch, HBASE-7502-v2.patch
TestScannerTimeout#test3686a fails consistently on snapshot branch. This is
because there is an increase in the number of watches on the rs znode and its
deletion takes more time now. The repercussion is that when test3686a starts,
it ensures that there are two regionservers and it counts the aborted
regionserver as a live one. While processing, it kills one of its server, and
also the znode of the previously aborted server expires. Overall effect is
there are no regionservers now, and client hangs.
{code}
Error Message
test timed out after 30 milliseconds
Stacktrace
java.lang.Exception: test timed out after 30 milliseconds
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.close(HConnectionManager.java:1769)
at org.apache.hadoop.hbase.client.HTable.close(HTable.java:961)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:180)
at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
at
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6466) Enable multi-thread for memstore flush
[ https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551436#comment-13551436 ] Sergey Shelukhin commented on HBASE-6466: - By the way, do we want this in 0.94? I can backport Enable multi-thread for memstore flush -- Key: HBASE-6466 URL: https://issues.apache.org/jira/browse/HBASE-6466 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.96.0 Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0 Attachments: HBASE-6466.patch, HBASE-6466v2.patch, HBASE-6466v3.1.patch, HBASE-6466v3.patch, HBASE-6466-v4.patch, HBASE-6466-v4.patch If the KV is large or Hlog is closed with high-pressure putting, we found memstore is often above the high water mark and block the putting. So should we enable multi-thread for Memstore Flush? Some performance test data for reference, 1.test environment ： random writting；upper memstore limit 5.6GB;lower memstore limit 4.8GB;400 regions per regionserver；row len=50 bytes, value len=1024 bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler per client for writing 2.test results: one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per regionserver, appears many aboveGlobalMemstoreLimit blocking two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per regionserver, 200 thread handler per client two cacheFlush handlers, tps:16.1k/s per regionserver, Flush:18.6MB/s per regionserver -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7441) Make ClusterManager in IntegrationTestingUtility pluggable
[ https://issues.apache.org/jira/browse/HBASE-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551438#comment-13551438 ] Enis Soztutar commented on HBASE-7441: -- Actually this hasn't been committed to 0.94. We should either commit it there, or remove the fix versions. I'd say go with the commit since this is pretty simple patch just affecting ITs. Make ClusterManager in IntegrationTestingUtility pluggable -- Key: HBASE-7441 URL: https://issues.apache.org/jira/browse/HBASE-7441 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Labels: newbie, patch Fix For: 0.96.0, 0.94.5 Attachments: HBASE-7441-0.94-v1.patch, HBASE-7441-trunk-v1.patch, HBASE-7441-trunk-v2.patch Original Estimate: 3h Remaining Estimate: 3h After the patch HBASE-7009, we can use ChaosMonkey to test the Hbase cluster. The ClusterManager use ssh to stop/start the rs or master without passwd. To support other cluster manager tool, we need to make clusterManager in IntegrationTestingUtility pluggable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7533) Write an RPC Specification for 0.96
[ https://issues.apache.org/jira/browse/HBASE-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551453#comment-13551453 ] Elliott Clark commented on HBASE-7533: -- The Union message types are structured after https://developers.google.com/protocol-buffers/docs/techniques#union. We should move the enum out of the Request/Response messages. That will clean some things up a little. As far as the exception being in the enum, my thinking was as follows: * For multi we could say what type the response should be. * Then if there's an exception put that in the exceptions field. * If there was a partial result it could still go in the result field. This would allow us to give partial results (a new feature for the 0.98ish time frame). It would also allow us to just give the features that we have currently. Write an RPC Specification for 0.96 --- Key: HBASE-7533 URL: https://issues.apache.org/jira/browse/HBASE-7533 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.96.0 RPC format is changing for 0.96 to accomodate our protobufing all around. Here is a first cut. Please shred: https://docs.google.com/document/d/1-1RJMLXzYldmHgKP7M7ynK6euRpucD03fZ603DlZfGI/edit -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[
https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551455#comment-13551455
]
Hadoop QA commented on HBASE-7540:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12564465/HBASE-7540-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.replication.TestReplicationWithCompression
org.apache.hadoop.hbase.TestLocalHBaseCluster
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup.testBalancerWithRackLocality(TestBalancerWithNodeGroup.java:220)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3981//console
This message is automatically generated.
Make znode dump to print a dump of replciation znodes
-
Key: HBASE-7540
URL: https://issues.apache.org/jira/browse/HBASE-7540
Project: HBase
Issue Type: Improvement
Components: Replication, UI
Affects Versions: 0.94.3
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.96.0, 0.94.4
Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot
2013-01-11 at 10.23.50 AM.png
It will be nice to have a dump of replication related znodes on the master UI
(along with other znode dump). It helps while using replication.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server response path
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551460#comment-13551460 ] stack commented on HBASE-5945: -- [~devaraj] Looking at the patch, there is a bunch of overlap with HBASE-7533 where we are trying to spec out what goes on wire before coding it up. What you reckon? If we remove the 7533 stuff from this patch, what is left over? What buffer copies are we saving in particular? Good on you boss. Reduce buffer copies in IPC server response path Key: HBASE-5945 URL: https://issues.apache.org/jira/browse/HBASE-5945 Project: HBase Issue Type: Improvement Components: IPC/RPC Affects Versions: 0.96.0 Reporter: Todd Lipcon Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 5945-in-progress.2.1.patch, 5945-in-progress.2.patch, 5945-in-progress.patch, buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7441) Make ClusterManager in IntegrationTestingUtility pluggable
[ https://issues.apache.org/jira/browse/HBASE-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551464#comment-13551464 ] stack commented on HBASE-7441: -- Committed to 0.94 branch too (hope you don't mind Lars -- just change in IT stuff) Make ClusterManager in IntegrationTestingUtility pluggable -- Key: HBASE-7441 URL: https://issues.apache.org/jira/browse/HBASE-7441 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Labels: newbie, patch Fix For: 0.96.0, 0.94.5 Attachments: HBASE-7441-0.94-v1.patch, HBASE-7441-trunk-v1.patch, HBASE-7441-trunk-v2.patch Original Estimate: 3h Remaining Estimate: 3h After the patch HBASE-7009, we can use ChaosMonkey to test the Hbase cluster. The ClusterManager use ssh to stop/start the rs or master without passwd. To support other cluster manager tool, we need to make clusterManager in IntegrationTestingUtility pluggable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server response path
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551467#comment-13551467 ] Devaraj Das commented on HBASE-5945: [~stack], yes, there is overlap... I am currently trying to fix up HBASE-7213's test failures. I'll circle back on HBASE-7533 sometime by end of today or tomorrow. We can keep 7533 as a spec jira and use this for impl. Does it work. Reduce buffer copies in IPC server response path Key: HBASE-5945 URL: https://issues.apache.org/jira/browse/HBASE-5945 Project: HBase Issue Type: Improvement Components: IPC/RPC Affects Versions: 0.96.0 Reporter: Todd Lipcon Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 5945-in-progress.2.1.patch, 5945-in-progress.2.patch, 5945-in-progress.patch, buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551468#comment-13551468 ] Enis Soztutar commented on HBASE-7411: -- Thanks Ted for the review. I'll address those in the next one. From my understanding so far for Curator, and the discussions here, we should be doing either the approach in the patch, or close this as won't fix. I would prefer to reach a consensus sooner, rather than later if possible. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6640) [0.89-fb] Allow multiple regions to be opened simultaneously
[ https://issues.apache.org/jira/browse/HBASE-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-6640. -- Resolution: Fixed Fix Version/s: (was: 0.96.0) 0.89-fb This has been committed to 0.89-fb. Removing out of 0.96, since we already handle region opening / closing in parallel using ExecutorService. [0.89-fb] Allow multiple regions to be opened simultaneously Key: HBASE-6640 URL: https://issues.apache.org/jira/browse/HBASE-6640 Project: HBase Issue Type: Improvement Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Critical Fix For: 0.89-fb Attachments: 0001-HBASE-6640-0.89-fb-Allow-multiple-regions-to-be-open.patch Allow regions to be opened in parallel. This should reduce the time it takes to replay and reopen regions in case of a unclean restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7295) Contention in HBaseClient.getConnection
[ https://issues.apache.org/jira/browse/HBASE-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551490#comment-13551490 ] Gary Helmling commented on HBASE-7295: -- The HBASE-7460 changes don't directly touch the internal HBaseClient PoolMap of connection instances. What it's changing instead by removing the ClientCache is making there be a 1-1 correlation for each HConnectionManager.HConnectionImplementation instance and a HBaseClient instance. HBaseClient internally, in the current version at least, remains intact. Contention in HBaseClient.getConnection --- Key: HBASE-7295 URL: https://issues.apache.org/jira/browse/HBASE-7295 Project: HBase Issue Type: Improvement Affects Versions: 0.94.3 Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.96.0, 0.94.5 Attachments: 7295-0.94.txt, 7295-0.94-v2.txt, 7295-0.94-v3.txt, 7295-0.94-v4.txt, 7295-0.94-v5.txt, 7295-trunk.txt, 7295-trunk.txt, 7295-trunk-v2.txt, 7295-trunk-v3.txt, 7295-trunk-v3.txt HBaseClient.getConnection() synchronizes on the connections object. We found severe contention on a thrift gateway which was fanning out roughly 3000+ calls per second to hbase region servers. The thrift gateway had 2000+ threads for handling incoming connections. Threads were blocked on the syncrhonized block - we set ipc.pool.size to 200. Since we are using RoundRobin/ThreadLocal pool only - its not necessary to synchronize on connections - it might lead to cases where we might go slightly over the ipc.max.pool.size() but the additional connections would timeout after maxIdleTime - underlying PoolMap connections object is thread safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551495#comment-13551495 ] Hudson commented on HBASE-7213: --- Integrated in HBase-TRUNK #3733 (See [https://builds.apache.org/job/HBase-TRUNK/3733/]) HBASE-7213 Have HLog files for .META. edits only; REVERT (Revision 1432234) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetaLogRoller.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogFactory.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java Have HLog files for .META. edits only - Key: HBASE-7213 URL: https://issues.apache.org/jira/browse/HBASE-7213 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 7213-2.10.patch, 7213-2.11.patch, 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-2.9.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch, 7213v13.txt, TEST-org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster.xml Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7540) Make znode dump to print a dump of replciation znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551496#comment-13551496 ] Hudson commented on HBASE-7540: --- Integrated in HBase-TRUNK #3733 (See [https://builds.apache.org/job/HBase-TRUNK/3733/]) HBASE-7540 Make znode dump to print a dump of replciation znodes (Revision 1432235) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java Make znode dump to print a dump of replciation znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7544) Transparent HFile encryption
Andrew Purtell created HBASE-7544:
-
Summary: Transparent HFile encryption
Key: HBASE-7544
URL: https://issues.apache.org/jira/browse/HBASE-7544
Project: HBase
Issue Type: New Feature
Components: HFile, io
Affects Versions: 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Introduce transparent encryption of HBase on disk data.
Depends on a separate contribution of an encryption codec framework to Hadoop
core and an AES-NI (native code) codec.
I have an experimental patch that introduces encryption at the HFile level,
with all necessary changes propagated up to the HStore level. For the most
part, the changes are straightforward and mechanical. After HBASE-7414, we can
introduce specification of an optional encryption codec in the file trailer.
The work is not ready to go yet because key management and the HBCK pieces are
TBD.
Requirements:
- Mechanisms not exposed to or modifiable by users
- Transparent encryption at the CF or table level
- Built-in key management
- Flexible and non-intrusive key rotation
- Two-tier key architecture for consistency with best practices for this
feature in the RDBMS world
- Transparent encryption of sensitive application columns
- Protect against all data leakage from files at rest
- Hardware security module integration (via Java KeyStore)
- HBCK support for transparently encrypted files (+ plugin architecture for
HBCK)
We're aiming for rough parity with Oracle's transparent tablespace encryption
feature, described in
http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
{quote}
“Transparent Data Encryption uses a 2-tier key architecture for flexible and
non-intrusive key rotation and least operational and performance impact: Each
application table with at least one encrypted column has its own table key,
which is applied to all encrypted columns in that table. Equally, each
encrypted tablespace has its own tablespace key. Table keys are stored in the
data dictionary of the database, while tablespace keys are stored in the header
of the tablespace and additionally, the header of each underlying OS file that
makes up the tablespace. Each of these keys is encrypted with the TDE master
encryption key, which is stored outside of the database in an external security
module: either the Oracle Wallet (a PKCS#12 formatted file that is encrypted
using a passphrase supplied either by the designated security administrator or
DBA during setup), or a Hardware Security Module (HSM) device for higher
assurance […]”
{quote}
Further design details forthcoming in a design document and patch as soon as we
have all of the clearances in place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7441) Make ClusterManager in IntegrationTestingUtility pluggable
[ https://issues.apache.org/jira/browse/HBASE-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551497#comment-13551497 ] Hudson commented on HBASE-7441: --- Integrated in HBase-0.94 #724 (See [https://builds.apache.org/job/HBase-0.94/724/]) HBASE-7441 Make ClusterManager in IntegrationTestingUtility pluggable (Revision 1432251) Result = ABORTED stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/IntegrationTestingUtility.java Make ClusterManager in IntegrationTestingUtility pluggable -- Key: HBASE-7441 URL: https://issues.apache.org/jira/browse/HBASE-7441 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Labels: newbie, patch Fix For: 0.96.0, 0.94.5 Attachments: HBASE-7441-0.94-v1.patch, HBASE-7441-trunk-v1.patch, HBASE-7441-trunk-v2.patch Original Estimate: 3h Remaining Estimate: 3h After the patch HBASE-7009, we can use ChaosMonkey to test the Hbase cluster. The ClusterManager use ssh to stop/start the rs or master without passwd. To support other cluster manager tool, we need to make clusterManager in IntegrationTestingUtility pluggable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7544) Transparent HFile encryption
[
https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-7544:
--
Description:
Introduce transparent encryption of HBase on disk data.
Depends on a separate contribution of an encryption codec framework to Hadoop
core and an AES-NI (native code) codec.
I have an experimental patch that introduces encryption at the HFile level,
with all necessary changes propagated up to the HStore level. For the most
part, the changes are straightforward and mechanical. After HBASE-7414, we can
introduce specification of an optional encryption codec in the file trailer.
The work is not ready to go yet because key management and the HBCK pieces are
TBD.
Requirements:
- Transparent encryption at the CF or table level
- Protect against all data leakage from files at rest
- Two-tier key architecture for consistency with best practices for this
feature in the RDBMS world
- Built-in key management
- Flexible and non-intrusive key rotation
- Mechanisms not exposed to or modifiable by users
- Hardware security module integration (via Java KeyStore)
- HBCK support for transparently encrypted files (+ plugin architecture for
HBCK)
Additional goals:
- Shell support for administrative functions
- Avoid performance impact for the null crypto codec case
- Play nicely with other changes underway: in HFile, block coding, etc.
We're aiming for rough parity with Oracle's transparent tablespace encryption
feature, described in
http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
{quote}
“Transparent Data Encryption uses a 2-tier key architecture for flexible and
non-intrusive key rotation and least operational and performance impact: Each
application table with at least one encrypted column has its own table key,
which is applied to all encrypted columns in that table. Equally, each
encrypted tablespace has its own tablespace key. Table keys are stored in the
data dictionary of the database, while tablespace keys are stored in the header
of the tablespace and additionally, the header of each underlying OS file that
makes up the tablespace. Each of these keys is encrypted with the TDE master
encryption key, which is stored outside of the database in an external security
module: either the Oracle Wallet (a PKCS#12 formatted file that is encrypted
using a passphrase supplied either by the designated security administrator or
DBA during setup), or a Hardware Security Module (HSM) device for higher
assurance […]”
{quote}
Further design details forthcoming in a design document and patch as soon as we
have all of the clearances in place.
was:
Introduce transparent encryption of HBase on disk data.
Depends on a separate contribution of an encryption codec framework to Hadoop
core and an AES-NI (native code) codec.
I have an experimental patch that introduces encryption at the HFile level,
with all necessary changes propagated up to the HStore level. For the most
part, the changes are straightforward and mechanical. After HBASE-7414, we can
introduce specification of an optional encryption codec in the file trailer.
The work is not ready to go yet because key management and the HBCK pieces are
TBD.
Requirements:
- Mechanisms not exposed to or modifiable by users
- Transparent encryption at the CF or table level
- Built-in key management
- Flexible and non-intrusive key rotation
- Two-tier key architecture for consistency with best practices for this
feature in the RDBMS world
- Transparent encryption of sensitive application columns
- Protect against all data leakage from files at rest
- Hardware security module integration (via Java KeyStore)
- HBCK support for transparently encrypted files (+ plugin architecture for
HBCK)
We're aiming for rough parity with Oracle's transparent tablespace encryption
feature, described in
http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
{quote}
“Transparent Data Encryption uses a 2-tier key architecture for flexible and
non-intrusive key rotation and least operational and performance impact: Each
application table with at least one encrypted column has its own table key,
which is applied to all encrypted columns in that table. Equally, each
encrypted tablespace has its own tablespace key. Table keys are stored in the
data dictionary of the database, while tablespace keys are stored in the header
of the tablespace and additionally, the header of each underlying OS file that
makes up the tablespace. Each of these keys is encrypted with the TDE master
encryption key, which is stored outside of the database in an external security
module: either the Oracle Wallet (a PKCS#12 formatted file that is encrypted
using a passphrase supplied either by the designated security administrator or
DBA during setup), or a Hardware Security Module (HSM)
[jira] [Updated] (HBASE-6824) Introduce ${hbase.local.dir} and save coprocessor jars there
[
https://issues.apache.org/jira/browse/HBASE-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-6824:
-
Fix Version/s: 0.94.5
Introduce ${hbase.local.dir} and save coprocessor jars there
Key: HBASE-6824
URL: https://issues.apache.org/jira/browse/HBASE-6824
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.96.0, 0.94.5
Attachments: hbase-6824_v1-0.94.patch, hbase-6824_v1-trunk.patch,
hbase-6824_v2-0.94.patch, hbase-6824_v2-trunk.patch, hbase-6824_v3.patch
We need to make the temp directory where coprocessor jars are saved
configurable. For this we will add hbase.local.dir configuration parameter.
Windows tests are failing due to the pathing problems for coprocessor jars:
Two HBase TestClassLoading unit tests failed due to a failiure in loading the
test file from HDFS:
{code}
testClassLoadingFromHDFS(org.apache.hadoop.hbase.coprocessor.TestClassLoading):
Class TestCP1 was missing on a region
testClassLoadingFromLibDirInJar(org.apache.hadoop.hbase.coprocessor.TestClassLoading):
Class TestCP1 was missing on a region
{code}
The problem is that CoprocessorHost.load() copies the jar file locally, and
schedules the local file to be deleted on exit, but calling
FileSystem.deleteOnExit(). However, the filesystem is not the file system of
the local file, it is the distributed file system, so on windows, the Path
fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6824) Introduce ${hbase.local.dir} and save coprocessor jars there
[
https://issues.apache.org/jira/browse/HBASE-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-6824:
-
Attachment: hbase-6824_v3-0.94.patch
Committed to 0.94 as well. Attaching what is committed.
Introduce ${hbase.local.dir} and save coprocessor jars there
Key: HBASE-6824
URL: https://issues.apache.org/jira/browse/HBASE-6824
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.96.0, 0.94.5
Attachments: hbase-6824_v1-0.94.patch, hbase-6824_v1-trunk.patch,
hbase-6824_v2-0.94.patch, hbase-6824_v2-trunk.patch,
hbase-6824_v3-0.94.patch, hbase-6824_v3.patch
We need to make the temp directory where coprocessor jars are saved
configurable. For this we will add hbase.local.dir configuration parameter.
Windows tests are failing due to the pathing problems for coprocessor jars:
Two HBase TestClassLoading unit tests failed due to a failiure in loading the
test file from HDFS:
{code}
testClassLoadingFromHDFS(org.apache.hadoop.hbase.coprocessor.TestClassLoading):
Class TestCP1 was missing on a region
testClassLoadingFromLibDirInJar(org.apache.hadoop.hbase.coprocessor.TestClassLoading):
Class TestCP1 was missing on a region
{code}
The problem is that CoprocessorHost.load() copies the jar file locally, and
schedules the local file to be deleted on exit, but calling
FileSystem.deleteOnExit(). However, the filesystem is not the file system of
the local file, it is the distributed file system, so on windows, the Path
fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7540) Make znode dump to print a dump of replication znodes
[ https://issues.apache.org/jira/browse/HBASE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7540: --- Summary: Make znode dump to print a dump of replication znodes (was: Make znode dump to print a dump of replciation znodes) Make znode dump to print a dump of replication znodes - Key: HBASE-7540 URL: https://issues.apache.org/jira/browse/HBASE-7540 Project: HBase Issue Type: Improvement Components: Replication, UI Affects Versions: 0.94.3 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.4 Attachments: HBASE-7540-trunk.patch, HBASE-7540-v1.patch, Screen Shot 2013-01-11 at 10.23.50 AM.png It will be nice to have a dump of replication related znodes on the master UI (along with other znode dump). It helps while using replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7544) Transparent HFile encryption
[
https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551510#comment-13551510
]
Andrew Purtell commented on HBASE-7544:
---
The design also covers encrypting WALedits for sensitive CFs but I'm debating
if that should be a separate JIRA. More shortly.
Transparent HFile encryption
Key: HBASE-7544
URL: https://issues.apache.org/jira/browse/HBASE-7544
Project: HBase
Issue Type: New Feature
Components: HFile, io
Affects Versions: 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Introduce transparent encryption of HBase on disk data.
Depends on a separate contribution of an encryption codec framework to Hadoop
core and an AES-NI (native code) codec.
I have an experimental patch that introduces encryption at the HFile level,
with all necessary changes propagated up to the HStore level. For the most
part, the changes are straightforward and mechanical. After HBASE-7414, we
can introduce specification of an optional encryption codec in the file
trailer. The work is not ready to go yet because key management and the HBCK
pieces are TBD.
Requirements:
- Transparent encryption at the CF or table level
- Protect against all data leakage from files at rest
- Two-tier key architecture for consistency with best practices for this
feature in the RDBMS world
- Built-in key management
- Flexible and non-intrusive key rotation
- Mechanisms not exposed to or modifiable by users
- Hardware security module integration (via Java KeyStore)
- HBCK support for transparently encrypted files (+ plugin architecture for
HBCK)
Additional goals:
- Shell support for administrative functions
- Avoid performance impact for the null crypto codec case
- Play nicely with other changes underway: in HFile, block coding, etc.
We're aiming for rough parity with Oracle's transparent tablespace encryption
feature, described in
http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
{quote}
“Transparent Data Encryption uses a 2-tier key architecture for flexible and
non-intrusive key rotation and least operational and performance impact: Each
application table with at least one encrypted column has its own table key,
which is applied to all encrypted columns in that table. Equally, each
encrypted tablespace has its own tablespace key. Table keys are stored in the
data dictionary of the database, while tablespace keys are stored in the
header of the tablespace and additionally, the header of each underlying OS
file that makes up the tablespace. Each of these keys is encrypted with the
TDE master encryption key, which is stored outside of the database in an
external security module: either the Oracle Wallet (a PKCS#12 formatted file
that is encrypted using a passphrase supplied either by the designated
security administrator or DBA during setup), or a Hardware Security Module
(HSM) device for higher assurance […]”
{quote}
Further design details forthcoming in a design document and patch as soon as
we have all of the clearances in place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7268) correct local region location cache information can be overwritten w/stale information from an old server
[ https://issues.apache.org/jira/browse/HBASE-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551516#comment-13551516 ] Ted Yu commented on HBASE-7268: --- I ran TestFromClientSide and TestCatalogTracker locally based on patch v6 - they passed. +1 on v6. correct local region location cache information can be overwritten w/stale information from an old server - Key: HBASE-7268 URL: https://issues.apache.org/jira/browse/HBASE-7268 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7268-v0.patch, HBASE-7268-v0.patch, HBASE-7268-v1.patch, HBASE-7268-v2.patch, HBASE-7268-v2-plus-masterTs.patch, HBASE-7268-v2-plus-masterTs.patch, HBASE-7268-v3.patch, HBASE-7268-v4.patch, HBASE-7268-v5.patch, HBASE-7268-v6.patch Discovered via HBASE-7250; related to HBASE-5877. Test is writing from multiple threads. Server A has region R; client knows that. R gets moved from A to server B. B gets killed. R gets moved by master to server C. ~15 seconds later, client tries to write to it (on A?). Multiple client threads report from RegionMoved exception processing logic R moved from C to B, even though such transition never happened (neither in nor before the sequence described below). Not quite sure how the client learned of the transition to C, I assume it's from meta from some other thread... Then, put fails (it may fail due to accumulated errors that are not logged, which I am investigating... but the bogus cache update is there nonwithstanding). I have a patch but not sure if it works, test still fails locally for yet unknown reason. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7268) correct local region location cache information can be overwritten w/stale information from an old server
[ https://issues.apache.org/jira/browse/HBASE-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7268: -- Attachment: 7268-v6.patch re-attaching patch v6 correct local region location cache information can be overwritten w/stale information from an old server - Key: HBASE-7268 URL: https://issues.apache.org/jira/browse/HBASE-7268 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.96.0 Attachments: 7268-v6.patch, HBASE-7268-v0.patch, HBASE-7268-v0.patch, HBASE-7268-v1.patch, HBASE-7268-v2.patch, HBASE-7268-v2-plus-masterTs.patch, HBASE-7268-v2-plus-masterTs.patch, HBASE-7268-v3.patch, HBASE-7268-v4.patch, HBASE-7268-v5.patch, HBASE-7268-v6.patch Discovered via HBASE-7250; related to HBASE-5877. Test is writing from multiple threads. Server A has region R; client knows that. R gets moved from A to server B. B gets killed. R gets moved by master to server C. ~15 seconds later, client tries to write to it (on A?). Multiple client threads report from RegionMoved exception processing logic R moved from C to B, even though such transition never happened (neither in nor before the sequence described below). Not quite sure how the client learned of the transition to C, I assume it's from meta from some other thread... Then, put fails (it may fail due to accumulated errors that are not logged, which I am investigating... but the bogus cache update is there nonwithstanding). I have a patch but not sure if it works, test still fails locally for yet unknown reason. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
[ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551519#comment-13551519 ] Enis Soztutar commented on HBASE-5487: -- [~ndimiduk] I guess you won't be working on this for some time, but wanna chime in the latest status? Thinking about the use case, what we want is to ensure that client operations does outlive master failover. Which is why in accumulo/fate, the state is kept in zk, and the master just provides execution. I think we can achieve the same thing if we add a WAL for master. Again, we have to break up the operation (like create table) into adempotent pieces, and sync the WAL before executing them. On master failover we just have to replay the WAL. Not sure which one would be simpler though. Generic framework for Master-coordinated tasks -- Key: HBASE-5487 URL: https://issues.apache.org/jira/browse/HBASE-5487 Project: HBase Issue Type: New Feature Components: master, regionserver, Zookeeper Affects Versions: 0.94.0 Reporter: Mubarak Seyed Assignee: Nick Dimiduk Need a framework to execute master-coordinated tasks in a fault-tolerant manner. Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s) based on start/end key) can make use of this framework. The advantages of framework are 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated tasks 2. Ability to abstract the common functions across Master - ZK and RS - ZK 3. Easy to plugin new master-coordinated tasks without adding code to core components -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
[ https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6775: -- Description: This issue introduces the ability for the HMaster to make use of ZooKeeper's multi-update functionality. This allows certain ZooKeeper operations to complete more quickly and prevents some issues with rare ZooKeeper failure scenarios (see the release note of HBASE-6710 for an example). This feature is off by default; to enable set hbase.zookeeper.useMulti to true in the configuration of the HMaster. IMPORTANT: hbase.zookeeper.useMulti should only be set to true if all ZooKeeper servers in the cluster are on version 3.4+ and will not be downgraded. ZooKeeper versions before 3.4 do not support multi-update and will not fail gracefully if multi-update is invoked (see ZOOKEEPER-1495). was: HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in different formats. If a ZK failure occurs between the writing of the two znodes, strange behavior can result. This issue is a reminder to change these two ZK writes to use ZK.multi when we require ZK 3.4+. Release Note: hbase.zookeeper.useMulti Instructs HBase to make use of ZooKeeper's multi-update functionality. +This allows certain ZooKeeper operations to complete more quickly and prevents some issues +with rare ZooKeeper failure scenarios (see the release note of HBASE-6710 for an example). +IMPORTANT: only set this to true if all ZooKeeper servers in the cluster are on version 3.4+ +and will not be downgraded. ZooKeeper versions before 3.4 do not support multi-update and will +not fail gracefully if multi-update is invoked (see ZOOKEEPER-1495). Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix -- Key: HBASE-6775 URL: https://issues.apache.org/jira/browse/HBASE-6775 Project: HBase Issue Type: Improvement Components: Zookeeper Affects Versions: 0.94.2 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.94.4 Attachments: HBASE-6775-v2.patch This issue introduces the ability for the HMaster to make use of ZooKeeper's multi-update functionality. This allows certain ZooKeeper operations to complete more quickly and prevents some issues with rare ZooKeeper failure scenarios (see the release note of HBASE-6710 for an example). This feature is off by default; to enable set hbase.zookeeper.useMulti to true in the configuration of the HMaster. IMPORTANT: hbase.zookeeper.useMulti should only be set to true if all ZooKeeper servers in the cluster are on version 3.4+ and will not be downgraded. ZooKeeper versions before 3.4 do not support multi-update and will not fail gracefully if multi-update is invoked (see ZOOKEEPER-1495). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7493) Refactor snapshotting tests so that all they all are exercised the same way.
[ https://issues.apache.org/jira/browse/HBASE-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551534#comment-13551534 ] Aleksandr Shulman commented on HBASE-7493: -- This seems like a relatively straightforward task. For tests that are written to test offline snapshots: Basically we'd factor out existing tests and add a boolean parameter indicating whether it's online or offline (yes for online). Then, in our test, we would wrap all relevant calls to enable and disable table, with the boolean. If it's online, then we don't enable/disable the table. By relevant calls, it would be the last call to disable at table before a snapshot and the first call to enable the table after the snapshot. For tests that are written to test online snapshots: We would add a call before and after the snapshot request to disable and then enable the table, respectively. We'd wrap these calls with the same logic as above and refactor the tests to accept a parameter. The caveat would be tests that are actually testing table enable/disable during snapshots. We have to be careful with those tests. Refactor snapshotting tests so that all they all are exercised the same way. Key: HBASE-7493 URL: https://issues.apache.org/jira/browse/HBASE-7493 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Currently we have two flavors of snapshots -- offline and flush snapshots. We may have a few more with different consistency policies and we should test all of them with the same basic correctness tests (see TestFlushSnapshotFromClient). We should refactor or use parameterized tests to make this easy. I prefer refactor, easier to follow and to test only what you want individually. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551540#comment-13551540 ] Andrew Purtell commented on HBASE-7411: --- I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? Won't we then be forking Curator with a custom patch and hostnig a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used. Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7545) [replication] Break out TestReplication into manageable classes
Jean-Daniel Cryans created HBASE-7545:
-
Summary: [replication] Break out TestReplication into manageable
classes
Key: HBASE-7545
URL: https://issues.apache.org/jira/browse/HBASE-7545
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.96.0
This has been discussed before but after trying to debug the last failure on
Jenkins where I saw the time go back and forth (if you don't care about your
own sanity do checkout
https://builds.apache.org/job/HBase-TRUNK/3726/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationWithCompression/testDeleteTypes/)
I think it is time to break out TestReplication.
The difficulty is that the setup for the 2 clusters is a lot of code I don't
want to duplicate. I'm thinking that we can keep {{setUpBeforeClass}} there and
have the other classes extend TestReplication (which should also change name).
I'm thinking of the following new classes:
- TestReplicationSmallTests, contains the easy methods that don't mess around
too much.
- TestReplicationQueueFailover, contains one test of the same name
- TestReplicationDisableInactivePeer, contains one test of the same name
- Rename TestReplicationWithCompression
TestReplicationQueueFailoverWithCompression and make it extends
TestReplicationQueueFailover.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551540#comment-13551540 ] Andrew Purtell edited comment on HBASE-7411 at 1/11/13 9:37 PM: I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? Won't we then be forking Curator with a custom patch and hosting a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used (and likely to have community bandwidth). Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. was (Author: apurtell): I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? Won't we then be forking Curator with a custom patch and hostnig a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used. Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551540#comment-13551540 ] Andrew Purtell edited comment on HBASE-7411 at 1/11/13 9:40 PM: I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? In the interim, won't we then be forking Curator with a custom patch and hosting a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used (and likely to have community bandwidth). Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. Edit: Make argument more intelligible, please pardon. was (Author: apurtell): I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? Won't we then be forking Curator with a custom patch and hosting a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used (and likely to have community bandwidth). Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7546) Obtain a table read lock on region split operations
Enis Soztutar created HBASE-7546: Summary: Obtain a table read lock on region split operations Key: HBASE-7546 URL: https://issues.apache.org/jira/browse/HBASE-7546 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar As discussed in the parent issue HBASE-7305, we should be coordinating between splits and table operations to ensure that they don't happen at the same time. In this issue we will acquire shared read locks for region splits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7527) integration tests STILL won't run from tar.gz in trunk
[ https://issues.apache.org/jira/browse/HBASE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7527: Attachment: HBASE-7527-v0.patch Integration tests rely on lots of code from common test so it looks like deploying common test jar is the only viable solution unless another project is created. integration tests STILL won't run from tar.gz in trunk -- Key: HBASE-7527 URL: https://issues.apache.org/jira/browse/HBASE-7527 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7527-v0.patch The problem is that the class used to find test classes sits in common test jar, which is not included in the package. However, if we move the class to the common jar itself, we'll have a JUnit dependency. And if we cannot just move it to integration tests, because a test that verifies test categories makes use of it too. This is all very sad. I will see if there's any way to not have junit dependency (we already seem to deploy junit so it might not be such a big deal). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
