date:20130805

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729128#comment-13729128
]

Hadoop QA commented on HBASE-6580:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595855/6580-trunk-v4.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:

org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//console

This message is automatically generated.

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Key: HBASE-6580
URL: https://issues.apache.org/jira/browse/HBASE-6580
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.6, 0.95.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.98.0, 0.95.2, 0.94.11

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

Update:
I now propose deprecating HTablePool and instead introduce a getTable method
on HConnection and allow HConnection to manage the ThreadPool.
Initial proposal:
Here I propose a very simple TablePool.
It could be called LightHTablePool (or something - if you have a better name).
Internally it would maintain an HConnection and an Executor service and each
invocation of getTable(...) would create a new HTable and close() would just
close it.
In testing I find this more light weight than HTablePool and easier to
monitor in terms of resources used.
It would hardly be more than a few dozen lines of code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.

2013-08-05 Thread gautam (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729300#comment-13729300
 ] 

gautam commented on HBASE-9108:
---

The sanity tests have gone fine here. Can we expect this to be checked in into 
94.x  mainline soon?

 LoadTestTool need to have a way to ignore keys which were failed during 
 write. 
 ---

 Key: HBASE-9108
 URL: https://issues.apache.org/jira/browse/HBASE-9108
 Project: HBase
  Issue Type: Improvement
  Components: test
Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10
Reporter: gautam
Assignee: gautam
Priority: Critical
 Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._trunk

   Original Estimate: 48h
  Remaining Estimate: 48h

 While running the chaosmonkey integration tests, it is found that write 
 sometimes fails when the cluster components are restarted/stopped/killed etc..
 The data key which was being put, using the LoadTestTool, is added to the 
 failed key set, and at the end of the test, this failed key set is checked 
 for any entries to assert failures.
 While doing fail-over testing, it is expected that some of the keys may go 
 un-written. The point here is to validate that whatever gets into hbase for 
 an unstable cluster really goes in, and hence read should be 100% for 
 whatever keys went in successfully.
 Currently LoadTestTool has strict checks to validate every key being written 
 or not. In case any keys is not written, it fails.
 I wanted to loosen this constraint by allowing users to pass in a set of 
 exceptions they expect when doing put/write operations over hbase. If one of 
 these expected exception set is thrown while writing key to hbase, the failed 
 key would be ignored, and hence wont even be considered again for subsequent 
 write as well as read.
 This can be passed to the load test tool as csv list parameter 
 -allowed_write_exceptions, or it can be passed through hbase-site.xml by 
 writing a value for test.ignore.exceptions.during.write
 Here is the usage:
 -allowed_write_exceptions 
 java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException
 Hence, by doing this the existing integration tests can also make use of this 
 change by passing it as property in hbase-site.xml, as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables


[ 
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729505#comment-13729505
 ] 

Matteo Bertozzi commented on HBASE-8778:


+1, v5 looks good to me

 Region assigments scan table directory making them slow for huge tables
 ---

 Key: HBASE-8778
 URL: https://issues.apache.org/jira/browse/HBASE-8778
 Project: HBase
  Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Priority: Critical
 Fix For: 0.98.0, 0.95.2

 Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch, 
 HBASE-8778-0.94.5-v2.patch, HBASE-8778.patch, HBASE-8778-v2.patch, 
 HBASE-8778-v3.patch, HBASE-8778-v4.patch, HBASE-8778-v5.patch


 On a table with 130k regions it takes about 3 seconds for a region server to 
 open a region once it has been assigned.
 Watching the threads for a region server running 0.94.5 that is opening many 
 such regions shows the thread opening the reigon in code like this:
 {noformat}
 PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000 
 nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
 at java.lang.String.indexOf(String.java:1521)
 at java.net.URI$Parser.scan(URI.java:2912)
 at java.net.URI$Parser.parse(URI.java:3004)
 at java.net.URI.init(URI.java:736)
 at org.apache.hadoop.fs.Path.initialize(Path.java:145)
 at org.apache.hadoop.fs.Path.init(Path.java:126)
 at org.apache.hadoop.fs.Path.init(Path.java:50)
 at 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
 at 
 org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
 at 
 org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
 at 
 org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
 at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
 at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
 at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
 at 
 org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
 at 
 org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
 at 
 org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
 at 
 org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
 at 
 org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
 {noformat}
 To open the region, the region server first loads the latest 
 HTableDescriptor.  Since HBASE-4553 HTableDescriptor's are stored in the file 
 system at /hbase/tableDir/.tableinfo.sequenceNum.  The file with the 
 largest sequenceNum is the current descriptor.  This is done so that the 
 current descirptor is updated atomically.  However, since the filename is not 
 known in advance FSTableDescriptors it has to do a FileSystem.listStatus 
 operation which has to list all files in the directory to find it.  The 
 directory also contains all the region directories, so in our case it has to 
 load 130k FileStatus objects.  Even using a globStatus matching function 
 still transfers all the objects to the client before performing the pattern 
 matching.  Furthermore HDFS uses a default of transferring 1000 directory 
 entries in each RPC call, so it requires 130 roundtrips to the namenode to 
 fetch all the directory entries.
 Consequently, to reassign all the regions of a table (or a constant fraction 
 thereof) requires time proportional to the square of the number of regions.
 In our case, if a region server fails with 200 such regions, it takes 10+ 
 minutes for them all to be reassigned, after the zk expiration and log 
 splitting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

[jira] [Commented] (HBASE-8409) Security support for namespaces


[ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729552#comment-13729552
 ] 

Matteo Bertozzi commented on HBASE-8409:


Do you plan to add a migration step to the NamespaceUpgrade tool? migrating the 
_acl_ table to hbase._acl_?
How do you handle the row keys migration? just a simple scan, put new values 
and delete the old ones?

 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8760) possible loss of data in snapshot taken after region split


[ 
https://issues.apache.org/jira/browse/HBASE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729572#comment-13729572
 ] 

Ted Yu commented on HBASE-8760:
---

{code}
+  LOG.warn(msg);
+  //throw new CorruptedSnapshotException(msg, snapshot);
{code}
Remove the commented out code ?

 possible loss of data in snapshot taken after region split
 --

 Key: HBASE-8760
 URL: https://issues.apache.org/jira/browse/HBASE-8760
 Project: HBase
  Issue Type: Bug
  Components: snapshots
Affects Versions: 0.94.8, 0.95.1
Reporter: Jerry He
 Fix For: 0.98.0, 0.95.2, 0.94.12

 Attachments: HBase-8760-0.94.8.patch, HBase-8760-0.94.8-v1.patch, 
 HBASE-8760-thz-v0.patch, HBASE-8760-thz-v1.patch


 Right after a region split but before the daughter regions are compacted, we 
 have two daughter regions containing Reference files to the parent hfiles.
 If we take snapshot right at the moment, the snapshot will succeed, but it 
 will only contain the daughter Reference files. Since there is no hold on the 
 parent hfiles, they will be deleted by the HFile Cleaner after they are no 
 longer needed by the daughter regions soon after.
 A minimum we need to do is the keep these parent hfiles from being deleted. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.


[ 
https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729646#comment-13729646
 ] 

Ted Yu commented on HBASE-9108:
---

{code}
+LOG.error(Class: [ + className + ] in not an instance of 
Exception.);
{code}
typo: 'in not'
{code}
+  public boolean ignoredKey(long k) {
{code}
The above can be package private.
{code}
+  protected static final String OPT_ALLOWED_WRITE_EXCEPTIONS = 
allowed_write_exceptions;
{code}
Can we use a shorter string for option ?

 LoadTestTool need to have a way to ignore keys which were failed during 
 write. 
 ---

 Key: HBASE-9108
 URL: https://issues.apache.org/jira/browse/HBASE-9108
 Project: HBase
  Issue Type: Improvement
  Components: test
Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10
Reporter: gautam
Assignee: gautam
Priority: Critical
 Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._trunk

   Original Estimate: 48h
  Remaining Estimate: 48h

 While running the chaosmonkey integration tests, it is found that write 
 sometimes fails when the cluster components are restarted/stopped/killed etc..
 The data key which was being put, using the LoadTestTool, is added to the 
 failed key set, and at the end of the test, this failed key set is checked 
 for any entries to assert failures.
 While doing fail-over testing, it is expected that some of the keys may go 
 un-written. The point here is to validate that whatever gets into hbase for 
 an unstable cluster really goes in, and hence read should be 100% for 
 whatever keys went in successfully.
 Currently LoadTestTool has strict checks to validate every key being written 
 or not. In case any keys is not written, it fails.
 I wanted to loosen this constraint by allowing users to pass in a set of 
 exceptions they expect when doing put/write operations over hbase. If one of 
 these expected exception set is thrown while writing key to hbase, the failed 
 key would be ignored, and hence wont even be considered again for subsequent 
 write as well as read.
 This can be passed to the load test tool as csv list parameter 
 -allowed_write_exceptions, or it can be passed through hbase-site.xml by 
 writing a value for test.ignore.exceptions.during.write
 Here is the usage:
 -allowed_write_exceptions 
 java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException
 Hence, by doing this the existing integration tests can also make use of this 
 change by passing it as property in hbase-site.xml, as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position


[ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729655#comment-13729655
 ] 

Nick Dimiduk commented on HBASE-9091:
-

[~stack] Since we're not running on flintstones computers, my preference is to 
use a single class that wraps both a range in a byte[] and a position marker, a 
la {{ByteBuffer}}.

[~mcorgan] Having ready through Guava's {{Precondition}} code, I'm not as 
concerned about the performance impact of the additional checks. I don't have a 
microbenchmark to quantify the lack of concern, but the code describes the 
pains taken to shape the condition checking statements for the benefit of the 
JIT, at least not in the face of giving helpful feedback in a user-facing API.

[~mcorgan] Thanks for outlining your two use-cases above, that's helpful. I 
think my intended use-case is an extension of #2, where position tracking is 
the added feature. I can't think of a case where a user would want to build a 
compound rowkey across multiple threads.

I'll assemble a new patch that leaves {{ByteRange}} intact, extending it with a 
subclass that adds the position tracking. I will omit the precondition checking 
for the time being, so long as we all understand that (1) omission opens the 
user up to easily detected bugs and (2) adding it later will impact a 
(admittedly brand new, {{Evolving}}) public API.

I will not update the other two patches until we've found something here that's 
agreeable. I'd rather ship 8201 and 8693 as they are than miss the window 
because of this bit of API sugar.

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9099) logReplay could trigger double region assignment


 [ 
https://issues.apache.org/jira/browse/HBASE-9099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Zhong updated HBASE-9099:
-

   Resolution: Fixed
Fix Version/s: 0.95.2
   0.98.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

 logReplay could trigger double region assignment
 

 Key: HBASE-9099
 URL: https://issues.apache.org/jira/browse/HBASE-9099
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.0, 0.95.2

 Attachments: hbase-9099.patch, hbase-9099-v1.patch


 The symptom is the first region assignment submitted in SSH is in progress 
 while when am.waitOnRegionToClearRegionsInTransition times out we will 
 re-submitted another SSH which will invoke another region assignment for the 
 region. It will cause the region get stuck in RIT status.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9099) logReplay could trigger double region assignment


[ 
https://issues.apache.org/jira/browse/HBASE-9099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729686#comment-13729686
 ] 

Jeffrey Zhong commented on HBASE-9099:
--

Integrated the v1 patch into 0.95 and trunk branch. Thanks!

 logReplay could trigger double region assignment
 

 Key: HBASE-9099
 URL: https://issues.apache.org/jira/browse/HBASE-9099
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Attachments: hbase-9099.patch, hbase-9099-v1.patch


 The symptom is the first region assignment submitted in SSH is in progress 
 while when am.waitOnRegionToClearRegionsInTransition times out we will 
 re-submitted another SSH which will invoke another region assignment for the 
 region. It will cause the region get stuck in RIT status.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729695#comment-13729695
]

Lars Hofhansl commented on HBASE-6580:
--

TestZKInterProcessReadWriteLock passes locally.

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729699#comment-13729699
 ] 

Jeffrey Zhong commented on HBASE-8960:
--

[~saint@gmail.com] A good suggestion. I'm thinking to extend the tool to 
include stats for skipped ones as well so we can cover the test suite timeout 
situations. The end result is printing stats of failed and skipped tests in 
last N runs instead of just failed ones. 

 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8322) Re-enable hbase checksums by default

2013-08-05 Thread Jean-Daniel Cryans (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729703#comment-13729703
 ] 

Jean-Daniel Cryans commented on HBASE-8322:
---

[~enis] I was wondering, why did you pick CRC32C? It's not available in Hadoop 
1 and it's what's failing. CRC32 is fine though.

 Re-enable hbase checksums by default
 

 Key: HBASE-8322
 URL: https://issues.apache.org/jira/browse/HBASE-8322
 Project: HBase
  Issue Type: Improvement
  Components: Filesystem Integration
Reporter: Enis Soztutar
Assignee: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.98.0, 0.95.2

 Attachments: hbase-8322_v1.patch, HBASE-8322-v2.patch


 Double checksumming issues in HBase level checksums(HBASE-5074) has been 
 fixed in HBASE-6868. However, that patch also disables hbase checksums by 
 default. I think we should re-enable it by default, and document the 
 interaction with shortcircuit reads. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7709) Infinite loop possible in Master/Master replication

2013-08-05 Thread Jean-Daniel Cryans (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729707#comment-13729707
]

Jean-Daniel Cryans commented on HBASE-7709:
---

Ah ok that's a fancy setup you got there. Sounds ok to me.

Infinite loop possible in Master/Master replication
---

Key: HBASE-7709
URL: https://issues.apache.org/jira/browse/HBASE-7709
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.94.6, 0.95.1
Reporter: Lars Hofhansl
Fix For: 0.98.0, 0.95.2, 0.94.12

We just discovered the following scenario:
# Cluster A and B are setup in master/master replication
# By accident we had Cluster C replicate to Cluster A.
Now all edit originating from C will be bouncing between A and B. Forever!
The reason is that when the edit come in from C the cluster ID is already set
and won't be reset.
We have a couple of options here:
# Optionally only support master/master (not cycles of more than two
clusters). In that case we can always reset the cluster ID in the
ReplicationSource. That means that now cycles 2 will have the data cycle
forever. This is the only option that requires no changes in the HLog format.
# Instead of a single cluster id per edit maintain a (unordered) set of
cluster id that have seen this edit. Then in ReplicationSource we drop any
edit that the sink has seen already. The is the cleanest approach, but it
might need a lot of data stored per edit if there are many clusters involved.
# Maintain a configurable counter of the maximum cycle side we want to
support. Could default to 10 (even maybe even just). Store a hop-count in the
WAL and the ReplicationSource increases that hop-count on each hop. If we're
over the max, just drop the edit.

[jira] [Commented] (HBASE-8939) Hanging unit tests


[ 
https://issues.apache.org/jira/browse/HBASE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729706#comment-13729706
 ] 

stack commented on HBASE-8939:
--

Failed again: 
http://54.241.6.143/job/HBase-0.95-Hadoop-2/759/org.apache.hbase$hbase-server/

 Hanging unit tests
 --

 Key: HBASE-8939
 URL: https://issues.apache.org/jira/browse/HBASE-8939
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
 Fix For: 0.95.2

 Attachments: 8939.txt


 We have hanging tests.  Here's a few from this morning's review:
 {code}
 durruti:0.95 stack$ ./dev-support/findHangingTest.sh  
 https://builds.apache.org/job/hbase-0.95-on-hadoop2/176/consoleText
   % Total% Received % Xferd  Average Speed   TimeTime Time  
 Current
  Dload  Upload   Total   SpentLeft  Speed
 100 3300k0 3300k0 0   508k  0 --:--:--  0:00:06 --:--:--  621k
 Hanging test: Running org.apache.hadoop.hbase.TestIOFencing
 Hanging test: Running org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
 {code}
 And...
 {code}
 durruti:0.95 stack$ ./dev-support/findHangingTest.sh 
 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/396/consoleText
   % Total% Received % Xferd  Average Speed   TimeTime Time  
 Current
  Dload  Upload   Total   SpentLeft  Speed
 100  779k0  779k0 0   538k  0 --:--:--  0:00:01 --:--:--  559k
 Hanging test: Running org.apache.hadoop.hbase.TestIOFencing
 Hanging test: Running 
 org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
 Hanging test: Running org.apache.hadoop.hbase.client.TestFromClientSide3
 {code}
 and
 {code}
 durruti:0.95 stack$ ./dev-support/findHangingTest.sh  
 http://54.241.6.143/job/HBase-0.95/607/consoleText
   % Total% Received % Xferd  Average Speed   TimeTime Time  
 Current
  Dload  Upload   Total   SpentLeft  Speed
 100  445k0  445k0 0   490k  0 --:--:-- --:--:-- --:--:--  522k
 Hanging test: Running 
 org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
 Hanging test: Running org.apache.hadoop.hbase.master.TestAssignmentManager
 Hanging test: Running org.apache.hadoop.hbase.util.TestHBaseFsck
 Hanging test: Running 
 org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary
 Hanging test: Running 
 org.apache.hadoop.hbase.IntegrationTestDataIngestSlowDeterministic
 {code}
 and...
 {code}
 durruti:0.95 stack$ ./dev-support/findHangingTest.sh  
 http://54.241.6.143/job/HBase-0.95-Hadoop-2/607/consoleText
   % Total% Received % Xferd  Average Speed   TimeTime Time  
 Current
  Dload  Upload   Total   SpentLeft  Speed
 100  781k0  781k0 0   240k  0 --:--:--  0:00:03 --:--:--  244k
 Hanging test: Running 
 org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
 Hanging test: Running org.apache.hadoop.hbase.client.TestFromClientSide
 Hanging test: Running org.apache.hadoop.hbase.TestIOFencing
 Hanging test: Running 
 org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
 Hanging test: Running 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


[ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729709#comment-13729709
 ] 

stack commented on HBASE-9072:
--

Failed here this morning: 
http://54.241.6.143/job/HBase-0.95/org.apache.hbase$hbase-server/753/testReport/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack

 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
 {code}
 [~mbertozzi] Opinion?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7980) TestZKInterProcessReadWriteLock fails occasionally in QA test run


[ 
https://issues.apache.org/jira/browse/HBASE-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729711#comment-13729711
 ] 

stack commented on HBASE-7980:
--

Failed here: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6604//testReport/  Anyone 
want to check these out?

 TestZKInterProcessReadWriteLock fails occasionally in QA test run
 -

 Key: HBASE-7980
 URL: https://issues.apache.org/jira/browse/HBASE-7980
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
 Fix For: 0.95.2


 {code}
 testReadLockExcludesWriters(org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock)
   Time elapsed: 0.003 sec   ERROR!
 java.lang.Exception: test timed out after 3000 milliseconds
   at sun.misc.Unsafe.park(Native Method)
 {code}
 You can find the test output here:
 https://builds.apache.org/job/PreCommit-HBASE-Build/4634/artifact/trunk/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock-output.txt

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729718#comment-13729718
]

stack commented on HBASE-6580:
--

HBASE-7980 TestZKInterProcessReadWriteLock is known flakey (I should just
disable it since no one is looking at it).

Patch lgtm on skim. Hurray HTablePool is deprecated!

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

[jira] [Updated] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HBASE-9095:
---

Attachment: 9095-2.txt

I had to do some more changes to have the tests work (TestMaster and
TestTestZKBasedOpenCloseRegion). I was trying to make the tests work with
Mockito/spy but it didn't work and at some point I gave up. This approach is
not as elegant but works.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

While debugging a case where a region was getting opened on a RegionServer
and then closed soon after (and then never re-opened anywhere thereafter), it
seemed like the processing in handleRegion to do with deletion of ZK nodes
should be non-asynchronous. This achieves two things:
1. The synchronous deletion prevents more than one processing on the same
event data twice. Assuming that we do get more than one notification (on
let's say, region OPENED event), the subsequent processing(s) in handleRegion
for the same znode would end up with a zookeeper node not found exception.
The return value of the data read would be null and that's already handled.
If it is asynchronous, it leads to issues like - master opens a region on a
certain RegionServer and soon after it sends that RegionServer a close for
the same region, and then the znode is deleted.
2. The deletion is currently handled in an executor service. This is
problematic since by design the events for a given region should be processed
in order. By delegating a part of the processing to executor service we are
somewhat violating this contract since there is no guarantee of the ordering
in the executor service executions...
Thanks to [~jeffreyz] and [~enis] for the discussions on this issue.

[jira] [Commented] (HBASE-9079) FilterList getNextKeyHint skips rows that should be included in the results

2013-08-05 Thread Viral Bajaria (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729738#comment-13729738
]

Viral Bajaria commented on HBASE-9079:
--

[~lhofhansl] Can you review the patch when you get a chance ? I have already
deployed this to my production cluster and have not had any issues.

FilterList getNextKeyHint skips rows that should be included in the results
---

Key: HBASE-9079
URL: https://issues.apache.org/jira/browse/HBASE-9079
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.94.10
Reporter: Viral Bajaria
Attachments: HBASE-9079-0.94.patch, HBASE-9079-trunk.patch

I hit a weird issue/bug and am able to reproduce the error consistently. The
problem arises when FilterList has two filters where each implements the
getNextKeyHint method.
The way the current implementation works is, StoreScanner will call
matcher.getNextKeyHint() whenever it gets a SEEK_NEXT_USING_HINT. This in
turn will call filter.getNextKeyHint() which at this stage is of type
FilterList. The implementation in FilterList iterates through all the filters
and keeps the max KeyValue that it sees. All is fine if you wrap filters in
FilterList in which only one of them implements getNextKeyHint. but if
multiple of them implement then that's where things get weird.
For example:
- create two filters: one is FuzzyRowFilter and second is ColumnRangeFilter.
Both of them implement getNextKeyHint
- wrap them in FilterList with MUST_PASS_ALL
- FuzzyRowFilter will seek to the correct first row and then pass it to
ColumnRangeFilter which will return the SEEK_NEXT_USING_HINT code.
- Now in FilterList when getNextKeyHint is called, it calls the one on
FuzzyRow first which basically says what the next row should be. While in
reality we want the ColumnRangeFilter to give the seek hint.
- The above behavior skips data that should be returned, which I have
verified by using a RowFilter with RegexStringComparator.
I updated the FilterList to maintain state on which filter returns the
SEEK_NEXT_USING_HINT and in getNextKeyHint, I invoke the method on the saved
filter and reset that state. I tested it with my current queries and it works
fine but I need to run the entire test suite to make sure I have not
introduced any regression. In addition to that I need to figure out what
should be the behavior when the opeation is MUST_PASS_ONE, but I doubt it
should be any different.
Is my understanding of it being a bug correct ? Or am I trivializing it and
ignoring something very important ? If it's tough to wrap your head around
the explanation, then I can open a JIRA and upload a patch against 0.94 head.

[jira] [Commented] (HBASE-8409) Security support for namespaces


[ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729736#comment-13729736
 ] 

stack commented on HBASE-8409:
--

Can the new acl table be called hbase.acl instead of hbase._acl_?

bq. The work related to migration of the existing acl table to the new 
namespace is remaining and will be completed in the follow up patch.

As per our Matteo, how is the above going?

bq.  Permissions can be granted to a namespace by the hbase admin, by appending 
'@' to the namespace name.

What is the '@' about?  Seems awkward?  Is this because our current grant 
didn't include ns:

{code}
grant user permissions [ table [ column family [ column qualifier ] ] 
]
{code}

... the '@' is how we get it in there?  Can we do better?

Thanks.








 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-8409) Security support for namespaces


 [ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8409:
-

Priority: Critical  (was: Major)

ns can't go in w/o this fixed (migration and '@' thingy)

 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
Priority: Critical
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729739#comment-13729739
]

Devaraj Das commented on HBASE-9095:

bq. Since these handlers no longer run in executors, are there executors we
start in the RS that are now unused?
Not sure on this. The executor service is used during the master recovery (look
at AM.processRegionsInTransition) and I am not sure I want to touch that part
in this patch...

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-7525) A canary monitoring program specifically for regionserver


[ 
https://issues.apache.org/jira/browse/HBASE-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729743#comment-13729743
 ] 

stack commented on HBASE-7525:
--

[~takeshi.miao] Pardon us for overlooking this addition.  Please rebase (I 
think the scan instead of get for the first key in table has been addressed) 
and lets get it in.

 A canary monitoring program specifically for regionserver
 -

 Key: HBASE-7525
 URL: https://issues.apache.org/jira/browse/HBASE-7525
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.94.0
Reporter: takeshi.miao
Priority: Minor
 Fix For: 0.95.0

 Attachments: HBASE-7525-0.95-v0.patch, HBASE-7525-0.95-v1.patch, 
 HBASE-7525-0.95-v3.patch, HBASE-7525-v0.patch, RegionServerCanary.java


 *Motivation*
 This ticket is to provide a canary monitoring tool specifically for 
 HRegionserver, details as follows
 1. This tool is required by operation team due to they thought that the 
 canary for each region of a HBase is too many for them, so I implemented this 
 coarse-granular one based on the original o.a.h.h.tool.Canary for them
 2. And this tool is implemented by multi-threading, which means the each Get 
 request sent by a thread. the reason I use this way is due to we suffered the 
 region server hung issue by now the root cause is still not clear. so this 
 tool can help operation team to detect hung region server if any.
 *example*
 1. the tool docs
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -help
 Usage: [opts] [regionServerName 1 [regionServrName 2...]]
  regionServerName - FQDN serverName, can use linux command:hostname -f to 
 check your serverName
  where [-opts] are:
-help Show this help and exit.
-eUse regionServerName as regular expression
   which means the regionServerName is regular expression pattern
-f B stop whole program if first error occurs, default is true
-t N timeout for a check, default is 60 (milisecs)
-daemonContinuous check at defined intervals.
-interval N  Interval between checks (sec)
 2. Will send a request to each regionserver in a HBase cluster
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary
 3. Will send a request to a regionserver by given name
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary rs1.domainname
 4. Will send a request to regionserver(s) by given regular-expression
 /opt/trend/circus-opstool/bin/hbase-canary-monitor-each-regionserver.sh -e 
 rs1.domainname.pattern
 // another example
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -e 
 tw-poc-tm-puppet-hdn[0-9]\{1,2\}.client.tw.trendnet.org
 5. Will send a request to a regionserver and also set a timeout limit for 
 this test
 // query regionserver:rs1.domainname with timeout limit 10sec
 // -f false, means that will not exit this program even test failed
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -f false -t 1 
 rs1.domainname
 // echo 1 if timeout
 echo $?
 6. Will run as daemon mode, which means it will send request to each 
 regionserver periodically
 ./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -daemon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

2013-08-05 Thread Jimmy Xiang (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729744#comment-13729744
]

Jimmy Xiang commented on HBASE-9095:

Currently, the executor service is still needed. We can get rid of it later on
when we remove the timeout monitor.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position


 [ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-9091:


Attachment: 0001-HBASE-9091-Introduce-PositionedByteRange.patch

Adds the new subclass described previously.

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


 [ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-9072:
---

Assignee: Matteo Bertozzi
  Status: Patch Available  (was: Open)

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Matteo Bertozzi
 Attachments: HBASE-9072-fix0.patch


 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
 {code}
 [~mbertozzi] Opinion?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


 [ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-9072:
---

Attachment: HBASE-9072-fix0.patch

This is a different error, from the one above.

The test looks flaky by design, it relies on a strange voodoo of the data 
inserted is always 4 files...

attached a -fix0 patch, that ensures the 4 files flush/compaction... but now it 
relies on the compaction to compact all the store files... so as soon we change 
the compaction algorithm this test is flaky again.

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
 Attachments: HBASE-9072-fix0.patch


 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
 {code}
 [~mbertozzi] Opinion?

--
This message is automatically

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729772#comment-13729772
 ] 

Jeffrey Zhong commented on HBASE-8960:
--

There is one more time out situation happened on 
http://54.241.6.143/job/HBase-0.95-Hadoop-2/759/. Let me split 
TestDistributedLogSplitting into two test suites one with replay on and the 
rest with replay off to see if we can make things better and possibly get some 
error messages.

 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


[ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729803#comment-13729803
 ] 

stack commented on HBASE-9072:
--

[~mbertozzi] Thanks boss for taking a look.  I'd rather just disable a crappy 
test.

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Matteo Bertozzi
 Attachments: HBASE-9072-fix0.patch


 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
 {code}
 [~mbertozzi] Opinion?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-9126) Make HFile MIN VERSION as 2

2013-08-05 Thread Anoop Sam John (JIRA)

Anoop Sam John created HBASE-9126:
-

 Summary: Make HFile MIN VERSION as 2
 Key: HBASE-9126
 URL: https://issues.apache.org/jira/browse/HBASE-9126
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.95.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 0.98.0, 0.95.2


Removed the HFile V1 support from version95. We can make the min version 
supported as 2? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729809#comment-13729809
 ] 

stack commented on HBASE-8960:
--

[~jeffreyz] Thanks boss.

 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9126) Make HFile MIN VERSION as 2

2013-08-05 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-9126:
--

Status: Patch Available  (was: Open)

 Make HFile MIN VERSION as 2
 ---

 Key: HBASE-9126
 URL: https://issues.apache.org/jira/browse/HBASE-9126
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.95.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9126.patch


 Removed the HFile V1 support from version95. We can make the min version 
 supported as 2? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9126) Make HFile MIN VERSION as 2

2013-08-05 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-9126:
--

Attachment: HBASE-9126.patch

Mostly code removal

 Make HFile MIN VERSION as 2
 ---

 Key: HBASE-9126
 URL: https://issues.apache.org/jira/browse/HBASE-9126
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.95.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9126.patch


 Removed the HFile V1 support from version95. We can make the min version 
 supported as 2? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-6580:
-

Attachment: 6580-trunk-v5.txt

Added getTable(String, ExecutorService) API. Added simple test to TestHCM to
exercise new APIs.

Should be getting close.

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, 6580-trunk-v5.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position

[
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729813#comment-13729813
]

stack commented on HBASE-9091:
--

[~ndimiduk] So, this is not 'api sugar', is it? Because it will show through
in the API? We need this for the other stuff to go in (unless we want
flintstone computing)?

Update ByteRange to maintain consumer's position

Key: HBASE-9091
URL: https://issues.apache.org/jira/browse/HBASE-9091
Project: HBase
Issue Type: Improvement
Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments: 0001-HBASE-9091-Extend-ByteRange.patch,
0001-HBASE-9091-Extend-ByteRange.patch,
0001-HBASE-9091-Introduce-PositionedByteRange.patch

ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is
mutable and an instance can be assigned over a byte[] after instantiation.
This is valuable as a performance consideration when working with byte[]
slices in a tight loop. Its current design is such that it is not possible to
consume a portion of the range while performing activities like decoding an
object without altering the definition of the range. It should provide a
position that is independent from the range's offset and length to make
partial reads easier.

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position


[ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729846#comment-13729846
 ] 

Nick Dimiduk commented on HBASE-9091:
-

Yes, it does show through on the API. I consider it sugar at this point 
because the same features are supported using {{byte[], int}} vs 
{{PositionedByteRange}}. The former will be a little slower because of the 
extra traversal and pushes that much more into user code, but the features are 
identical. That's why I suggest the APIs could be extended later to support 
{{PositionedByteRange}} if we can't get consensus today.

[~mcorgan], [~giacomotaylor], [~nkeywal], [~enis]: you've all expressed 
opinions about these API. Do these two classes look reasonable to you?

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729844#comment-13729844
]

Hadoop QA commented on HBASE-9095:
--

{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12596163/9095-2.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6605//console

This message is automatically generated.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)


[ 
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729849#comment-13729849
 ] 

Lars Hofhansl commented on HBASE-6580:
--

I might want to move this to 0.94.12, so that we can do proper review/testing 
on this (I would like to spin 0.94.11 today).

 Deprecate HTablePool in favor of HConnection.getTable(...)
 --

 Key: HBASE-6580
 URL: https://issues.apache.org/jira/browse/HBASE-6580
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.6, 0.95.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.98.0, 0.95.2, 0.94.11

 Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt, 
 6580-trunk-v4.txt, 6580-trunk-v5.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch


 Update:
 I now propose deprecating HTablePool and instead introduce a getTable method 
 on HConnection and allow HConnection to manage the ThreadPool.
 Initial proposal:
 Here I propose a very simple TablePool.
 It could be called LightHTablePool (or something - if you have a better name).
 Internally it would maintain an HConnection and an Executor service and each 
 invocation of getTable(...) would create a new HTable and close() would just 
 close it.
 In testing I find this more light weight than HTablePool and easier to 
 monitor in terms of resources used.
 It would hardly be more than a few dozen lines of code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-9127) HFileContext

2013-08-05 Thread Andrew Purtell (JIRA)

Andrew Purtell created HBASE-9127:
-

 Summary: HFileContext
 Key: HBASE-9127
 URL: https://issues.apache.org/jira/browse/HBASE-9127
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Purtell


We can roll up at least some of the state we are leaking between the Store 
layer and the HFile layer by introducing an IO context for passing between the 
two. This idea has come up in other discussions. Here I am calling it 
'HFileContext' because the particulars are regarding how to configure HFile 
readers and writers. This will be easier to maintain than various and sundry 
method parameters and (duplicated) instance variables sprinkled about, and will 
make adding or modifying persistence features easier and less disruptive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-6580:
-

Fix Version/s: (was: 0.94.11)
0.94.12

Moving to 0.94.12. Preparing for 0.94.11

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, 6580-trunk-v5.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like

2013-08-05 Thread ramkrishna.s.vasudevan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729855#comment-13729855
]

ramkrishna.s.vasudevan commented on HBASE-8496:
---

Pls note that we would not be persisting any tag related information on the
HFileBlock.
HFileBlock would not be modified in terms of what is persisted. In the recent
patch that we have worked up on we are just enclosing some of the state
variables into a POJO.
So specifically for tags there are no modifications on HfileBlock. So there is
no need for subclassing HFileBlock or add a new version to the hfileblock.
Based on the Encoding/Decoding context state the Encoder and decoding logic
of the algos would handle tags.
In the previous version of the patches the includeMemstoreTS and the
includeTags created changes to the interface apis. So as to avoid this we
enclosed these things inside the Context. Iin the recent patches the context
would have the POJO mentioned above).
So from the context we would be able to get the various decision points like to
include tags or not, includememstoreTS or not.
The encoding apis carry the context but the actual seekers that does the actual
decoding does not use contexts. So we have made changes such that the encoding
logic and the seekers use the context related objects.

Implement tags and the internals of how a tag should look like
--

Key: HBASE-8496
URL: https://issues.apache.org/jira/browse/HBASE-8496
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.0, 0.95.2
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Attachments: Comparison.pdf, HBASE-8496_2.patch, HBASE-8496.patch,
Tag design.pdf, Tag design_updated.pdf, Tag_In_KV_Buffer_For_reference.patch

The intent of this JIRA comes from HBASE-7897.
This would help us to decide on the structure and format of how the tags
should look like.

[jira] [Updated] (HBASE-9127) HFileContext

2013-08-05 Thread Andrew Purtell (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-9127:
--

Attachment: 0003-Move-data-block-encoder-into-HFileContext.patch
0002-Move-compression-algorithm-into-HFileContext.patch
0001-Introduce-HFileContext.patch

Attached are three patches as a thought experiment. They are based on 0.94 but
taking the same actions in later versions would be similar.

0001 - Introduces HFileContext and moves CacheConfig into it.

0002 - Moves compression algorithm selection into HFileContext.

0003 - Moves in cache and on disk data block encoder selection into
HFileContext. This one is not tested, so a few unit tests may need more tweaks.

Next steps could be:

- Move BloomType into HFileContext

- Move the MVCC read point into HFileContext

- Test if the additional indirection has a performance impact.

- Consider what more of HFileContext state can be made final/immutable to
facilitate inlining. This is interesting wherever a reader wants to reconfigure
based on file trailer or other metadata discovered when opening the HFile.

- CacheConfig could be fully rolled into HFileContext.

- In the HFile V3 work we have a patch that, like in the existing case with
memstoreTS, does whole file optimization if no tags will be stored (just like
if memstoreTS would be 0 for all KVs in an HFile), which breaks encapsulation
in the Store and Compactor. Putting this into HFileContext (at some future
time) would fix that.

- HBASE-7544 could be reworked to use this instead of sprinkling additional
method parameters throughout Store and HFile.

HFileContext

Key: HBASE-9127
URL: https://issues.apache.org/jira/browse/HBASE-9127
Project: HBase
Issue Type: Improvement
Reporter: Andrew Purtell
Attachments: 0001-Introduce-HFileContext.patch,
0002-Move-compression-algorithm-into-HFileContext.patch,
0003-Move-data-block-encoder-into-HFileContext.patch

We can roll up at least some of the state we are leaking between the Store
layer and the HFile layer by introducing an IO context for passing between
the two. This idea has come up in other discussions. Here I am calling it
'HFileContext' because the particulars are regarding how to configure HFile
readers and writers. This will be easier to maintain than various and sundry
method parameters and (duplicated) instance variables sprinkled about, and
will make adding or modifying persistence features easier and less disruptive.

[jira] [Commented] (HBASE-9118) Optimization in HFile V3 when no tags are present in a file

2013-08-05 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729857#comment-13729857
 ] 

ramkrishna.s.vasudevan commented on HBASE-9118:
---

We could use HBASE-9127(something similar) and refactor how mvcc works along 
with the tags optimization.

 Optimization in HFile V3 when no tags are present in a file
 ---

 Key: HBASE-9118
 URL: https://issues.apache.org/jira/browse/HBASE-9118
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.98.0, 0.95.2
Reporter: Anoop Sam John
Assignee: Anoop Sam John

 Now with V3 we will write tags length (short) along with every KV after value 
 bytes. When no tags at all present it will be wasting 2 bytes for every KV. 
 We can avoid this. During flush let the tags length be written. In fileInfo 
 we can add info like max tags length. During compaction if all the files 
 undergoing the compaction having a 0 max tags length, we can avoid writing 2 
 bytes (0) along with every KV..
 Note : Similar optimization available with mvcc

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

2013-08-05 Thread ramkrishna.s.vasudevan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729870#comment-13729870
]

ramkrishna.s.vasudevan commented on HBASE-9095:
---

Ok to do this in 0.94 also to avoid many of the concurrency issues? Need to
see the code to see how much it helps though?
What [~jxiang], [~jeffreyz] feel?

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-9099) logReplay could trigger double region assignment

2013-08-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729876#comment-13729876
 ] 

Hudson commented on HBASE-9099:
---

SUCCESS: Integrated in HBase-TRUNK #4343 (See 
[https://builds.apache.org/job/HBase-TRUNK/4343/])
HBASE-9099: logReplay could trigger double region assignment (jeffreyz: rev 
1510615)
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java


 logReplay could trigger double region assignment
 

 Key: HBASE-9099
 URL: https://issues.apache.org/jira/browse/HBASE-9099
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.0, 0.95.2

 Attachments: hbase-9099.patch, hbase-9099-v1.patch


 The symptom is the first region assignment submitted in SSH is in progress 
 while when am.waitOnRegionToClearRegionsInTransition times out we will 
 re-submitted another SSH which will invoke another region assignment for the 
 region. It will cause the region get stuck in RIT status.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

2013-08-05 Thread Jimmy Xiang (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729882#comment-13729882
]

Jimmy Xiang commented on HBASE-9095:

For 0.94, just one thread is used to process assignment events. Will this slow
down assignments?

For trunk, it is not a problem since we have up to 20 zk event workers by
default.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-9126) Make HFile MIN VERSION as 2


[ 
https://issues.apache.org/jira/browse/HBASE-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729887#comment-13729887
 ] 

stack commented on HBASE-9126:
--

+1

Makes sense.

 Make HFile MIN VERSION as 2
 ---

 Key: HBASE-9126
 URL: https://issues.apache.org/jira/browse/HBASE-9126
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.95.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9126.patch


 Removed the HFile V1 support from version95. We can make the min version 
 supported as 2? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


[ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729894#comment-13729894
 ] 

Hadoop QA commented on HBASE-9072:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12596174/HBASE-9072-fix0.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 6 new 
or modified tests.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6606//console

This message is automatically generated.

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Matteo Bertozzi
 Attachments: HBASE-9072-fix0.patch


 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position

[
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729898#comment-13729898
]

stack commented on HBASE-9091:
--

Fix:

{code}
+ * The state of a {@code ByteRange} is not synchronized and does to support
+ * concurrent access.
{code}

This seems superfluous: + public ByteRange(int capacity, int offset, int
length) {

Nice language ...it becomes a husk...

Is this sentence missing a tail? + * Designed to be a mutable alternative to

lgtm

Update ByteRange to maintain consumer's position

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729897#comment-13729897
]

Devaraj Das commented on HBASE-9095:

Although a couple of you gave +1s before, mind taking a look at the patch once
to see if it still is okay (main changes are to do with the unit test fixes).

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729902#comment-13729902
 ] 

stack commented on HBASE-8960:
--

Thanks [~jeffreyz]

Failed this morning: 
http://54.241.6.143/job/HBase-0.95-Hadoop-2/759/org.apache.hbase$hbase-server/


 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9055) HBaseAdmin#isTableEnabled() should return false for non-existent table


 [ 
https://issues.apache.org/jira/browse/HBASE-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-9055:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 HBaseAdmin#isTableEnabled() should return false for non-existent table 
 ---

 Key: HBASE-9055
 URL: https://issues.apache.org/jira/browse/HBASE-9055
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.1
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0

 Attachments: 9055-v1.txt, 9055-v2.txt, 9055-v3.txt


 Currently HBaseAdmin#isTableEnabled() returns true for a table which doesn't 
 exist.
 We should check table existence.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9064) test-patch.sh would silently fail if compilation against hadoop 1.0 fails


 [ 
https://issues.apache.org/jira/browse/HBASE-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-9064:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

 test-patch.sh would silently fail if compilation against hadoop 1.0 fails
 -

 Key: HBASE-9064
 URL: https://issues.apache.org/jira/browse/HBASE-9064
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
 Attachments: 9064-v1.txt


 {code}
   if [[ $? != 0 ]] ; then
 JIRA_COMMENT=$JIRA_COMMENT
 {color:red}-1 hadoop1.0{color}.  The patch failed to compile against the 
 hadoop 1.0 profile.
 cleanupAndExit 1
   fi
 {code}
 There is currently no post back to JIRA when there is compilation error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8409) Security support for namespaces

2013-08-05 Thread Francis Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729904#comment-13729904
 ] 

Francis Liu commented on HBASE-8409:


Yes, that's the plan. The table migration will be part of 8408 just to keep the 
patches more independent. Yep it'll be done in NamespaceUpgrade. I don't plan 
to scan since I know the table names. I was hoping I could get by just deleting 
the old keys and writing the new ones. Also will have to rewrite .regioninfo 
since the table name change. All of this has to be done will the system is 
offline. Does this sound feasible?

Vandana is on vacation this week. So I'll take care of the work in the interest 
of expediency.

 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
Priority: Critical
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9099) logReplay could trigger double region assignment

2013-08-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729906#comment-13729906
 ] 

Hudson commented on HBASE-9099:
---

SUCCESS: Integrated in hbase-0.95 #404 (See 
[https://builds.apache.org/job/hbase-0.95/404/])
HBASE-9099: logReplay could trigger double region assignment (jeffreyz: rev 
1510616)
* 
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java
* 
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java


 logReplay could trigger double region assignment
 

 Key: HBASE-9099
 URL: https://issues.apache.org/jira/browse/HBASE-9099
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.0, 0.95.2

 Attachments: hbase-9099.patch, hbase-9099-v1.patch


 The symptom is the first region assignment submitted in SSH is in progress 
 while when am.waitOnRegionToClearRegionsInTransition times out we will 
 re-submitted another SSH which will invoke another region assignment for the 
 region. It will cause the region get stuck in RIT status.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8409) Security support for namespaces

2013-08-05 Thread Francis Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729914#comment-13729914
 ] 

Francis Liu commented on HBASE-8409:


{quote}
Can the new acl table be called hbase.acl instead of hbase.acl?
{quote}
I assuming rename _acl_ = hbase:acl? That's the plan.

{quote}
What is the '@' about? Seems awkward? Is this because our current grant didn't 
include ns:
{quote}
It's to disambiguate granting per table and granting per namespace. The same 
paradigm (prefix with '@') is used to disambiguate user and group in the grant 
command. We could also go with 

grant user permissions namespace:

ie

grant 'foo', 'A', 'my_ns:'

To disambiguate NS from table. We couldn't do this before since we were using 
'.'. Does this sound ok?



 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
Priority: Critical
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8409) Security support for namespaces


[ 
https://issues.apache.org/jira/browse/HBASE-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729919#comment-13729919
 ] 

stack commented on HBASE-8409:
--

I forgot that group is done w/ a '@'.  So there is precedent.  Might be ok 
as-is then.

On rewriting .regioninfo, FYI HBASE-8778 moves stuff around and is about to go 
in.

 Security support for namespaces
 ---

 Key: HBASE-8409
 URL: https://issues.apache.org/jira/browse/HBASE-8409
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
Priority: Critical
 Attachments: HBASE-8049_trunk.patch


 This task adds the security piece to the namespace feature. The work related 
 to migration of the existing acl table to the new namespace is remaining and 
 will be completed in the follow up patch. Permissions can be granted to a 
 namespace by the hbase admin, by appending '@' to the namespace name. A user 
 with write or admin permissions on a given namespace can create tables in 
 that namespace. The other privileges (R, X, C ) do not have any special 
 meaning w.r.t namespaces.  Any users of hbase can list tables in a namespace.
  
 The following commands can only be executed by HBase admins.
 1. Grant privileges for user on Namespace.
 2. Revoke privileges for user on Namespace
 Grant Command:
 hbase grant 'tenant-A' 'W' '@N1'
  In the above example, the command will grant the user 'tenant-A' write 
 privileges for a namespace named N1.
 Revoke Command:
 hbase revoke 'tenant-A''@N1'
  In the above example, the command will revoke  all privileges from user 
 'tenant-A' for namespace named N1.
 Lets see an example on how privileges work with namespaces.
  
 User Mike request for a namespace named hbase_perf with the hbase admin.
   whoami: hbase
   hbase shell  namespace_create 'hbase_perf'
   hbase shell  grant 'mike', 'W', '@hbase_perf'
 Mike creates two tables table20 and table50 in the above workspace.
   whoami: mike
   hbase shell  create 'hbase_perf.table20', 'family1'
   hbase shell  create 'hbase_perf.table50', 'family1'
   Note: As Mike was able to create tables 'hbase_perf.table20', 
 'hbase_perf.table50', he becomes the owner of those tables. 
   This means he has RWXCA perms on those tables.
 Another team member of Mike, Alice wants also to share the same workspace 
 hbase_perf. HBase admin grants Alice also permission to create tables in 
 hbase_perf namespace.
   whoami: hbase
   hbase shell  grant 'alice', 'W', '@hbase_perf'
 Now Alice can create new tables under hbase_perf namespace, but cannot 
 read,write,alter,delete existing tables in the namespace.
  
   whoami: alice
   hbase shell  namespace_list_tables 'hbase_perf'
  hbase_perf.table20
  hbase_perf.table50
   hbase shell  scan 'hbase_perf.table20'
  AccessDeniedException  
  
 If Alice wants to read or write to existing tables in the hbase_perf 
 namespace, hbase admins need to explicitly grant permission.
  
   whoami: hbase
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table20'
   hbase shell  grant 'alice', 'RW', 'hbase_perf.table50'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails


[ 
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729920#comment-13729920
 ] 

Ted Yu commented on HBASE-9072:
---

+1
{code}
+waitForTableToBeOnline(util, table.getTableName());
{code}
The above method waits for flushes. Maybe rename the method to reflect this 
waiting.

 TestSnapshotFromMaster.testSnapshotHFileArchiving fails
 ---

 Key: HBASE-9072
 URL: https://issues.apache.org/jira/browse/HBASE-9072
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Matteo Bertozzi
 Attachments: HBASE-9072-fix0.patch


 http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
 {code}
 Failed 4600 actions: SocketTimeoutException: 4600 times, 
 Stacktrace
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
 org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
 4600 actions: SocketTimeoutException: 4600 times, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
   at 
 org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
   at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
   at 
 org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
 {code}
 [~mbertozzi] Opinion?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729925#comment-13729925
]

Lars Hofhansl commented on HBASE-6580:
--

In the fact the 0.94 patch will be quite a bit different.

Deprecate HTablePool in favor of HConnection.getTable(...)
--

Attachments: 6580-trunk.txt, 6580-trunk-v2.txt, 6580-trunk-v3.txt,
6580-trunk-v4.txt, 6580-trunk-v5.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch

[jira] [Updated] (HBASE-8760) possible loss of data in snapshot taken after region split


 [ 
https://issues.apache.org/jira/browse/HBASE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-8760:
---

Attachment: HBASE-8760-thz-v2.patch

 possible loss of data in snapshot taken after region split
 --

 Key: HBASE-8760
 URL: https://issues.apache.org/jira/browse/HBASE-8760
 Project: HBase
  Issue Type: Bug
  Components: snapshots
Affects Versions: 0.94.8, 0.95.1
Reporter: Jerry He
 Fix For: 0.98.0, 0.95.2, 0.94.12

 Attachments: HBase-8760-0.94.8.patch, HBase-8760-0.94.8-v1.patch, 
 HBASE-8760-thz-v0.patch, HBASE-8760-thz-v1.patch, HBASE-8760-thz-v2.patch


 Right after a region split but before the daughter regions are compacted, we 
 have two daughter regions containing Reference files to the parent hfiles.
 If we take snapshot right at the moment, the snapshot will succeed, but it 
 will only contain the daughter Reference files. Since there is no hold on the 
 parent hfiles, they will be deleted by the HFile Cleaner after they are no 
 longer needed by the daughter regions soon after.
 A minimum we need to do is the keep these parent hfiles from being deleted. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8760) possible loss of data in snapshot taken after region split


[ 
https://issues.apache.org/jira/browse/HBASE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729934#comment-13729934
 ] 

Matteo Bertozzi commented on HBASE-8760:


v2 has the fix requested above, no real code changes.

I want to test this patch more, with all the 3 cases above (normal table, clone 
table, clone of a clone)

[~jerryjch] are you testing the patch on a cluster or just looking at the code?

 possible loss of data in snapshot taken after region split
 --

 Key: HBASE-8760
 URL: https://issues.apache.org/jira/browse/HBASE-8760
 Project: HBase
  Issue Type: Bug
  Components: snapshots
Affects Versions: 0.94.8, 0.95.1
Reporter: Jerry He
 Fix For: 0.98.0, 0.95.2, 0.94.12

 Attachments: HBase-8760-0.94.8.patch, HBase-8760-0.94.8-v1.patch, 
 HBASE-8760-thz-v0.patch, HBASE-8760-thz-v1.patch, HBASE-8760-thz-v2.patch


 Right after a region split but before the daughter regions are compacted, we 
 have two daughter regions containing Reference files to the parent hfiles.
 If we take snapshot right at the moment, the snapshot will succeed, but it 
 will only contain the daughter Reference files. Since there is no hold on the 
 parent hfiles, they will be deleted by the HFile Cleaner after they are no 
 longer needed by the daughter regions soon after.
 A minimum we need to do is the keep these parent hfiles from being deleted. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

2013-08-05 Thread Jimmy Xiang (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729951#comment-13729951
]

Jimmy Xiang commented on HBASE-9095:

+1. Probably we can change how the unit test does verifications later on, so
that we don't need the special counter/tracker.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position

2013-08-05 Thread Matt Corgan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729952#comment-13729952
 ] 

Matt Corgan commented on HBASE-9091:


Looks good

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position


 [ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-9091:


Attachment: 0001-HBASE-9091-Introduce-PositionedByteRange.patch

Addressing Stack's comments.

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position


 [ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-9091:


Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position


 [ 
https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-9091:


Status: Open  (was: Patch Available)

 Update ByteRange to maintain consumer's position
 

 Key: HBASE-9091
 URL: https://issues.apache.org/jira/browse/HBASE-9091
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
 Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Extend-ByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch, 
 0001-HBASE-9091-Introduce-PositionedByteRange.patch


 ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is 
 mutable and an instance can be assigned over a byte[] after instantiation. 
 This is valuable as a performance consideration when working with byte[] 
 slices in a tight loop. Its current design is such that it is not possible to 
 consume a portion of the range while performing activities like decoding an 
 object without altering the definition of the range. It should provide a 
 position that is independent from the range's offset and length to make 
 partial reads easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9095) AssignmentManager's handleRegion should respect the single threaded nature of the processing

[
https://issues.apache.org/jira/browse/HBASE-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729975#comment-13729975
]

Devaraj Das commented on HBASE-9095:

Thanks [~jxiang]. I'll commit the patch in an hour or so unless I hear any
objection.

AssignmentManager's handleRegion should respect the single threaded nature of
the processing

Key: HBASE-9095
URL: https://issues.apache.org/jira/browse/HBASE-9095
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.95.2

Attachments: 9095-1 (1).txt, 9095-1.txt, 9095-1.txt, 9095-1.txt,
9095-2.txt

[jira] [Commented] (HBASE-8408) Implement namespace


[ 
https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729981#comment-13729981
 ] 

stack commented on HBASE-8408:
--

Trying the patch, LoadTestTool is only hunk that would not apply (Easy fix of 
an import).  Also has prob w/

b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java

This pretty brutal in the shell:

{code}
hbase(main):002:0 scan '.META.'

ERROR: java.lang.IllegalArgumentException: Illegal first character 46 at 0. 
Namespaces can only start with alphanumeric characters': i.e. [a-zA-Z_0-9]: 
.META.

Here is some help for this command:
Scan a table; pass table name and optionally a dictionary of scanner
specifications.  Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, TIMESTAMP, MAXLENGTH,
or COLUMNS, CACHE
{code}

Should probably give folks a bit of a nicer letdown.

If I do 'list', I do not see system tables and the help gives me no clue they 
exist.  I suppose this is how it always was.

Should I be able to do this?

hbase(main):013:0 create 'hbase:stack', 'stack'
0 row(s) in 0.1790 seconds

I created a system table named 'stack'?

The 'create' command needs updating so  it has some namespace stuff going on?  
Doesn't it?  It has nothing on it currently.



 Implement namespace
 ---

 Key: HBASE-8408
 URL: https://issues.apache.org/jira/browse/HBASE-8408
 Project: HBase
  Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Francis Liu
 Attachments: HBASE-8015_11.patch, HBASE-8015_12.patch, 
 HBASE-8015_13.patch, HBASE-8015_14.patch, HBASE-8015_15.patch, 
 HBASE-8015_16.patch, HBASE-8015_1.patch, HBASE-8015_2.patch, 
 HBASE-8015_3.patch, HBASE-8015_4.patch, HBASE-8015_5.patch, 
 HBASE-8015_6.patch, HBASE-8015_7.patch, HBASE-8015_8.patch, 
 HBASE-8015_9.patch, HBASE-8015.patch, TestNamespaceMigration.tgz, 
 TestNamespaceUpgrade.tgz




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position