date:20130304


[ 
https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592079#comment-13592079
 ] 

Andrew Purtell commented on HBASE-6222:
---

Some feedback from the Feb 28 HUG: Consider storing a reference to an ACL in 
the KV (logically in the ACL CF or inline as a tag) instead of the ACL itself. 
The ACL could be stored, versioned, in the _acl_ table. 

This is an interesting idea. It would address the case where the same ACL is 
stored in hundreds or thousands (or more) of values. We should save some space 
by writing the actual ACL only to one location. This also facilitates easy 
updates of the ACL, as a single atomic change, especially useful in the case 
where the initial mutations may have included an incorrect ACL. However, the 
tradeoff is two lookups instead of one for deciding what to do with every 
KeyValue: first, to the ACL CF to get the reference, if any, associated with 
the KV (but at least this is within the region); the second to a probably 
remote region of the _acl_ table to retrieve the ACL itself. It could be 
possible to cache the results of the ACL data lookups for a limited time, for 
however long a policy decision can be allowed to be out of date. If we have KV 
tags, the first lookup goes away but unfortunately the second remains, taking 
back some (most?) of the performance/latency gains we expect from inline 
storage.

I'm not sure the benefits outweigh the drawbacks.

 Add per-KeyValue Security
 -

 Key: HBASE-6222
 URL: https://issues.apache.org/jira/browse/HBASE-6222
 Project: HBase
  Issue Type: New Feature
  Components: security
Affects Versions: 0.96.0, 0.98.0
Reporter: stack
Assignee: Andrew Purtell
 Attachments: 6222-aclcf.patch, 6222.pdf, 
 cell-acls-kv-tags-not-for-review.zip, 
 HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx


 Saw an interesting article: 
 http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
 The  Senate Armed Services Committee version of the fiscal 2013 national 
 defense authorization act (S. 3254) would require DoD agencies to foreswear 
 the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO 
 certifies that there exists either no viable commercial open source database 
 with security features comparable to [Accumulo] (such as the HBase or 
 Cassandra databases)...
 Not sure what a 'commercial open source database' is, and I'm not sure whats 
 going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' 
 like Accumulo's, we might put ourselves in the running for federal 
 contributions?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7987) Snapshot Manifest file instead of multiple empty files

Matteo Bertozzi created HBASE-7987:
--

 Summary: Snapshot Manifest file instead of multiple empty files
 Key: HBASE-7987
 URL: https://issues.apache.org/jira/browse/HBASE-7987
 Project: HBase
  Issue Type: Improvement
  Components: snapshots
Reporter: Matteo Bertozzi
Priority: Minor


Currently taking a snapshot means creating one empty file for each file in the 
source table directory, plus copying the .regioninfo file for each region, the 
table descriptor file and a snapshotInfo file.

during the restore or snapshot verification we traverse the filesystem 
(fs.listStatus()) to find the snapshot files, and we open the .regioninfo files 
to get the information.

to avoid hammering the NameNode and having lots of empty files, we can use a 
manifest file that contains the list of files and information that we need.
To keep the RS parallelism that we have, each RS can write its own manifest.

{code}
message SnapshotDescriptor {
  required string name;
  optional string table;
  optional int64 creationTime;
  optional Type type;
  optional int32 version;
}

message SnapshotRegionManifest {
  required RegionInfo regionInfo;
  repeated FamilyFiles familyFiles;

  message StoreFile {
required string name;
optional Reference reference;
  }

  message FamilyFiles {
required bytes familyName;
repeated StoreFile storeFiles;
  }
}
{code}

{code}
/hbase/.snapshot/snapshotName
/hbase/.snapshot/snapshotName/snapshotInfo
/hbase/.snapshot/snapshotName/tableName
/hbase/.snapshot/snapshotName/tableName/tableInfo
/hbase/.snapshot/snapshotName/tableName/regionManifest(.n)
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7544) Transparent table/CF encryption


[ 
https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592085#comment-13592085
 ] 

Andrew Purtell commented on HBASE-7544:
---

Feedback from the Feb 28 HUG: Row key data may leak into encoded region names 
in META and in ZooKeeper znodes. We have not addressed this yet, mainly because 
of the challenge of dealing with META. It should be straightforward to encrypt 
znode data on write and decrypt on read. For META we cannot change the region 
name encoding without disrupting sort order. The solution for obscuring on disk 
META data is for the admin to enable encryption on the META table (and for 
HBase to support META schema configuration changes).

We may simply want to clearly document that constructing row keys with 
sensitive data should be avoided, as it may leak among users of the system. 

Transparent encryption does not address protection of the data of one user from 
another. For this we might propose HTable support for compression codecs for 
mutation data. Aside from being useful for transparent compression, encryption 
codecs can stand in for compression codecs, thus the user can at their option 
encrypt keys and data. (It's an application concern so HTable support for this 
would be convenient but not essential.) Encrypting keys will have obvious 
consequences that should still be documented.

 Transparent table/CF encryption
 ---

 Key: HBASE-7544
 URL: https://issues.apache.org/jira/browse/HBASE-7544
 Project: HBase
  Issue Type: New Feature
  Components: HFile, io
Affects Versions: 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Attachments: 7544.patch, 7544.pdf


 Introduce transparent encryption of HBase on disk data.
 Depends on a separate contribution of an encryption codec framework to Hadoop 
 core and an AES-NI (native code) codec. This is work done in the context of 
 MAPREDUCE-4491 but I'd gather there will be additional JIRAs for common and 
 HDFS parts of it.
 Requirements:
 - Transparent encryption at the CF or table level
 - Protect against all data leakage from files at rest
 - Two-tier key architecture for consistency with best practices for this 
 feature in the RDBMS world
 - Built-in key management
 - Flexible and non-intrusive key rotation
 - Mechanisms not exposed to or modifiable by users
 - Hardware security module integration (via Java KeyStore)
 - HBCK support for transparently encrypted files (+ plugin architecture for 
 HBCK)
 Additional goals:
 - Shell support for administrative functions
 - Avoid performance impact for the null crypto codec case
 - Play nicely with other changes underway: in HFile, block coding, etc.
 We're aiming for rough parity with Oracle's transparent tablespace encryption 
 feature, described in 
 http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
  as
 {quote}
 “Transparent Data Encryption uses a 2-tier key architecture for flexible and 
 non-intrusive key rotation and least operational and performance impact: Each 
 application table with at least one encrypted column has its own table key, 
 which is applied to all encrypted columns in that table. Equally, each 
 encrypted tablespace has its own tablespace key. Table keys are stored in the 
 data dictionary of the database, while tablespace keys are stored in the 
 header of the tablespace and additionally, the header of each underlying OS 
 file that makes up the tablespace.  Each of these keys is encrypted with the 
 TDE master encryption key, which is stored outside of the database in an 
 external security module: either the Oracle Wallet (a PKCS#12 formatted file 
 that is encrypted using a passphrase supplied either by the designated 
 security administrator or DBA during setup),  or a Hardware Security Module 
 (HSM) device for higher assurance […]”
 {quote}
 Further design details forthcoming in a design document and patch as soon as 
 we have all of the clearances in place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-7544) Transparent table/CF encryption

[
https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592085#comment-13592085
]

Andrew Purtell edited comment on HBASE-7544 at 3/4/13 9:36 AM:
---

Feedback from the Feb 28 HUG: Row key data may leak into encoded region names
in META and in ZooKeeper znodes. We have not addressed this yet, mainly because
of the challenge of dealing with META. It should be straightforward to encrypt
znode data on write and decrypt on read. For META we cannot change the region
name encoding without disrupting sort order. The solution for obscuring on disk
META data is for the admin to enable encryption on the META table (and for
HBase to support META schema configuration changes).

We may simply want to clearly document that constructing row keys with
sensitive data should be avoided, as it may leak among users of the system.

Transparent encryption does not address protection of the data of one user from
another. This is outside the scope of this JIRA. To address this other use
case, we might propose HTable support for compression codecs for mutation data.
Aside from being useful for transparent compression, encryption codecs can
stand in for compression codecs, thus the user can at their option encrypt keys
and data. (It's an application concern so HTable support for this would be
convenient but not essential.) Encrypting keys will have obvious consequences
that should still be documented.

was (Author: apurtell):
Feedback from the Feb 28 HUG: Row key data may leak into encoded region
names in META and in ZooKeeper znodes. We have not addressed this yet, mainly
because of the challenge of dealing with META. It should be straightforward to
encrypt znode data on write and decrypt on read. For META we cannot change the
region name encoding without disrupting sort order. The solution for obscuring
on disk META data is for the admin to enable encryption on the META table (and
for HBase to support META schema configuration changes).

We may simply want to clearly document that constructing row keys with
sensitive data should be avoided, as it may leak among users of the system.

Transparent encryption does not address protection of the data of one user from
another. For this we might propose HTable support for compression codecs for
mutation data. Aside from being useful for transparent compression, encryption
codecs can stand in for compression codecs, thus the user can at their option
encrypt keys and data. (It's an application concern so HTable support for this
would be convenient but not essential.) Encrypting keys will have obvious
consequences that should still be documented.

Transparent table/CF encryption
---

Key: HBASE-7544
URL: https://issues.apache.org/jira/browse/HBASE-7544
Project: HBase
Issue Type: New Feature
Components: HFile, io
Affects Versions: 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 7544.patch, 7544.pdf

Introduce transparent encryption of HBase on disk data.
Depends on a separate contribution of an encryption codec framework to Hadoop
core and an AES-NI (native code) codec. This is work done in the context of
MAPREDUCE-4491 but I'd gather there will be additional JIRAs for common and
HDFS parts of it.
Requirements:
- Transparent encryption at the CF or table level
- Protect against all data leakage from files at rest
- Two-tier key architecture for consistency with best practices for this
feature in the RDBMS world
- Built-in key management
- Flexible and non-intrusive key rotation
- Mechanisms not exposed to or modifiable by users
- Hardware security module integration (via Java KeyStore)
- HBCK support for transparently encrypted files (+ plugin architecture for
HBCK)
Additional goals:
- Shell support for administrative functions
- Avoid performance impact for the null crypto codec case
- Play nicely with other changes underway: in HFile, block coding, etc.
We're aiming for rough parity with Oracle's transparent tablespace encryption
feature, described in
http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
{quote}
“Transparent Data Encryption uses a 2-tier key architecture for flexible and
non-intrusive key rotation and least operational and performance impact: Each
application table with at least one encrypted column has its own table key,
which is applied to all encrypted columns in that table. Equally, each
encrypted tablespace has its own tablespace key. Table keys are stored in the
data dictionary of the database, while tablespace keys are stored in the
header of the tablespace and additionally, the header of each underlying OS

[jira] [Updated] (HBASE-7987) Snapshot Manifest file instead of multiple empty files


 [ 
https://issues.apache.org/jira/browse/HBASE-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7987:
---

Description: 
Currently taking a snapshot means creating one empty file for each file in the 
source table directory, plus copying the .regioninfo file for each region, the 
table descriptor file and a snapshotInfo file.

during the restore or snapshot verification we traverse the filesystem 
(fs.listStatus()) to find the snapshot files, and we open the .regioninfo files 
to get the information.

to avoid hammering the NameNode and having lots of empty files, we can use a 
manifest file that contains the list of files and information that we need.
To keep the RS parallelism that we have, each RS can write its own manifest.

{code}
message SnapshotDescriptor {
  required string name;
  optional string table;
  optional int64 creationTime;
  optional Type type;
  optional int32 version;
}

message SnapshotRegionManifest {
  optional int32 version;
  required RegionInfo regionInfo;
  repeated FamilyFiles familyFiles;

  message StoreFile {
required string name;
optional Reference reference;
  }

  message FamilyFiles {
required bytes familyName;
repeated StoreFile storeFiles;
  }
}
{code}

{code}
/hbase/.snapshot/snapshotName
/hbase/.snapshot/snapshotName/snapshotInfo
/hbase/.snapshot/snapshotName/tableName
/hbase/.snapshot/snapshotName/tableName/tableInfo
/hbase/.snapshot/snapshotName/tableName/regionManifest(.n)
{code}


  was:
Currently taking a snapshot means creating one empty file for each file in the 
source table directory, plus copying the .regioninfo file for each region, the 
table descriptor file and a snapshotInfo file.

during the restore or snapshot verification we traverse the filesystem 
(fs.listStatus()) to find the snapshot files, and we open the .regioninfo files 
to get the information.

to avoid hammering the NameNode and having lots of empty files, we can use a 
manifest file that contains the list of files and information that we need.
To keep the RS parallelism that we have, each RS can write its own manifest.

{code}
message SnapshotDescriptor {
  required string name;
  optional string table;
  optional int64 creationTime;
  optional Type type;
  optional int32 version;
}

message SnapshotRegionManifest {
  required RegionInfo regionInfo;
  repeated FamilyFiles familyFiles;

  message StoreFile {
required string name;
optional Reference reference;
  }

  message FamilyFiles {
required bytes familyName;
repeated StoreFile storeFiles;
  }
}
{code}

{code}
/hbase/.snapshot/snapshotName
/hbase/.snapshot/snapshotName/snapshotInfo
/hbase/.snapshot/snapshotName/tableName
/hbase/.snapshot/snapshotName/tableName/tableInfo
/hbase/.snapshot/snapshotName/tableName/regionManifest(.n)
{code}



 Snapshot Manifest file instead of multiple empty files
 --

 Key: HBASE-7987
 URL: https://issues.apache.org/jira/browse/HBASE-7987
 Project: HBase
  Issue Type: Improvement
  Components: snapshots
Reporter: Matteo Bertozzi
Priority: Minor

 Currently taking a snapshot means creating one empty file for each file in 
 the source table directory, plus copying the .regioninfo file for each 
 region, the table descriptor file and a snapshotInfo file.
 during the restore or snapshot verification we traverse the filesystem 
 (fs.listStatus()) to find the snapshot files, and we open the .regioninfo 
 files to get the information.
 to avoid hammering the NameNode and having lots of empty files, we can use a 
 manifest file that contains the list of files and information that we need.
 To keep the RS parallelism that we have, each RS can write its own manifest.
 {code}
 message SnapshotDescriptor {
   required string name;
   optional string table;
   optional int64 creationTime;
   optional Type type;
   optional int32 version;
 }
 message SnapshotRegionManifest {
   optional int32 version;
   required RegionInfo regionInfo;
   repeated FamilyFiles familyFiles;
   message StoreFile {
 required string name;
 optional Reference reference;
   }
   message FamilyFiles {
 required bytes familyName;
 repeated StoreFile storeFiles;
   }
 }
 {code}
 {code}
 /hbase/.snapshot/snapshotName
 /hbase/.snapshot/snapshotName/snapshotInfo
 /hbase/.snapshot/snapshotName/tableName
 /hbase/.snapshot/snapshotName/tableName/tableInfo
 /hbase/.snapshot/snapshotName/tableName/regionManifest(.n)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6222) Add per-KeyValue Security


[ 
https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592095#comment-13592095
 ] 

nkeywal commented on HBASE-6222:


If I understand well, the choice depends on the number of different ACL are we 
expecting and if we have access patterns that would make caching efficient. Do 
we know this? With a reference based implementation, for a scan it would make 
sense  to first check the reference of the ACL that gives us a read access, and 
then use this for all the kv in the table. The caching duration policy could be 
per scan. 

Lastly, if it's considered as nearly immutable data, it's possible to use a 
ZooKeeper node to invalidate the cache on an update. This works well if you 
have a few update per hour (with as many insert as you like).  Then policy is 
'infinite cache'.

 Add per-KeyValue Security
 -

 Key: HBASE-6222
 URL: https://issues.apache.org/jira/browse/HBASE-6222
 Project: HBase
  Issue Type: New Feature
  Components: security
Affects Versions: 0.96.0, 0.98.0
Reporter: stack
Assignee: Andrew Purtell
 Attachments: 6222-aclcf.patch, 6222.pdf, 
 cell-acls-kv-tags-not-for-review.zip, 
 HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx


 Saw an interesting article: 
 http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
 The  Senate Armed Services Committee version of the fiscal 2013 national 
 defense authorization act (S. 3254) would require DoD agencies to foreswear 
 the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO 
 certifies that there exists either no viable commercial open source database 
 with security features comparable to [Accumulo] (such as the HBase or 
 Cassandra databases)...
 Not sure what a 'commercial open source database' is, and I'm not sure whats 
 going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' 
 like Accumulo's, we might put ourselves in the running for federal 
 contributions?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6222) Add per-KeyValue Security


[ 
https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592103#comment-13592103
 ] 

Andrew Purtell commented on HBASE-6222:
---

bq. the choice depends on the number of different ACL are we expecting and if 
we have access patterns that would make caching efficient. Do we know this?

Maybe for a given row. Wouldn't say so in general but there's no real world 
user usage data. We should expect best practice for a very common cell ACL is a 
factoring of it to out a CF or table grant, to avoid any IO checking cover for 
mutations. So at the cell level either no data or probably lots of varying 
ACLs. 

 Add per-KeyValue Security
 -

 Key: HBASE-6222
 URL: https://issues.apache.org/jira/browse/HBASE-6222
 Project: HBase
  Issue Type: New Feature
  Components: security
Affects Versions: 0.96.0, 0.98.0
Reporter: stack
Assignee: Andrew Purtell
 Attachments: 6222-aclcf.patch, 6222.pdf, 
 cell-acls-kv-tags-not-for-review.zip, 
 HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx


 Saw an interesting article: 
 http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
 The  Senate Armed Services Committee version of the fiscal 2013 national 
 defense authorization act (S. 3254) would require DoD agencies to foreswear 
 the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO 
 certifies that there exists either no viable commercial open source database 
 with security features comparable to [Accumulo] (such as the HBase or 
 Cassandra databases)...
 Not sure what a 'commercial open source database' is, and I'm not sure whats 
 going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' 
 like Accumulo's, we might put ourselves in the running for federal 
 contributions?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7968) Packaging of Trunk and 0.95 does not create the dependent jars in the lib folder


[ 
https://issues.apache.org/jira/browse/HBASE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592108#comment-13592108
 ] 

ramkrishna.s.vasudevan commented on HBASE-7968:
---

@N
You going to commit this patch?

 Packaging of Trunk and 0.95 does not create the dependent jars in the lib 
 folder
 

 Key: HBASE-7968
 URL: https://issues.apache.org/jira/browse/HBASE-7968
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0, 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: nkeywal
Priority: Critical
 Fix For: 0.95.0, 0.98.0

 Attachments: 7968.v1.patch


 After recent changes to trunk and 0.95 branch when i try to build and 
 package, i do not find the dependent jars in the lib folder.
 Prior to the changes, it was working fine.
 Am not a maven expert.  Will try to see what is going wrong here. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7962) Native folder not available under lib folder in the HBase trunk deployment


[ 
https://issues.apache.org/jira/browse/HBASE-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592117#comment-13592117
 ] 

ramkrishna.s.vasudevan commented on HBASE-7962:
---

Still even if HBASE-7968 gets solved we need to see how to bring in the native 
folder inside lib.  

 Native folder not available under lib folder in the HBase trunk deployment
 --

 Key: HBASE-7962
 URL: https://issues.apache.org/jira/browse/HBASE-7962
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan

 I was trying to install Snappy.  Found this link.
 http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with#.US7JNzDX_D4.
 As per this link there is a native folder under lib directory in the HBase 
 installation path and it is available in 0.94.
 But when i install HBase trunk i could not see one
 If am not finding this in the correct place, pls invalidate the issue. Will 
 try figuring out what could be the problem

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592137#comment-13592137
 ] 

Matteo Bertozzi commented on HBASE-7360:


v1 committed to 0.94

 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: New Feature
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.94.6

 Attachments: 7360-v1.patch, HBASE-7360-jiras.txt, HBASE-7360-v0.patch


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 - HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 - Take a Snapshot interface
  * HBASE-6571 - Generic multi-thread/cross-process error handling framework
  * HBASE-6353 - Snapshots shell
  * HBASE-7107 - Snapshot References Utils (FileSystem Visitor)
  * HBASE-6863 - Offline snapshots
  * HBASE-6865 - Snapshot File Cleaners
  * HBASE-6777 - Snapshot Restore Interface
  * HBASE-6230 - Clone/Restore Snapshots
  * HBASE-6802 - Export Snapshot
  * HBASE-7864 - Rename HMaster#listSnapshots as getCompletedSnapshots()
  * HBASE-7858 - cleanup before merging snapshots branch to trunk
  * HBASE-7969 - Rename HBaseAdmin#getCompletedSnapshots as 
 HBaseAdmin#listSnapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7360) Snapshot 0.94 Backport


 [ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7360:
---

Description: 
Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs 
writable rpc.

Offline Snapshot
HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
 * HBASE-6765 Take a Snapshot interface
 * HBASE-6571 Generic multi-thread/cross-process error handling framework
 * HBASE-6353 Snapshots shell
 * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
 * HBASE-6836 Offline snapshots
 * HBASE-6865 Snapshot File Cleaners
 * HBASE-6777 Snapshot Restore Interface
 * HBASE-6230 Clone/Restore Snapshots
 * HBASE-6802 Export Snapshot
 * HBASE-7240 Cleanup old snapshots on start
 * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
 * HBASE-7367 Snapshot coprocessor and ACL security
 * HBASE-7311 Add snapshot information to hbase master webui
 * HBASE-7418 HFileLink flaky test
 * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing test 
category annotation and failing TestCheckTestClasses
 * HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)
 * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
(EnvironmentEdge issue)
 * HBASE-7436 Improve stack trace info dumped by 
ForeignExceptionSnare#rethrowException
 * HBASE-7339 Splitting an HFileLink causes region servers to go down.
 * HBASE-7439 HFileLink should not use the configuration from the Filesystem
 * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
#receive(FE)
 * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
for merge
 * HBASE-7207 Consolidate snapshot related classes into fewer packages
 * HBASE-7400 ExportSnapshot mapper closes the FileSystem
 * HBASE-7352 clone operation from HBaseAdmin can hang forever
 * HBASE-7294 Check for snapshot file cleaners on start
 * HBASE-7354 Snapshot Info/Debug Tool
 * HBASE-7423 HFileArchiver should not use the configuration from the Filesystem
 * HBASE-7453 HBASE-7423 snapshot followup
 * HBASE-7212 Globally Barriered Procedure Mechanism
 * HBASE-6864 Online snapshots scaffolding
 * HBASE-7321 Simple Flush Snapshot
 * HBASE-7496 TestZKProcedure fails interrmittently.
 * HBASE-7484 Fix Restore with schema changes
 * HBASE-7419 revisit hfilelink file name format
 * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
directories
 * HBASE-7214 CleanerChore logs too much, so much so it obscures all else that 
is going on
 * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
fails sometimes.
 * HBASE-7480 Explicit message for not allowed snapshot on meta tables
 * HBASE-7537 .regioninfo not created by createHRegion()
 * HBASE-7535 Fix restore reference files
 * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
FileLink.open method, and remove unnecessary import packages.
 * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
.META. table
 * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
 * HBASE-7547 Fix findbugs warnings in snapshot classes
 * HBASE-7548 Fix javadoc warnings in snapshot classes
 * HBASE-7538 Improve snapshot related error and exception messages
 * HBASE-7536 Add test that confirms that multiple concurrent snapshot requests 
are rejected
 * HBASE-7583 Fixes and cleanups
 * HBASE-7604 Remove duplicated code from HFileLink
 * HBASE-7616 NPE in ZKProcedure.nodeCreated
 * HBASE-7625 Remove duplicated logFSTree() from 
TestRestoreFlushSnapshotFromClient
 * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
 * HBASE-7622 Add table descriptor verification after snapshot restore
 * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
previous snapshot fails in per region task
 * HBASE-7666 More logging improvements in online snapshots code
 * HBASE-7689 CloneTableHandler notify completion too early
 * HBASE-7699 Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
 * HBASE-7703 Eventually all online snapshots failing due to Timeout at same 
regionserver.
 * HBASE-7720 Improve logging messages of failed snapshot attempts.
 * HBASE-7795 Race in the Restore Archiving
 * HBASE-7788 Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
 * HBASE-7858 cleanup before merging snapshots branch to trunk
 * HBASE-7889 Fix javadoc warnings in snapshot classes
 * HBASE-7911 Remove duplicated code from CreateTableHandler
 * HBASE-7969 Rename HBaseAdmin#getCompletedSnapshots as 
HBaseAdmin#listSnapshots

  was:
Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs 
writable rpc.

Offline Snapshot
 * HBASE-6610 - HFileLink: Hardlink alternative for snapshot restore
 * HBASE-6765 - Take a Snapshot interface
 *

[jira] [Commented] (HBASE-6222) Add per-KeyValue Security


[ 
https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592138#comment-13592138
 ] 

nkeywal commented on HBASE-6222:


Ok. For the use cases, do we know how Accumulo is used today?

 Add per-KeyValue Security
 -

 Key: HBASE-6222
 URL: https://issues.apache.org/jira/browse/HBASE-6222
 Project: HBase
  Issue Type: New Feature
  Components: security
Affects Versions: 0.96.0, 0.98.0
Reporter: stack
Assignee: Andrew Purtell
 Attachments: 6222-aclcf.patch, 6222.pdf, 
 cell-acls-kv-tags-not-for-review.zip, 
 HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx


 Saw an interesting article: 
 http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
 The  Senate Armed Services Committee version of the fiscal 2013 national 
 defense authorization act (S. 3254) would require DoD agencies to foreswear 
 the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO 
 certifies that there exists either no viable commercial open source database 
 with security features comparable to [Accumulo] (such as the HBase or 
 Cassandra databases)...
 Not sure what a 'commercial open source database' is, and I'm not sure whats 
 going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' 
 like Accumulo's, we might put ourselves in the running for federal 
 contributions?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7360) Snapshot 0.94 Backport


 [ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7360:
---

Description: 
Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs 
writable rpc.

Offline Snapshot
HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
 * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6571 Generic multi-thread/cross-process error handling framework
 * HBASE-6353 Snapshots shell
 * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
 * HBASE-6836 Offline snapshots
 * HBASE-6865 Snapshot File Cleaners
 * HBASE-6777 Snapshot Restore Interface
 ** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6230 Clone/Restore Snapshots
 * HBASE-6802 Export Snapshot
 * HBASE-7240 Cleanup old snapshots on start
 * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
 * HBASE-7367 Snapshot coprocessor and ACL security
 * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-7418 HFileLink flaky test
 * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing test 
category annotation and failing TestCheckTestClasses
 * HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)
 * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
(EnvironmentEdge issue)
 * HBASE-7436 Improve stack trace info dumped by 
ForeignExceptionSnare#rethrowException
 * HBASE-7339 Splitting an HFileLink causes region servers to go down.
 * HBASE-7439 HFileLink should not use the configuration from the Filesystem
 * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
#receive(FE)
 * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
for merge
 * HBASE-7207 Consolidate snapshot related classes into fewer packages
 * HBASE-7400 ExportSnapshot mapper closes the FileSystem
 * HBASE-7352 clone operation from HBaseAdmin can hang forever
 * HBASE-7294 Check for snapshot file cleaners on start
 * HBASE-7354 Snapshot Info/Debug Tool
 * HBASE-7423 HFileArchiver should not use the configuration from the Filesystem
 * HBASE-7453 HBASE-7423 snapshot followup
 * HBASE-7212 Globally Barriered Procedure Mechanism
 * HBASE-6864 Online snapshots scaffolding
 * HBASE-7321 Simple Flush Snapshot
 * HBASE-7496 TestZKProcedure fails interrmittently.
 * HBASE-7484 Fix Restore with schema changes
 * HBASE-7419 revisit hfilelink file name format
 * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
directories
 * HBASE-7214 CleanerChore logs too much, so much so it obscures all else that 
is going on
 * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
fails sometimes.
 * HBASE-7480 Explicit message for not allowed snapshot on meta tables
 * HBASE-7537 .regioninfo not created by createHRegion()
 * HBASE-7535 Fix restore reference files
 * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
FileLink.open method, and remove unnecessary import packages.
 * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
.META. table
 * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
 * HBASE-7547 Fix findbugs warnings in snapshot classes
 * HBASE-7548 Fix javadoc warnings in snapshot classes
 * HBASE-7538 Improve snapshot related error and exception messages
 * HBASE-7536 Add test that confirms that multiple concurrent snapshot requests 
are rejected
 * HBASE-7583 Fixes and cleanups
 * HBASE-7604 Remove duplicated code from HFileLink
 * HBASE-7616 NPE in ZKProcedure.nodeCreated
 * HBASE-7625 Remove duplicated logFSTree() from 
TestRestoreFlushSnapshotFromClient
 * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
 * HBASE-7622 Add table descriptor verification after snapshot restore
 * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
previous snapshot fails in per region task
 * HBASE-7666 More logging improvements in online snapshots code
 * HBASE-7689 CloneTableHandler notify completion too early
 * HBASE-7699 Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
 * HBASE-7703 Eventually all online snapshots failing due to Timeout at same 
regionserver.
 * HBASE-7720 Improve logging messages of failed snapshot attempts.
 * HBASE-7795 Race in the Restore Archiving
 * HBASE-7788 Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
 * HBASE-7858 cleanup before merging snapshots branch to trunk
 * HBASE-7889 Fix javadoc warnings in snapshot classes
 * HBASE-7911 Remove duplicated code from CreateTableHandler
 * HBASE-7969 Rename HBaseAdmin#getCompletedSnapshots as 
HBaseAdmin#listSnapshots

  was:

[jira] [Updated] (HBASE-7360) Snapshot 0.94 Backport


 [ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7360:
---

Description: 
Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs 
writable rpc.

Offline Snapshot
 * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
 * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6571 Generic multi-thread/cross-process error handling framework
 * HBASE-6353 Snapshots shell
 * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
 * HBASE-6836 Offline snapshots
 * HBASE-6865 Snapshot File Cleaners
 * HBASE-6777 Snapshot Restore Interface
** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6230 Clone/Restore Snapshots
 * HBASE-6802 Export Snapshot
 * HBASE-7240 Cleanup old snapshots on start
 * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
 * HBASE-7367 Snapshot coprocessor and ACL security
 * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-7418 HFileLink flaky test
 * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing test 
category annotation and failing TestCheckTestClasses
 * HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)
 * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
(EnvironmentEdge issue)
 * HBASE-7436 Improve stack trace info dumped by 
ForeignExceptionSnare#rethrowException
 * HBASE-7339 Splitting an HFileLink causes region servers to go down.
 * HBASE-7439 HFileLink should not use the configuration from the Filesystem
 * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
#receive(FE)
 * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
for merge
 * HBASE-7207 Consolidate snapshot related classes into fewer packages
 * HBASE-7400 ExportSnapshot mapper closes the FileSystem
 * HBASE-7352 clone operation from HBaseAdmin can hang forever
 * HBASE-7294 Check for snapshot file cleaners on start
 * HBASE-7354 Snapshot Info/Debug Tool
 * HBASE-7423 HFileArchiver should not use the configuration from the Filesystem
 * HBASE-7453 HBASE-7423 snapshot followup
 * HBASE-7212 Globally Barriered Procedure Mechanism
 * HBASE-6864 Online snapshots scaffolding
 * HBASE-7321 Simple Flush Snapshot
 * HBASE-7496 TestZKProcedure fails interrmittently.
 * HBASE-7484 Fix Restore with schema changes
 * HBASE-7419 revisit hfilelink file name format
 * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
directories
 * HBASE-7214 CleanerChore logs too much, so much so it obscures all else that 
is going on
 * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
fails sometimes.
 * HBASE-7480 Explicit message for not allowed snapshot on meta tables
 * HBASE-7537 .regioninfo not created by createHRegion()
 * HBASE-7535 Fix restore reference files
 * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
FileLink.open method, and remove unnecessary import packages.
 * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
.META. table
 * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
 * HBASE-7547 Fix findbugs warnings in snapshot classes
 * HBASE-7548 Fix javadoc warnings in snapshot classes
 * HBASE-7538 Improve snapshot related error and exception messages
 * HBASE-7536 Add test that confirms that multiple concurrent snapshot requests 
are rejected
 * HBASE-7583 Fixes and cleanups
 * HBASE-7604 Remove duplicated code from HFileLink
 * HBASE-7616 NPE in ZKProcedure.nodeCreated
 * HBASE-7625 Remove duplicated logFSTree() from 
TestRestoreFlushSnapshotFromClient
 * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
 * HBASE-7622 Add table descriptor verification after snapshot restore
 * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
previous snapshot fails in per region task
 * HBASE-7666 More logging improvements in online snapshots code
 * HBASE-7689 CloneTableHandler notify completion too early
 * HBASE-7699 Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
 * HBASE-7703 Eventually all online snapshots failing due to Timeout at same 
regionserver.
 * HBASE-7720 Improve logging messages of failed snapshot attempts.
 * HBASE-7795 Race in the Restore Archiving
 * HBASE-7788 Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
 * HBASE-7858 cleanup before merging snapshots branch to trunk
 * HBASE-7889 Fix javadoc warnings in snapshot classes
 * HBASE-7911 Remove duplicated code from CreateTableHandler
 * HBASE-7969 Rename HBaseAdmin#getCompletedSnapshots as 
HBaseAdmin#listSnapshots

[jira] [Updated] (HBASE-7360) Snapshot 0.94 Backport


 [ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7360:
---

Description: 
Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs 
writable rpc.

Offline Snapshot
 * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
 * HBASE-6765 Take a Snapshot interface
** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6571 Generic multi-thread/cross-process error handling framework
 * HBASE-6353 Snapshots shell
 * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
 * HBASE-6836 Offline snapshots
 * HBASE-6865 Snapshot File Cleaners
 * HBASE-6777 Snapshot Restore Interface
** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-6230 Clone/Restore Snapshots
 * HBASE-6802 Export Snapshot
 * HBASE-7240 Cleanup old snapshots on start
 * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
 * HBASE-7367 Snapshot coprocessor and ACL security
 * HBASE-7311 Add snapshot information to hbase master webui
** This patch is significantly different because of the RPC changes between 
0.94/0.96
 * HBASE-7418 HFileLink flaky test
 * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing test 
category annotation and failing TestCheckTestClasses
 * HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)
 * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
(EnvironmentEdge issue)
 * HBASE-7436 Improve stack trace info dumped by 
ForeignExceptionSnare#rethrowException
 * HBASE-7339 Splitting an HFileLink causes region servers to go down.
 * HBASE-7439 HFileLink should not use the configuration from the Filesystem
 * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
#receive(FE)
 * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
for merge
 * HBASE-7207 Consolidate snapshot related classes into fewer packages
 * HBASE-7400 ExportSnapshot mapper closes the FileSystem
 * HBASE-7352 clone operation from HBaseAdmin can hang forever
 * HBASE-7294 Check for snapshot file cleaners on start
 * HBASE-7354 Snapshot Info/Debug Tool
 * HBASE-7423 HFileArchiver should not use the configuration from the Filesystem
 * HBASE-7453 HBASE-7423 snapshot followup
 * HBASE-7212 Globally Barriered Procedure Mechanism
 * HBASE-6864 Online snapshots scaffolding
 * HBASE-7321 Simple Flush Snapshot
 * HBASE-7496 TestZKProcedure fails interrmittently.
 * HBASE-7484 Fix Restore with schema changes
 * HBASE-7419 revisit hfilelink file name format
 * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
directories
 * HBASE-7214 CleanerChore logs too much, so much so it obscures all else that 
is going on
 * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
fails sometimes.
 * HBASE-7480 Explicit message for not allowed snapshot on meta tables
 * HBASE-7537 .regioninfo not created by createHRegion()
 * HBASE-7535 Fix restore reference files
 * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
FileLink.open method, and remove unnecessary import packages.
 * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
.META. table
 * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
 * HBASE-7547 Fix findbugs warnings in snapshot classes
 * HBASE-7548 Fix javadoc warnings in snapshot classes
 * HBASE-7538 Improve snapshot related error and exception messages
 * HBASE-7536 Add test that confirms that multiple concurrent snapshot requests 
are rejected
 * HBASE-7583 Fixes and cleanups
 * HBASE-7604 Remove duplicated code from HFileLink
 * HBASE-7616 NPE in ZKProcedure.nodeCreated
 * HBASE-7625 Remove duplicated logFSTree() from 
TestRestoreFlushSnapshotFromClient
 * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
 * HBASE-7622 Add table descriptor verification after snapshot restore
 * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
previous snapshot fails in per region task
 * HBASE-7666 More logging improvements in online snapshots code
 * HBASE-7689 CloneTableHandler notify completion too early
 * HBASE-7699 Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
 * HBASE-7703 Eventually all online snapshots failing due to Timeout at same 
regionserver.
 * HBASE-7720 Improve logging messages of failed snapshot attempts.
 * HBASE-7795 Race in the Restore Archiving
 * HBASE-7788 Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
 * HBASE-7858 cleanup before merging snapshots branch to trunk
 * HBASE-7889 Fix javadoc warnings in snapshot classes
 * HBASE-7911 Remove duplicated code from CreateTableHandler
 * HBASE-7969 Rename HBaseAdmin#getCompletedSnapshots as 
HBaseAdmin#listSnapshots

  was:

[jira] [Resolved] (HBASE-7968) Packaging of Trunk and 0.95 does not create the dependent jars in the lib folder


 [ 
https://issues.apache.org/jira/browse/HBASE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal resolved HBASE-7968.


  Resolution: Fixed
Hadoop Flags: Reviewed

 Packaging of Trunk and 0.95 does not create the dependent jars in the lib 
 folder
 

 Key: HBASE-7968
 URL: https://issues.apache.org/jira/browse/HBASE-7968
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0, 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: nkeywal
Priority: Critical
 Fix For: 0.95.0, 0.98.0

 Attachments: 7968.v1.patch


 After recent changes to trunk and 0.95 branch when i try to build and 
 package, i do not find the dependent jars in the lib folder.
 Prior to the changes, it was working fine.
 Am not a maven expert.  Will try to see what is going wrong here. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7968) Packaging of Trunk and 0.95 does not create the dependent jars in the lib folder


[ 
https://issues.apache.org/jira/browse/HBASE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592139#comment-13592139
 ] 

nkeywal commented on HBASE-7968:


0.95 done, it seems to fail on 0.97. Looking.

 Packaging of Trunk and 0.95 does not create the dependent jars in the lib 
 folder
 

 Key: HBASE-7968
 URL: https://issues.apache.org/jira/browse/HBASE-7968
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0, 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: nkeywal
Priority: Critical
 Fix For: 0.95.0, 0.98.0

 Attachments: 7968.v1.patch


 After recent changes to trunk and 0.95 branch when i try to build and 
 package, i do not find the dependent jars in the lib folder.
 Prior to the changes, it was working fine.
 Am not a maven expert.  Will try to see what is going wrong here. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7968) Packaging of Trunk and 0.95 does not create the dependent jars in the lib folder


[ 
https://issues.apache.org/jira/browse/HBASE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592141#comment-13592141
 ] 

nkeywal commented on HBASE-7968:


I had an svn error message but the commit was done. 
So we're done. Sorry for the delay on committing, Ram, I was planning to do it 
this week end but I was finally not available, and I wanted to check as well 
that adding all the modules does not break anything for both hadoop 1 and 
hadoop 2 profiles on 95 and 97 (it seems to be ok... hopefully!).

 Packaging of Trunk and 0.95 does not create the dependent jars in the lib 
 folder
 

 Key: HBASE-7968
 URL: https://issues.apache.org/jira/browse/HBASE-7968
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0, 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: nkeywal
Priority: Critical
 Fix For: 0.95.0, 0.98.0

 Attachments: 7968.v1.patch


 After recent changes to trunk and 0.95 branch when i try to build and 
 package, i do not find the dependent jars in the lib folder.
 Prior to the changes, it was working fine.
 Am not a maven expert.  Will try to see what is going wrong here. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7966) ACL tests fail on trunk (flaky)

2013-03-04 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592147#comment-13592147
 ] 

Hudson commented on HBASE-7966:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #429 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/429/])
HBASE-7966 ACL tests fail on trunk (flaky) (Revision 1452161)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java


 ACL tests fail on trunk (flaky)
 ---

 Key: HBASE-7966
 URL: https://issues.apache.org/jira/browse/HBASE-7966
 Project: HBase
  Issue Type: Bug
  Components: security
Reporter: Enis Soztutar
Assignee: stack
 Fix For: 0.95.0, 0.98.0

 Attachments: 7966-v1.txt, 7966v3.txt, 7966v4.txt, debug.logging.txt


 I've noticed that these tests fail for me quite frequently:
 {code}
   org.apache.hadoop.hbase.security.access.TestTablePermissions: 
 org.apache.hadoop.hbase.exceptions.TableNotFoundException: _acl_
   
 testQualifierAccess(org.apache.hadoop.hbase.security.access.TestAccessControlFilter):
  org.apache.hadoop.hbase.exceptions.TableNotFoundException: _acl_
 {code}
 The root cause seems to be that, the AccessController only creates the table 
 in HMaster.postStartMaster(), but before _acl_ is created, master start 
 accepting other create table requests, hence race. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7808) Refactor Store to use HRegionFileSystem


 [ 
https://issues.apache.org/jira/browse/HBASE-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7808:
---

Attachment: HBASE-7808-v4.patch

 Refactor Store to use HRegionFileSystem
 ---

 Key: HBASE-7808
 URL: https://issues.apache.org/jira/browse/HBASE-7808
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: HBASE-7808-v0.patch, HBASE-7808-v1.patch, 
 HBASE-7808-v2.patch, HBASE-7808-v3.patch, HBASE-7808-v4.patch


 Move HStore fs related code inside the HRegionFileSystem. 
 File creation and retrival should be handled there, and the use will be 
 something like.
 {code}
 HRegionFileSystem fs = new HRegionFileSystem
 file = fs.createFile()
 fs.commitFile(family, file);
 for (storeFile: fs.getStoreFiles(family))
...
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-7615) Add metrics for snapshots


 [ 
https://issues.apache.org/jira/browse/HBASE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reassigned HBASE-7615:
--

Assignee: Matteo Bertozzi

 Add metrics for snapshots
 -

 Key: HBASE-7615
 URL: https://issues.apache.org/jira/browse/HBASE-7615
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Matteo Bertozzi

 Metrics should be added for snapshot.
 From Matteo: Output that we have in SnapshotInfo should be covered:
 %d HFiles (%d in archive), total size %s (%.2f%% %s shared with the source 
 table)
 Jesse mentioned snaphot counts, average time to completion.
 I think we should have counts for successful / failed snapshots.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6222) Add per-KeyValue Security


[ 
https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592170#comment-13592170
 ] 

Andrew Purtell commented on HBASE-6222:
---

bq. For the use cases, do we know how Accumulo is used today?

Accumulo provides labels, HBASE-7663

 Add per-KeyValue Security
 -

 Key: HBASE-6222
 URL: https://issues.apache.org/jira/browse/HBASE-6222
 Project: HBase
  Issue Type: New Feature
  Components: security
Affects Versions: 0.96.0, 0.98.0
Reporter: stack
Assignee: Andrew Purtell
 Attachments: 6222-aclcf.patch, 6222.pdf, 
 cell-acls-kv-tags-not-for-review.zip, 
 HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx


 Saw an interesting article: 
 http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
 The  Senate Armed Services Committee version of the fiscal 2013 national 
 defense authorization act (S. 3254) would require DoD agencies to foreswear 
 the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO 
 certifies that there exists either no viable commercial open source database 
 with security features comparable to [Accumulo] (such as the HBase or 
 Cassandra databases)...
 Not sure what a 'commercial open source database' is, and I'm not sure whats 
 going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' 
 like Accumulo's, we might put ourselves in the running for federal 
 contributions?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7962) Native folder not available under lib folder in the HBase trunk deployment


[ 
https://issues.apache.org/jira/browse/HBASE-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592173#comment-13592173
 ] 

Andrew Purtell commented on HBASE-7962:
---

Set LD_LIBRARY_PATH and java.library.path to where the native Hadoop libraries 
are installed.

 Native folder not available under lib folder in the HBase trunk deployment
 --

 Key: HBASE-7962
 URL: https://issues.apache.org/jira/browse/HBASE-7962
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan

 I was trying to install Snappy.  Found this link.
 http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with#.US7JNzDX_D4.
 As per this link there is a native folder under lib directory in the HBase 
 installation path and it is available in 0.94.
 But when i install HBase trunk i could not see one
 If am not finding this in the correct place, pls invalidate the issue. Will 
 try figuring out what could be the problem

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport

2013-03-04 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592180#comment-13592180
 ] 

Hudson commented on HBASE-7360:
---

Integrated in HBase-0.94 #878 (See 
[https://builds.apache.org/job/HBase-0.94/878/])
HBASE-7360 Backport Snapshots to 0.94 (Revision 1452257)

 Result = FAILURE
mbertozzi : 
Files : 
* 
/hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* 
/hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
* 
/hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/Chore.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/DaemonThreadFactory.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignException.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionDispatcher.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionListener.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionSnare.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutException.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutExceptionInjector.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/FileLink.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/HFileLink.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/HLogLink.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SnapshotSentinel.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileLinkCleaner.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/MasterSnapshotVerifier.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
*

[jira] [Updated] (HBASE-7360) Snapshot 0.94 Backport


 [ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7360:
---

Attachment: HBASE-7299-0.94.patch

https://builds.apache.org/job/HBase-0.94/878/
{code}
Failed tests:   
testFlushCommitsWithAbort(org.apache.hadoop.hbase.client.TestMultiParallel): 
Server count=2, abort=true expected:1 but was:2

Tests in error:
  testFlushCommitsNoAbort(org.apache.hadoop.hbase.client.TestMultiParallel): 
Server hemera.apache.org,32876,1362399716392 not running, aborting
{code}

this failure seems to be related to HBASE-7299. I've attached 94 patch here 
that seems to work.

 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: New Feature
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.94.6

 Attachments: 7360-v1.patch, HBASE-7299-0.94.patch, 
 HBASE-7360-jiras.txt, HBASE-7360-v0.patch


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6571 Generic multi-thread/cross-process error handling framework
  * HBASE-6353 Snapshots shell
  * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
  * HBASE-6836 Offline snapshots
  * HBASE-6865 Snapshot File Cleaners
  * HBASE-6777 Snapshot Restore Interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6230 Clone/Restore Snapshots
  * HBASE-6802 Export Snapshot
  * HBASE-7240 Cleanup old snapshots on start
  * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
  * HBASE-7367 Snapshot coprocessor and ACL security
  * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-7418 HFileLink flaky test
  * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing 
 test category annotation and failing TestCheckTestClasses
  * HBASE-7206 ForeignException framework v2 (simplifies and replaces 
 HBASE-6571)
  * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
 (EnvironmentEdge issue)
  * HBASE-7436 Improve stack trace info dumped by 
 ForeignExceptionSnare#rethrowException
  * HBASE-7339 Splitting an HFileLink causes region servers to go down.
  * HBASE-7439 HFileLink should not use the configuration from the Filesystem
  * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
 #receive(FE)
  * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
 for merge
  * HBASE-7207 Consolidate snapshot related classes into fewer packages
  * HBASE-7400 ExportSnapshot mapper closes the FileSystem
  * HBASE-7352 clone operation from HBaseAdmin can hang forever
  * HBASE-7294 Check for snapshot file cleaners on start
  * HBASE-7354 Snapshot Info/Debug Tool
  * HBASE-7423 HFileArchiver should not use the configuration from the 
 Filesystem
  * HBASE-7453 HBASE-7423 snapshot followup
  * HBASE-7212 Globally Barriered Procedure Mechanism
  * HBASE-6864 Online snapshots scaffolding
  * HBASE-7321 Simple Flush Snapshot
  * HBASE-7496 TestZKProcedure fails interrmittently.
  * HBASE-7484 Fix Restore with schema changes
  * HBASE-7419 revisit hfilelink file name format
  * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
 directories
  * HBASE-7214 CleanerChore logs too much, so much so it obscures all else 
 that is going on
  * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
 fails sometimes.
  * HBASE-7480 Explicit message for not allowed snapshot on meta tables
  * HBASE-7537 .regioninfo not created by createHRegion()
  * HBASE-7535 Fix restore reference files
  * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
 FileLink.open method, and remove unnecessary import packages.
  * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
 .META. table
  * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
  * HBASE-7547 Fix findbugs warnings in snapshot classes
  * HBASE-7548 Fix javadoc warnings in snapshot classes
  * HBASE-7538 Improve snapshot related error and exception messages
  * HBASE-7536 Add test that confirms that multiple concurrent snapshot 
 requests are rejected
  * HBASE-7583 Fixes and cleanups
  * HBASE-7604 Remove duplicated code from HFileLink
  * HBASE-7616 NPE in ZKProcedure.nodeCreated
  * HBASE-7625 Remove

[jira] [Commented] (HBASE-7808) Refactor Store to use HRegionFileSystem

2013-03-04 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592207#comment-13592207
 ] 

Hadoop QA commented on HBASE-7808:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12571875/HBASE-7808-v4.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 47 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
  org.apache.hadoop.hbase.TestZooKeeper

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4650//console

This message is automatically generated.

 Refactor Store to use HRegionFileSystem
 ---

 Key: HBASE-7808
 URL: https://issues.apache.org/jira/browse/HBASE-7808
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: HBASE-7808-v0.patch, HBASE-7808-v1.patch, 
 HBASE-7808-v2.patch, HBASE-7808-v3.patch, HBASE-7808-v4.patch


 Move HStore fs related code inside the HRegionFileSystem. 
 File creation and retrival should be handled there, and the use will be 
 something like.
 {code}
 HRegionFileSystem fs = new HRegionFileSystem
 file = fs.createFile()
 fs.commitFile(family, file);
 for (storeFile: fs.getStoreFiles(family))
...
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7988) Confusing log messages in assignment

nkeywal created HBASE-7988:
--

 Summary: Confusing log messages in assignment
 Key: HBASE-7988
 URL: https://issues.apache.org/jira/browse/HBASE-7988
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 0.98.0
Reporter: nkeywal
Priority: Minor


This appears in 7840, where we kill one server with 10 regions out of two 
servers.

I don't know why we explicitly test against 1 server here.
{code}
int servers = bulkPlan.size();
if (servers == 1 || (regions  bulkAssignThresholdRegions
 servers  bulkAssignThresholdServers)) {
{code}

At the end, the logs says once we're using bulk, once we're not:
2013-03-04 15:31:19,339 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Not use bulk assigning since we are assigning only 10 region(s) to 1 server(s)
2013-03-04 15:31:19,339 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) to box1,60020,1362407279158


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7968) Packaging of Trunk and 0.95 does not create the dependent jars in the lib folder


[ 
https://issues.apache.org/jira/browse/HBASE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592264#comment-13592264
 ] 

ramkrishna.s.vasudevan commented on HBASE-7968:
---

Ok N.  Thanks for committing it.

 Packaging of Trunk and 0.95 does not create the dependent jars in the lib 
 folder
 

 Key: HBASE-7968
 URL: https://issues.apache.org/jira/browse/HBASE-7968
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0, 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: nkeywal
Priority: Critical
 Fix For: 0.95.0, 0.98.0

 Attachments: 7968.v1.patch


 After recent changes to trunk and 0.95 branch when i try to build and 
 package, i do not find the dependent jars in the lib folder.
 Prior to the changes, it was working fine.
 Am not a maven expert.  Will try to see what is going wrong here. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7962) Native folder not available under lib folder in the HBase trunk deployment


[ 
https://issues.apache.org/jira/browse/HBASE-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592270#comment-13592270
 ] 

ramkrishna.s.vasudevan commented on HBASE-7962:
---

Ok Andy.  Will do that.  Anyway was thinking there will be a native folder 
under lib.  

 Native folder not available under lib folder in the HBase trunk deployment
 --

 Key: HBASE-7962
 URL: https://issues.apache.org/jira/browse/HBASE-7962
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.95.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan

 I was trying to install Snappy.  Found this link.
 http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with#.US7JNzDX_D4.
 As per this link there is a native folder under lib directory in the HBase 
 installation path and it is available in 0.94.
 But when i install HBase trunk i could not see one
 If am not finding this in the correct place, pls invalidate the issue. Will 
 try figuring out what could be the problem

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592287#comment-13592287
 ] 

Ted Yu commented on HBASE-7360:
---

@Matteo:
Mind opening a JIRA for backporting HBASE-7299 ?

Thanks

 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: New Feature
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.94.6

 Attachments: 7360-v1.patch, HBASE-7299-0.94.patch, 
 HBASE-7360-jiras.txt, HBASE-7360-v0.patch


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6571 Generic multi-thread/cross-process error handling framework
  * HBASE-6353 Snapshots shell
  * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
  * HBASE-6836 Offline snapshots
  * HBASE-6865 Snapshot File Cleaners
  * HBASE-6777 Snapshot Restore Interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6230 Clone/Restore Snapshots
  * HBASE-6802 Export Snapshot
  * HBASE-7240 Cleanup old snapshots on start
  * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
  * HBASE-7367 Snapshot coprocessor and ACL security
  * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-7418 HFileLink flaky test
  * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing 
 test category annotation and failing TestCheckTestClasses
  * HBASE-7206 ForeignException framework v2 (simplifies and replaces 
 HBASE-6571)
  * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
 (EnvironmentEdge issue)
  * HBASE-7436 Improve stack trace info dumped by 
 ForeignExceptionSnare#rethrowException
  * HBASE-7339 Splitting an HFileLink causes region servers to go down.
  * HBASE-7439 HFileLink should not use the configuration from the Filesystem
  * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
 #receive(FE)
  * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
 for merge
  * HBASE-7207 Consolidate snapshot related classes into fewer packages
  * HBASE-7400 ExportSnapshot mapper closes the FileSystem
  * HBASE-7352 clone operation from HBaseAdmin can hang forever
  * HBASE-7294 Check for snapshot file cleaners on start
  * HBASE-7354 Snapshot Info/Debug Tool
  * HBASE-7423 HFileArchiver should not use the configuration from the 
 Filesystem
  * HBASE-7453 HBASE-7423 snapshot followup
  * HBASE-7212 Globally Barriered Procedure Mechanism
  * HBASE-6864 Online snapshots scaffolding
  * HBASE-7321 Simple Flush Snapshot
  * HBASE-7496 TestZKProcedure fails interrmittently.
  * HBASE-7484 Fix Restore with schema changes
  * HBASE-7419 revisit hfilelink file name format
  * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
 directories
  * HBASE-7214 CleanerChore logs too much, so much so it obscures all else 
 that is going on
  * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
 fails sometimes.
  * HBASE-7480 Explicit message for not allowed snapshot on meta tables
  * HBASE-7537 .regioninfo not created by createHRegion()
  * HBASE-7535 Fix restore reference files
  * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
 FileLink.open method, and remove unnecessary import packages.
  * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
 .META. table
  * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
  * HBASE-7547 Fix findbugs warnings in snapshot classes
  * HBASE-7548 Fix javadoc warnings in snapshot classes
  * HBASE-7538 Improve snapshot related error and exception messages
  * HBASE-7536 Add test that confirms that multiple concurrent snapshot 
 requests are rejected
  * HBASE-7583 Fixes and cleanups
  * HBASE-7604 Remove duplicated code from HFileLink
  * HBASE-7616 NPE in ZKProcedure.nodeCreated
  * HBASE-7625 Remove duplicated logFSTree() from 
 TestRestoreFlushSnapshotFromClient
  * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
  * HBASE-7622 Add table descriptor verification after snapshot restore
  * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
 previous snapshot fails in per region task
  * HBASE-7666 More logging improvements in online snapshots code
  *

[jira] [Assigned] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk


 [ 
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Kinley reassigned HBASE-7482:
---

Assignee: James Kinley  (was: Gary Helmling)

 Port HBASE-7442 HBase remote CopyTable not working when security enabled to 
 trunk
 -

 Key: HBASE-7482
 URL: https://issues.apache.org/jira/browse/HBASE-7482
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: James Kinley
Priority: Critical
 Fix For: 0.95.0


 Excerpt about the choice of solution from :
 The first option was actually quite messy to implement. {{clusterId}} and 
 {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by 
 *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass 
 the different cluster {{confs}} up through *{{HConnectionManager}}* and 
 *{{HBaseRPC}}* in order to override the clusterId in 
 *{{SecureClient#SecureConnection}}*.
 I've gone with the second option of creating and caching different 
 *{{SecureClients}}* for the local and remote clusters in 
 *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default 
 *{{SocketFactory}}*. I think this is a cleaner solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7989) Client with a cache info on a dead server will wait for 20s before trying another one.

nkeywal created HBASE-7989:
--

 Summary: Client with a cache info on a dead server will wait for 
20s before trying another one.
 Key: HBASE-7989
 URL: https://issues.apache.org/jira/browse/HBASE-7989
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 0.95.0, 0.98.0
Reporter: nkeywal


Scenario is:
- fetch the cache in the client
- a server dies
- try to use a region that is on the dead server

This will lead to a 20 second connect timeout. We don't have this in unit test 
because we have this only is the remote box does not answer. In the unit tests 
we have immediately a connection refused from the OS.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7686) TestSplitTransactionOnCluster fails occasionally in trunk builds


 [ 
https://issues.apache.org/jira/browse/HBASE-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7686:
---

Status: Patch Available  (was: Reopened)

 TestSplitTransactionOnCluster fails occasionally in trunk builds
 

 Key: HBASE-7686
 URL: https://issues.apache.org/jira/browse/HBASE-7686
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Priority: Critical
 Fix For: 0.95.0

 Attachments: HBASE-7686-v0.patch


 From trunk build #3808:
 {code} 
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
   
 testMasterRestartWhenSplittingIsPartial(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
   
 testExistingZnodeBlocksSplitAndWeRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
 {code}
 From HBase-TRUNK-on-Hadoop-2.0.0 #378 :
 {code}
 testShutdownSimpleFixup(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  Region not moved off .META. server
   
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7686) TestSplitTransactionOnCluster fails occasionally in trunk builds


 [ 
https://issues.apache.org/jira/browse/HBASE-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7686:
---

Attachment: HBASE-7686-v0.patch

I've looked a bit into this since is showing up in my jenkins.

There're a couple of strange things like the MockedSplitTransaction() with a 
null splitRow and no call to prepare() 
but trying to look into the failures seems that most of them are caused by 
regions not available...

v0 patch, tries to look for a splittable region (isAvailable()  
!hasReferences()) and adds the missing splitRow/prepare() just to be correct 
even if they are not really needed since the SplitTransaction is mocked.

 TestSplitTransactionOnCluster fails occasionally in trunk builds
 

 Key: HBASE-7686
 URL: https://issues.apache.org/jira/browse/HBASE-7686
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Priority: Critical
 Fix For: 0.95.0

 Attachments: HBASE-7686-v0.patch


 From trunk build #3808:
 {code} 
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
   
 testMasterRestartWhenSplittingIsPartial(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
   
 testExistingZnodeBlocksSplitAndWeRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
 {code}
 From HBase-TRUNK-on-Hadoop-2.0.0 #378 :
 {code}
 testShutdownSimpleFixup(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  Region not moved off .META. server
   
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk

2013-03-04 Thread Gary Helmling (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592370#comment-13592370
 ] 

Gary Helmling commented on HBASE-7482:
--

James, I have a patch for this I can post -- it's just the TableMapReduceUtil 
change from HBASE-7442.  But I don't have 2 secure clusters setup for easy 
testing.  Do you have test clusters where you can confirm the fix?

 Port HBASE-7442 HBase remote CopyTable not working when security enabled to 
 trunk
 -

 Key: HBASE-7482
 URL: https://issues.apache.org/jira/browse/HBASE-7482
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: James Kinley
Priority: Critical
 Fix For: 0.95.0


 Excerpt about the choice of solution from :
 The first option was actually quite messy to implement. {{clusterId}} and 
 {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by 
 *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass 
 the different cluster {{confs}} up through *{{HConnectionManager}}* and 
 *{{HBaseRPC}}* in order to override the clusterId in 
 *{{SecureClient#SecureConnection}}*.
 I've gone with the second option of creating and caching different 
 *{{SecureClients}}* for the local and remote clusters in 
 *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default 
 *{{SocketFactory}}*. I think this is a cleaner solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization

[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592385#comment-13592385
]

Nick Dimiduk commented on HBASE-7692:
-

bq. This is definitely not just client side. Phoenix would use this in the
innermost loops of coprocessor region scans and custom filter evaluation.

I believe maintainer of RS components tend to view consumption of the
coprocessor and filter interfaces as client concerns. What I believe [~stack]
means here is that the RS implementation would continue to view its whole world
as byte[], not make use of this library.

bq. I'm not completely against the RowKey suffix in the name, but if we want
to change it how about Codec or Type instead?

This segues nicely into the related conversation, which is that I believe HBase
should provide its own primitive types, just as the Management Systems of RDBMS
world provide. Again, nothing that needs baked into the internals of the
system, but some set of types that are language-neutral and clearly defined. I
think this is the best way to establish interoperability between tools. HBase
needs to think more actively about the world outside of Java.

Add utility class to generate ordered byte[] serialization
--

Key: HBASE-7692
URL: https://issues.apache.org/jira/browse/HBASE-7692
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 0.95.0

Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch

The current Bytes utility class works, but produces output that does not
maintain the native sort ordering of the input value. This results in, for
example, a negative value that does not necessarily sort before a positive
value. HBase should provide a canonical implementation of such a
serialization format so that third-parties can reliably build on top of
HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
HIVE-2903 that is compatible with similar features in Pig.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount

2013-03-04 Thread Tianying Chang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592394#comment-13592394
 ] 

Tianying Chang commented on HBASE-7818:
---

The unit test on my dev VM box always exit with unable to create new native 
thread at the end of the unit test suite, even when my patch is not there. I 
am trying to fix my dev VM to run the unit test, at the same time I am trying 
to find another machine that I can run the full unit test on.

 add region level metrics readReqeustCount and writeRequestCount 
 

 Key: HBASE-7818
 URL: https://issues.apache.org/jira/browse/HBASE-7818
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Affects Versions: 0.94.4
Reporter: Tianying Chang
Assignee: Tianying Chang
Priority: Minor
 Fix For: 0.94.7

 Attachments: HBASE-7818_1.patch, HBASE-7818_2.patch, HBASE-7818.patch


 Request rate at region server level can help identify the hot region server. 
 But it will be good if we can further identify the hot regions on that region 
 server. That way, we can easily find out unbalanced regions problem. 
 Currently, readRequestCount and writeReqeustCount per region is exposed at 
 webUI. It will be more useful to expose it through hadoop metrics framework 
 and/or JMX, so that people can see the history when the region is hot.
 I am exposing the existing readRequestCount/writeRequestCount into the 
 dynamic region level metrics framework. I am not changing/exposing it as rate 
 because our openTSDB is taking the raw data of read/write count, and apply 
 rate function to display the rate already. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94

Ted Yu created HBASE-7990:
-

 Summary: Backport HBASE-5837 'hbase shell deleteall to .META. 
allows insertion of malformed rowkey' to 0.94
 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu


This was brought up in discussion entitled 'HRegionInfo was null or empty in 
Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk


[ 
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592401#comment-13592401
 ] 

James Kinley commented on HBASE-7482:
-

Hi Gary. Yes, I have 2 secure clusters which I setup for HBASE-7442. I've been 
testing a patch this afternoon and had to make an additional change to 
HConnectionImplementation to get it to work. I'll attach it for review.

 Port HBASE-7442 HBase remote CopyTable not working when security enabled to 
 trunk
 -

 Key: HBASE-7482
 URL: https://issues.apache.org/jira/browse/HBASE-7482
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: James Kinley
Priority: Critical
 Fix For: 0.95.0


 Excerpt about the choice of solution from :
 The first option was actually quite messy to implement. {{clusterId}} and 
 {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by 
 *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass 
 the different cluster {{confs}} up through *{{HConnectionManager}}* and 
 *{{HBaseRPC}}* in order to override the clusterId in 
 *{{SecureClient#SecureConnection}}*.
 I've gone with the second option of creating and caching different 
 *{{SecureClients}}* for the local and remote clusters in 
 *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default 
 *{{SocketFactory}}*. I think this is a cleaner solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7991) Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could cause server abort when splitting' to 0.94

Ted Yu created HBASE-7991:
-

 Summary: Backport HBASE-6479 'HFileReaderV1 caching the same 
parent META block could cause server abort when splitting' to 0.94
 Key: HBASE-7991
 URL: https://issues.apache.org/jira/browse/HBASE-7991
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu


Bryan Beaudreault was hit by this issue in his 0.94.2 clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7941) Provide client API with support for primitive types


[ 
https://issues.apache.org/jira/browse/HBASE-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592419#comment-13592419
 ] 

Nick Dimiduk commented on HBASE-7941:
-

Why do you suggest adding an additional layer? Why not bake it into the real 
API classes we expect users to consume? Is this because client classes are 
consumed inside of the hbase-server module?

 Provide client API with support for primitive types
 ---

 Key: HBASE-7941
 URL: https://issues.apache.org/jira/browse/HBASE-7941
 Project: HBase
  Issue Type: Improvement
  Components: Client, Usability
Reporter: Nick Dimiduk

 Work is underway to provide a widely acceptable serialization format for 
 primitive and complex types (HBASE-7221, HBASE-7692). With this completed, 
 those serialization conveniences should be pushed up to users of the Client 
 API by way of additional method signatures on Operation implementations (Get, 
 Put, Delete, Scan, c.).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7896) make rename_table working in 92/94

2013-03-04 Thread Tianying Chang (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592421#comment-13592421
]

Tianying Chang commented on HBASE-7896:
---

While I am working on addressing the comments, I start to think maybe it is
better to make rename_table part of the functions of HBase Admin in Java code.
The reason is that being ruby and no unit test associated, we have to update
this script whenever there is API break change in the HBase java code(and we
don't even notice it until we run the script again). I fixed the script to work
for 92 in our production, and then found have to fix more things when we start
to use 94. It is a pain to keep updating this ruby script. I think it is
probably better idea to make it part of the HBase admin and with unit test.

Anyone think it is better to stay in Ruby or better to integrate as part of
HBase Admin? If people agree that Java version is better, I can start to work
on java version.

make rename_table working in 92/94
--

Key: HBASE-7896
URL: https://issues.apache.org/jira/browse/HBASE-7896
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.92.2, 0.94.5
Reporter: Tianying Chang
Assignee: Tianying Chang
Fix For: 0.92.3, 0.94.7

Attachments: rename_table.rb

The rename_table function is very useful for our customers. However,
rename_table.rb does not work for 92/94. It has several bugs. It will be
useful to fix them so that users can solve their problems.

[jira] [Commented] (HBASE-7982) TestReplicationQueueFailover* runs for a minute, spews 3/4million lines complaining 'Filesystem closed', has an NPE, and still passes?


[ 
https://issues.apache.org/jira/browse/HBASE-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592423#comment-13592423
 ] 

nkeywal commented on HBASE-7982:


bq. nkeywal Which part boss?
Surefire has multiple internal buffer, surefire-800 removes one of them, so may 
be it will manage the error properly.

 TestReplicationQueueFailover* runs for a minute, spews 3/4million lines 
 complaining 'Filesystem closed', has an NPE, and still passes?
 --

 Key: HBASE-7982
 URL: https://issues.apache.org/jira/browse/HBASE-7982
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: stack
Priority: Blocker
 Attachments: hbase-7982-NPE.patch


 I was trying to look at why the odd time Hudson OOMEs trying to make a report 
 on 0.95 build #4 https://builds.apache.org/job/hbase-0.95/4/console:
 {code}
 ERROR: Failed to archive test reports
 hudson.util.IOException2: remote file operation failed: 
 /home/jenkins/jenkins-slave/workspace/hbase-0.95 at 
 hudson.remoting.Channel@151a4e3e:ubuntu3
   at hudson.FilePath.act(FilePath.java:861)
   at hudson.FilePath.act(FilePath.java:838)
   at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
   at 
 ...
 Caused by: java.lang.OutOfMemoryError: Java heap space
   at java.nio.HeapCharBuffer.init(HeapCharBuffer.java:57)
   at java.nio.CharBuffer.allocate(CharBuffer.java:329)
   at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:792)
   at java.nio.charset.Charset.decode(Charset.java:791)
   at hudson.tasks.junit.SuiteResult.init(SuiteResult.java:215)
 ...
 {code}
 We are trying to allocate a big buffer and failing.
 Looking at reports being generated, we have quite a few that are  10MB in 
 size:
 {code}
 durruti:0.95 stack$ find hbase-* -type f -size +1k -exec ls -la {} \;
 -rw-r--r--@ 1 stack  staff  11126492 Feb 27 06:14 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.backup.TestHFileArchiving-output.txt
 -rw-r--r--@ 1 stack  staff  13296009 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestFromClientSide3-output.txt
 -rw-r--r--@ 1 stack  staff  10541898 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
 -rw-r--r--@ 1 stack  staff  25344601 Feb 27 05:51 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient-output.txt
 -rw-r--r--@ 1 stack  staff  17966969 Feb 27 06:12 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction-output.txt
 -rw-r--r--@ 1 stack  staff  17699068 Feb 27 06:09 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
 -rw-r--r--@ 1 stack  staff  17701832 Feb 27 06:07 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplitCompressed-output.txt
 -rw-r--r--@ 1 stack  staff  717853709 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailover-output.txt
 -rw-r--r--@ 1 stack  staff  563616793 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed-output.txt
 {code}
 ... with TestReplicationQueueFailover* being order of magnitude bigger than 
 the others.
 Looking in the test I see both spewing between 800 and 900 thousand lines in 
 about a minute.  Here is their fixation:
 {code}
 8908998 2013-02-27 06:17:48,176 ERROR 
 [RegionServer:1;hemera.apache.org,35712,1361945801803.logSyncer] 
 wal.FSHLog$LogSyncer(1012): Error while syncing, requesting close of hlog.
 8908999 java.io.IOException: Filesystem closed
 8909000 ,...at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:319)
 8909001 ,...at org.apache.hadoop.hdfs.DFSClient.access$1200(DFSClient.java:78)
 8909002 ,...at 
 org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.sync(DFSClient.java:3843)
 8909003 ,...at 
 org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:97)
 8909004 ,...at 
 org.apache.hadoop.io.SequenceFile$Writer.syncFs(SequenceFile.java:999)
 8909005 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:248)
 8909006 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.syncer(FSHLog.java:1120)
 8909007 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.syncer(FSHLog.java:1058)
 8909008 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:1228)
 8909009 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog$LogSyncer.run(FSHLog.java:1010)
 8909010 ,...at java.lang.Thread.run(Thread.java:722)

[jira] [Commented] (HBASE-7971) mvn clean compile on trunk and 0.95 does not produce target/cached_classpath.txt


[ 
https://issues.apache.org/jira/browse/HBASE-7971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592427#comment-13592427
 ] 

Nick Dimiduk commented on HBASE-7971:
-

bq. You are generating hbase/target/cached_classpath.txt, but referring to 
hbase/hbase-it/target/cached_classpath.txt.

Nope, it's generating and consuming hbase/hbase-it/target/cached_classpath.txt. 
The modified pom is not the top-level one, but for hbase-it.

bq. One other thing is that, this cached classpath won't contain test 
artifacts, because it is invoked from compile.

What invokes bin/hbase and also depends on the test jars? git-grep shows this 
file being produced in hbase-it.pom and consumed only in the launch scripts, 
bin/hbase and bin/hbase.cmd.

This looks discouraging:

{noformat}
$ git grep -n bin\/hbase
...
hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java:133:   
* CommandProvider to manage the service using bin/hbase-* scripts
hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java:150:
  return String.format(%s/bin/hbase-daemon.sh %s %s %s, getHBaseHome(), 
getConfig(),
...
hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java:1596:
System.err.println( $ bin/hbase  + this.getClass().getName()
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java:38:
 * code${HBASE_HOME}/bin/hbase ./bin/hbase 
org.apache.hadoop.hbase.OOMERegionServer start/code.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java:273:
System.err.printf(Usage: bin/hbase %s [options]\n, getClass().getName());
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java:291:
System.err.println( $ ./bin/hbase 
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEval
hbase-server/src/test/java/org/apache/hadoop/hbase/rest/PerformanceEvaluation.java:1148:
System.err.println( $ bin/hbase  + this.getClass().getName()
hbase-server/src/test/java/org/apache/hadoop/hbase/util/ProcessBasedLocalHBaseCluster.java:114:
hbaseDaemonScript = hbaseHome + /bin/hbase-daemon.sh;
{noformat}

 mvn clean compile on trunk and 0.95 does not produce 
 target/cached_classpath.txt
 

 Key: HBASE-7971
 URL: https://issues.apache.org/jira/browse/HBASE-7971
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.95.0, 0.98.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
 Attachments: 
 0001-HBASE-7971-fix-cached_classpath.txt-dev-sandbox.patch, 
 0001-HBASE-7971-fix-cached_classpath.txt-dev-sandbox.patch


 The usual workflow of {{mvn clean compile}} followed by {{./bin/hbase foo}} 
 no longer works. It looks like it was broken by HBASE-7637, switching the 
 phase in hbase-it/pom.xml from {{compile}} to {{test}}. Before I propose the 
 obvious patch, a couple questions:
 [~nkeywal]: why is this done in hbase-it instead of anywhere else, such as 
 the top-level pom?
 [~eclark]: why does this execution in the compile phase break hadoop-2 
 profile while running it in test does not?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7686) TestSplitTransactionOnCluster fails occasionally in trunk builds

2013-03-04 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592430#comment-13592430
 ] 

Hadoop QA commented on HBASE-7686:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12571897/HBASE-7686-v0.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/4651//console

This message is automatically generated.

 TestSplitTransactionOnCluster fails occasionally in trunk builds
 

 Key: HBASE-7686
 URL: https://issues.apache.org/jira/browse/HBASE-7686
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Priority: Critical
 Fix For: 0.95.0

 Attachments: HBASE-7686-v0.patch


 From trunk build #3808:
 {code} 
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
   
 testMasterRestartWhenSplittingIsPartial(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
   
 testExistingZnodeBlocksSplitAndWeRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 30 milliseconds
 {code}
 From HBase-TRUNK-on-Hadoop-2.0.0 #378 :
 {code}
 testShutdownSimpleFixup(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  Region not moved off .META. server
   
 testShouldFailSplitIfZNodeDoesNotExistDueToPrevRollBack(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster):
  test timed out after 2 milliseconds
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592434#comment-13592434
 ] 

Jeffrey Zhong commented on HBASE-7824:
--

The reason to add those previous dead non-meta region servers into deadServers 
is to let the new master instance start as a failover such as the following 
code in AssignmentManager. In addition, we don't want AM assign those regions 
before log splitting work complete that's why let AM skip them inside function 
processDeadServersAndRegionsInTransition but handle them in SSH.

{code}
if (!this.serverManager.getDeadServers().isEmpty()) {
  this.failover = true;
}
{code} 

Since those failed servers will be processed by SSH so their regions should be 
online and I did see test log message Finished processing of shutdown. 

There are couple of regions aren't assigned for some reason and I need to dig 
more in the test case and keep you updated.

Thanks,
-Jeffrey



 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6737) NullPointerException at regionserver.wal.SequenceFileLogWriter.append


 [ 
https://issues.apache.org/jira/browse/HBASE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-6737:


Assignee: (was: Sergey Shelukhin)

 NullPointerException at regionserver.wal.SequenceFileLogWriter.append
 -

 Key: HBASE-6737
 URL: https://issues.apache.org/jira/browse/HBASE-6737
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Critical

 Real cluster, scenario in HBASE-5843.
 There are two exceptions, I create a single JIRA with both of them.
 2012-09-04 18:14:49,264 FATAL 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: WriterThread-1 Got 
 while writing log entry to log
 java.io.IOException: java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.append(SequenceFileLogWriter.java:229)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:949)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:919)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:891)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.io.SequenceFile$Writer.checkAndWriteSync(SequenceFile.java:1026)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1068)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1035)
   at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.append(SequenceFileLogWriter.java:226)
   ... 3 more
 2012-09-04 18:15:52,546 ERROR 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Error in log splitting 
 write thread
 java.lang.reflect.UndeclaredThrowableException
   at $Proxy7.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:875)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:513)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:768)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getRegionSplitEditsPath(HLogSplitter.java:559)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.createWAP(HLogSplitter.java:974)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.access$800(HLogSplitter.java:82)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$OutputSink.getWriterAndPath(HLogSplitter.java:1309)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:942)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:919)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:891)
 Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:261)
   ... 11 more
 Caused by: java.io.IOException: Call to BOX1/192.168.15.5:9000 failed on 
 local exception: java.nio.channels.ClosedByInterruptException
   at org.apache.hadoop.ipc.Client.wrapException(Client.java:1107)
   at org.apache.hadoop.ipc.Client.call(Client.java:1075)
   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
   at $Proxy7.getFileInfo(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
   at $Proxy7.getFileInfo(Unknown Source)
   ... 15 more
 Caused by: java.nio.channels.ClosedByInterruptException
   at 
 java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
   at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
   at 
 org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
   at 
 org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
   at 
 org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
   at 
 org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
   at

[jira] [Commented] (HBASE-7589) Hudson findbugs check produces bogus results, and when/if they aren't it's hard to figure out what the bugs are


[ 
https://issues.apache.org/jira/browse/HBASE-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592455#comment-13592455
 ] 

Sergey Shelukhin commented on HBASE-7589:
-

Yeah...  What I meant was the cases where findbugs are reported but diff of the 
findbugs report shows none... don't have any handy unfortunately.
In fact, the diffing of findbugs reports is a problems as such (maybe there are 
really findbugs in the above case but they go undetected due to incorrect 
diffing; or incorrect reports?). The script should produce a list of new 
warnings, and then output the list length into the report, or something like  
that.

I will unassign myself since I won't have time for this for some time...

 Hudson findbugs check produces bogus results, and when/if they aren't it's 
 hard to figure out what the bugs are
 ---

 Key: HBASE-7589
 URL: https://issues.apache.org/jira/browse/HBASE-7589
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7589) Jenkins findbugs check produces bogus results, and when/if they aren't it's hard to figure out what the bugs are


 [ 
https://issues.apache.org/jira/browse/HBASE-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7589:


Assignee: (was: Sergey Shelukhin)

 Jenkins findbugs check produces bogus results, and when/if they aren't it's 
 hard to figure out what the bugs are
 

 Key: HBASE-7589
 URL: https://issues.apache.org/jira/browse/HBASE-7589
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Sergey Shelukhin



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7589) Jenkins findbugs check produces bogus results, and when/if they aren't it's hard to figure out what the bugs are


 [ 
https://issues.apache.org/jira/browse/HBASE-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7589:


Summary: Jenkins findbugs check produces bogus results, and when/if they 
aren't it's hard to figure out what the bugs are  (was: Hudson findbugs check 
produces bogus results, and when/if they aren't it's hard to figure out what 
the bugs are)

 Jenkins findbugs check produces bogus results, and when/if they aren't it's 
 hard to figure out what the bugs are
 

 Key: HBASE-7589
 URL: https://issues.apache.org/jira/browse/HBASE-7589
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592457#comment-13592457
 ] 

Lars Hofhansl commented on HBASE-7824:
--

I would like to roll 0.94.6 soon.
Should we revert this for 0.94.6 and put it back up for 0.94.7?


 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7679) implement store file management for stripe compactions


[ 
https://issues.apache.org/jira/browse/HBASE-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592460#comment-13592460
 ] 

Sergey Shelukhin commented on HBASE-7679:
-

Any more feedback?

 implement store file management for stripe compactions
 --

 Key: HBASE-7679
 URL: https://issues.apache.org/jira/browse/HBASE-7679
 Project: HBase
  Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HBASE-7667-and-7603-v0-incomplete.patch, 
 HBASE-7667-and-7603-v0-incomplete.patch, HBASE-7667-and-7603-v1.patch, 
 HBASE-7667-and-7603-v1.patch, HBASE-7667-v1.patch, HBASE-7667-v1.patch, 
 HBASE-7667-v2.patch, HBASE-7667-v2.patch, HBASE-7667-v3.patch, 
 HBASE-7679-v4.patch, HBASE-7679-v5.patch, HBASE-7679-v6.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592461#comment-13592461
 ] 

Lars Hofhansl commented on HBASE-7360:
--

I think we can do that as an addendum here, since this appears to be related to 
the snapshot checkin.

 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: New Feature
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.94.6

 Attachments: 7360-v1.patch, HBASE-7299-0.94.patch, 
 HBASE-7360-jiras.txt, HBASE-7360-v0.patch


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6571 Generic multi-thread/cross-process error handling framework
  * HBASE-6353 Snapshots shell
  * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
  * HBASE-6836 Offline snapshots
  * HBASE-6865 Snapshot File Cleaners
  * HBASE-6777 Snapshot Restore Interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6230 Clone/Restore Snapshots
  * HBASE-6802 Export Snapshot
  * HBASE-7240 Cleanup old snapshots on start
  * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
  * HBASE-7367 Snapshot coprocessor and ACL security
  * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-7418 HFileLink flaky test
  * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing 
 test category annotation and failing TestCheckTestClasses
  * HBASE-7206 ForeignException framework v2 (simplifies and replaces 
 HBASE-6571)
  * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
 (EnvironmentEdge issue)
  * HBASE-7436 Improve stack trace info dumped by 
 ForeignExceptionSnare#rethrowException
  * HBASE-7339 Splitting an HFileLink causes region servers to go down.
  * HBASE-7439 HFileLink should not use the configuration from the Filesystem
  * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
 #receive(FE)
  * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
 for merge
  * HBASE-7207 Consolidate snapshot related classes into fewer packages
  * HBASE-7400 ExportSnapshot mapper closes the FileSystem
  * HBASE-7352 clone operation from HBaseAdmin can hang forever
  * HBASE-7294 Check for snapshot file cleaners on start
  * HBASE-7354 Snapshot Info/Debug Tool
  * HBASE-7423 HFileArchiver should not use the configuration from the 
 Filesystem
  * HBASE-7453 HBASE-7423 snapshot followup
  * HBASE-7212 Globally Barriered Procedure Mechanism
  * HBASE-6864 Online snapshots scaffolding
  * HBASE-7321 Simple Flush Snapshot
  * HBASE-7496 TestZKProcedure fails interrmittently.
  * HBASE-7484 Fix Restore with schema changes
  * HBASE-7419 revisit hfilelink file name format
  * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
 directories
  * HBASE-7214 CleanerChore logs too much, so much so it obscures all else 
 that is going on
  * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
 fails sometimes.
  * HBASE-7480 Explicit message for not allowed snapshot on meta tables
  * HBASE-7537 .regioninfo not created by createHRegion()
  * HBASE-7535 Fix restore reference files
  * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
 FileLink.open method, and remove unnecessary import packages.
  * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
 .META. table
  * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
  * HBASE-7547 Fix findbugs warnings in snapshot classes
  * HBASE-7548 Fix javadoc warnings in snapshot classes
  * HBASE-7538 Improve snapshot related error and exception messages
  * HBASE-7536 Add test that confirms that multiple concurrent snapshot 
 requests are rejected
  * HBASE-7583 Fixes and cleanups
  * HBASE-7604 Remove duplicated code from HFileLink
  * HBASE-7616 NPE in ZKProcedure.nodeCreated
  * HBASE-7625 Remove duplicated logFSTree() from 
 TestRestoreFlushSnapshotFromClient
  * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
  * HBASE-7622 Add table descriptor verification after snapshot restore
  * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
 previous snapshot fails in per region task
  * HBASE-7666 More

[jira] [Updated] (HBASE-7841) Parallelize offline snapshot in DisabledTableSnapshotHandler


 [ 
https://issues.apache.org/jira/browse/HBASE-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7841:
--

Attachment: 7841.txt

Tentative patch.

Currently I reuse the EventType.C_M_SNAPSHOT_TABLE for the EventHandler


 Parallelize offline snapshot in DisabledTableSnapshotHandler
 

 Key: HBASE-7841
 URL: https://issues.apache.org/jira/browse/HBASE-7841
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu
 Attachments: 7841.txt


 In DisabledTableSnapshotHandler, there is TODO:
 {code}
   // TODO consider parallelizing these operations since they are independent. 
 Right now its just
   // easier to keep them serial though
   @Override
   public void snapshotRegions(ListPairHRegionInfo, ServerName 
 regionsAndLocations) throws IOException,
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7902) deletes may be removed during minor compaction, in non-standard compaction schemes


 [ 
https://issues.apache.org/jira/browse/HBASE-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7902:


  Resolution: Fixed
Release Note: committed to 0.95 and trunk last week
  Status: Resolved  (was: Patch Available)

 deletes may be removed during minor compaction,  in non-standard compaction 
 schemes
 ---

 Key: HBASE-7902
 URL: https://issues.apache.org/jira/browse/HBASE-7902
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HBASE-7902-v0.patch, HBASE-7902-v0-with-7843.patch, 
 HBASE-7902-v1-.patch, HBASE-7902-v1.patch


 Deletes are only removed during major compaction now. However, in presence of 
 file ordering, deletes can be removed during minor compaction too, as long as 
 there's no file that is not being compacted that is older than the files that 
 are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592466#comment-13592466
 ] 

Lars Hofhansl commented on HBASE-7360:
--

And it's test only anyway. Unless there're objection I'll commit Matteo's 
proposed patch (if you prefer, Ted, I'll file a separate ticket... Let me know).

 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: New Feature
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.94.6

 Attachments: 7360-v1.patch, HBASE-7299-0.94.patch, 
 HBASE-7360-jiras.txt, HBASE-7360-v0.patch


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 Take a Snapshot interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6571 Generic multi-thread/cross-process error handling framework
  * HBASE-6353 Snapshots shell
  * HBASE-7107 Snapshot References Utils (FileSystem Visitor)
  * HBASE-6836 Offline snapshots
  * HBASE-6865 Snapshot File Cleaners
  * HBASE-6777 Snapshot Restore Interface
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-6230 Clone/Restore Snapshots
  * HBASE-6802 Export Snapshot
  * HBASE-7240 Cleanup old snapshots on start
  * HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
  * HBASE-7367 Snapshot coprocessor and ACL security
  * HBASE-7311 Add snapshot information to hbase master webui
 ** This patch is significantly different because of the RPC changes between 
 0.94/0.96
  * HBASE-7418 HFileLink flaky test
  * HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing 
 test category annotation and failing TestCheckTestClasses
  * HBASE-7206 ForeignException framework v2 (simplifies and replaces 
 HBASE-6571)
  * HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests 
 (EnvironmentEdge issue)
  * HBASE-7436 Improve stack trace info dumped by 
 ForeignExceptionSnare#rethrowException
  * HBASE-7339 Splitting an HFileLink causes region servers to go down.
  * HBASE-7439 HFileLink should not use the configuration from the Filesystem
  * HBASE-7452 Change ForeignExceptionListener#receive(String, FE)  to only be 
 #receive(FE)
  * HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor 
 for merge
  * HBASE-7207 Consolidate snapshot related classes into fewer packages
  * HBASE-7400 ExportSnapshot mapper closes the FileSystem
  * HBASE-7352 clone operation from HBaseAdmin can hang forever
  * HBASE-7294 Check for snapshot file cleaners on start
  * HBASE-7354 Snapshot Info/Debug Tool
  * HBASE-7423 HFileArchiver should not use the configuration from the 
 Filesystem
  * HBASE-7453 HBASE-7423 snapshot followup
  * HBASE-7212 Globally Barriered Procedure Mechanism
  * HBASE-6864 Online snapshots scaffolding
  * HBASE-7321 Simple Flush Snapshot
  * HBASE-7496 TestZKProcedure fails interrmittently.
  * HBASE-7484 Fix Restore with schema changes
  * HBASE-7419 revisit hfilelink file name format
  * HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty 
 directories
  * HBASE-7214 CleanerChore logs too much, so much so it obscures all else 
 that is going on
  * HBASE-7523 Snapshot attempt with the name of a previously taken snapshot 
 fails sometimes.
  * HBASE-7480 Explicit message for not allowed snapshot on meta tables
  * HBASE-7537 .regioninfo not created by createHRegion()
  * HBASE-7535 Fix restore reference files
  * HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within 
 FileLink.open method, and remove unnecessary import packages.
  * HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in 
 .META. table
  * HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
  * HBASE-7547 Fix findbugs warnings in snapshot classes
  * HBASE-7548 Fix javadoc warnings in snapshot classes
  * HBASE-7538 Improve snapshot related error and exception messages
  * HBASE-7536 Add test that confirms that multiple concurrent snapshot 
 requests are rejected
  * HBASE-7583 Fixes and cleanups
  * HBASE-7604 Remove duplicated code from HFileLink
  * HBASE-7616 NPE in ZKProcedure.nodeCreated
  * HBASE-7625 Remove duplicated logFSTree() from 
 TestRestoreFlushSnapshotFromClient
  * HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
  * HBASE-7622 Add table descriptor verification after snapshot restore
  * HBASE-7651 RegionServerSnapshotManager fails with CancellationException if 
 previous

[jira] [Commented] (HBASE-7791) Compaction USER_PRIORITY is slightly broken


[ 
https://issues.apache.org/jira/browse/HBASE-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592468#comment-13592468
 ] 

Sergey Shelukhin commented on HBASE-7791:
-

Well, getSystemCompactionPriority or equivalent may be implemented separately 
by each compaction scheme, I wonder if we can trust all of them to ensure that.

 Compaction USER_PRIORITY is slightly broken
 ---

 Key: HBASE-7791
 URL: https://issues.apache.org/jira/browse/HBASE-7791
 Project: HBase
  Issue Type: Bug
  Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HBASE-7791-v0.patch


 The code to get compaction priority is as such:
 {code}   public int getCompactPriority(int priority) {
  // If this is a user-requested compaction, leave this at the highest 
 priority
  if(priority == Store.PRIORITY_USER) {
return Store.PRIORITY_USER;
  } else {
return this.blockingStoreFileCount - this.storefiles.size();
  }
}
 {code}.
 PRIORITY_USER is 1.
 The priorities are compared as numbers in HRegion, so compactions of blocking 
 stores will override user priority (probably intended); also, if you have 
 blockingFiles minus one, your priority is suddenly PRIORITY_USER, which may 
 cause at least this:
 LOG.debug(Warning, compacting more than  + 
 comConf.getMaxFilesToCompact() +
  files because of a user-requested major compaction);
 as well as some misleading logging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7982) TestReplicationQueueFailover* runs for a minute, spews 3/4million lines complaining 'Filesystem closed', has an NPE, and still passes?

2013-03-04 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592473#comment-13592473
 ] 

stack commented on HBASE-7982:
--

[~jeffreyz] Thanks for taking a look sir and attaching the patch.  The patch is 
a little odd here:

{code} if (this.reader == null || !this.lastPath.equals(path)) {
+  if (this.reader != null) {
+this.closeReader();
+  }
{code}

... in that we check this.reader for nullness twice in two lines.  If 
this.reader == null, nothing happens right?  So, just remove the check?

Will I wait on your table loading issue examination before committing this NPE 
fix?  Do you think it responsible for the log being so big?

[~nkeywal] Let me try.  Will wait see what Jeffrey says on above and will put 
the two patches together

 TestReplicationQueueFailover* runs for a minute, spews 3/4million lines 
 complaining 'Filesystem closed', has an NPE, and still passes?
 --

 Key: HBASE-7982
 URL: https://issues.apache.org/jira/browse/HBASE-7982
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: stack
Priority: Blocker
 Attachments: hbase-7982-NPE.patch


 I was trying to look at why the odd time Hudson OOMEs trying to make a report 
 on 0.95 build #4 https://builds.apache.org/job/hbase-0.95/4/console:
 {code}
 ERROR: Failed to archive test reports
 hudson.util.IOException2: remote file operation failed: 
 /home/jenkins/jenkins-slave/workspace/hbase-0.95 at 
 hudson.remoting.Channel@151a4e3e:ubuntu3
   at hudson.FilePath.act(FilePath.java:861)
   at hudson.FilePath.act(FilePath.java:838)
   at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
   at 
 ...
 Caused by: java.lang.OutOfMemoryError: Java heap space
   at java.nio.HeapCharBuffer.init(HeapCharBuffer.java:57)
   at java.nio.CharBuffer.allocate(CharBuffer.java:329)
   at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:792)
   at java.nio.charset.Charset.decode(Charset.java:791)
   at hudson.tasks.junit.SuiteResult.init(SuiteResult.java:215)
 ...
 {code}
 We are trying to allocate a big buffer and failing.
 Looking at reports being generated, we have quite a few that are  10MB in 
 size:
 {code}
 durruti:0.95 stack$ find hbase-* -type f -size +1k -exec ls -la {} \;
 -rw-r--r--@ 1 stack  staff  11126492 Feb 27 06:14 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.backup.TestHFileArchiving-output.txt
 -rw-r--r--@ 1 stack  staff  13296009 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestFromClientSide3-output.txt
 -rw-r--r--@ 1 stack  staff  10541898 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
 -rw-r--r--@ 1 stack  staff  25344601 Feb 27 05:51 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient-output.txt
 -rw-r--r--@ 1 stack  staff  17966969 Feb 27 06:12 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction-output.txt
 -rw-r--r--@ 1 stack  staff  17699068 Feb 27 06:09 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
 -rw-r--r--@ 1 stack  staff  17701832 Feb 27 06:07 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplitCompressed-output.txt
 -rw-r--r--@ 1 stack  staff  717853709 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailover-output.txt
 -rw-r--r--@ 1 stack  staff  563616793 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed-output.txt
 {code}
 ... with TestReplicationQueueFailover* being order of magnitude bigger than 
 the others.
 Looking in the test I see both spewing between 800 and 900 thousand lines in 
 about a minute.  Here is their fixation:
 {code}
 8908998 2013-02-27 06:17:48,176 ERROR 
 [RegionServer:1;hemera.apache.org,35712,1361945801803.logSyncer] 
 wal.FSHLog$LogSyncer(1012): Error while syncing, requesting close of hlog.
 8908999 java.io.IOException: Filesystem closed
 8909000 ,...at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:319)
 8909001 ,...at org.apache.hadoop.hdfs.DFSClient.access$1200(DFSClient.java:78)
 8909002 ,...at 
 org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.sync(DFSClient.java:3843)
 8909003 ,...at 
 org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:97)
 8909004 ,...at 
 org.apache.hadoop.io.SequenceFile$Writer.syncFs(SequenceFile.java:999)
 8909005 ,...at

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592474#comment-13592474
 ] 

Jeffrey Zhong commented on HBASE-7824:
--

[~lhofhansl]I'm fine to revert it for now and put it back up for 0.94.7 because 
there is no hurry for this.

Thanks,
-Jeffrey

 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7991) Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could cause server abort when splitting' to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592475#comment-13592475
 ] 

Lars Hofhansl commented on HBASE-7991:
--

Simple fix +1

 Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could 
 cause server abort when splitting' to 0.94
 --

 Key: HBASE-7991
 URL: https://issues.apache.org/jira/browse/HBASE-7991
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 Bryan Beaudreault was hit by this issue in his 0.94.2 clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592478#comment-13592478
 ] 

Lars Hofhansl commented on HBASE-7990:
--

Another one-liner. +1

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7990:
-

Fix Version/s: 0.94.6

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Fix For: 0.94.6


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7991) Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could cause server abort when splitting' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7991:
-

Fix Version/s: 0.94.6

 Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could 
 cause server abort when splitting' to 0.94
 --

 Key: HBASE-7991
 URL: https://issues.apache.org/jira/browse/HBASE-7991
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Fix For: 0.94.6


 Bryan Beaudreault was hit by this issue in his 0.94.2 clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7991) Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could cause server abort when splitting' to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592479#comment-13592479
 ] 

Lars Hofhansl commented on HBASE-7991:
--

In the light of recent discussions, I would not call the backports. It's 
simple bug fix that we did apply to trunk and forgot to apply to 0.94.

 Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could 
 cause server abort when splitting' to 0.94
 --

 Key: HBASE-7991
 URL: https://issues.apache.org/jira/browse/HBASE-7991
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Fix For: 0.94.6


 Bryan Beaudreault was hit by this issue in his 0.94.2 clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-7991) Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could cause server abort when splitting' to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592479#comment-13592479
 ] 

Lars Hofhansl edited comment on HBASE-7991 at 3/4/13 6:44 PM:
--

In the light of recent discussions, I would not call this a backport. It's 
simple bug fix that we did apply to trunk and forgot to apply to 0.94.

  was (Author: lhofhansl):
In the light of recent discussions, I would not call the backports. It's 
simple bug fix that we did apply to trunk and forgot to apply to 0.94.
  
 Backport HBASE-6479 'HFileReaderV1 caching the same parent META block could 
 cause server abort when splitting' to 0.94
 --

 Key: HBASE-7991
 URL: https://issues.apache.org/jira/browse/HBASE-7991
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Fix For: 0.94.6


 Bryan Beaudreault was hit by this issue in his 0.94.2 clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592481#comment-13592481
 ] 

Lars Hofhansl commented on HBASE-7824:
--

If we can work out the failure that would be preferable of course :)

 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7982) TestReplicationQueueFailover* runs for a minute, spews 3/4million lines complaining 'Filesystem closed', has an NPE, and still passes?


[ 
https://issues.apache.org/jira/browse/HBASE-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592484#comment-13592484
 ] 

Jeffrey Zhong commented on HBASE-7982:
--

yeah, the following check isn't needed because closeReader will do that check 
internally. The closeReader() here is to close the old reader before we create 
a new one.
{code}
if (this.reader != null)
{code}


 TestReplicationQueueFailover* runs for a minute, spews 3/4million lines 
 complaining 'Filesystem closed', has an NPE, and still passes?
 --

 Key: HBASE-7982
 URL: https://issues.apache.org/jira/browse/HBASE-7982
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: stack
Priority: Blocker
 Attachments: hbase-7982-NPE.patch


 I was trying to look at why the odd time Hudson OOMEs trying to make a report 
 on 0.95 build #4 https://builds.apache.org/job/hbase-0.95/4/console:
 {code}
 ERROR: Failed to archive test reports
 hudson.util.IOException2: remote file operation failed: 
 /home/jenkins/jenkins-slave/workspace/hbase-0.95 at 
 hudson.remoting.Channel@151a4e3e:ubuntu3
   at hudson.FilePath.act(FilePath.java:861)
   at hudson.FilePath.act(FilePath.java:838)
   at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
   at 
 ...
 Caused by: java.lang.OutOfMemoryError: Java heap space
   at java.nio.HeapCharBuffer.init(HeapCharBuffer.java:57)
   at java.nio.CharBuffer.allocate(CharBuffer.java:329)
   at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:792)
   at java.nio.charset.Charset.decode(Charset.java:791)
   at hudson.tasks.junit.SuiteResult.init(SuiteResult.java:215)
 ...
 {code}
 We are trying to allocate a big buffer and failing.
 Looking at reports being generated, we have quite a few that are  10MB in 
 size:
 {code}
 durruti:0.95 stack$ find hbase-* -type f -size +1k -exec ls -la {} \;
 -rw-r--r--@ 1 stack  staff  11126492 Feb 27 06:14 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.backup.TestHFileArchiving-output.txt
 -rw-r--r--@ 1 stack  staff  13296009 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestFromClientSide3-output.txt
 -rw-r--r--@ 1 stack  staff  10541898 Feb 27 05:47 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
 -rw-r--r--@ 1 stack  staff  25344601 Feb 27 05:51 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient-output.txt
 -rw-r--r--@ 1 stack  staff  17966969 Feb 27 06:12 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction-output.txt
 -rw-r--r--@ 1 stack  staff  17699068 Feb 27 06:09 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
 -rw-r--r--@ 1 stack  staff  17701832 Feb 27 06:07 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestHLogSplitCompressed-output.txt
 -rw-r--r--@ 1 stack  staff  717853709 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailover-output.txt
 -rw-r--r--@ 1 stack  staff  563616793 Feb 27 06:17 
 hbase-server/target/surefire-reports/org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed-output.txt
 {code}
 ... with TestReplicationQueueFailover* being order of magnitude bigger than 
 the others.
 Looking in the test I see both spewing between 800 and 900 thousand lines in 
 about a minute.  Here is their fixation:
 {code}
 8908998 2013-02-27 06:17:48,176 ERROR 
 [RegionServer:1;hemera.apache.org,35712,1361945801803.logSyncer] 
 wal.FSHLog$LogSyncer(1012): Error while syncing, requesting close of hlog.
 8908999 java.io.IOException: Filesystem closed
 8909000 ,...at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:319)
 8909001 ,...at org.apache.hadoop.hdfs.DFSClient.access$1200(DFSClient.java:78)
 8909002 ,...at 
 org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.sync(DFSClient.java:3843)
 8909003 ,...at 
 org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:97)
 8909004 ,...at 
 org.apache.hadoop.io.SequenceFile$Writer.syncFs(SequenceFile.java:999)
 8909005 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:248)
 8909006 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.syncer(FSHLog.java:1120)
 8909007 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.syncer(FSHLog.java:1058)
 8909008 ,...at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:1228)
 8909009 ,...at

[jira] [Updated] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk


 [ 
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Kinley updated HBASE-7482:


Attachment: HBASE-7482-trunk.patch

 Port HBASE-7442 HBase remote CopyTable not working when security enabled to 
 trunk
 -

 Key: HBASE-7482
 URL: https://issues.apache.org/jira/browse/HBASE-7482
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: James Kinley
Priority: Critical
 Fix For: 0.95.0

 Attachments: HBASE-7482-trunk.patch


 Excerpt about the choice of solution from :
 The first option was actually quite messy to implement. {{clusterId}} and 
 {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by 
 *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass 
 the different cluster {{confs}} up through *{{HConnectionManager}}* and 
 *{{HBaseRPC}}* in order to override the clusterId in 
 *{{SecureClient#SecureConnection}}*.
 I've gone with the second option of creating and caching different 
 *{{SecureClients}}* for the local and remote clusters in 
 *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default 
 *{{SocketFactory}}*. I think this is a cleaner solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592492#comment-13592492
 ] 

Jeffrey Zhong commented on HBASE-7824:
--

Yeah, in either way I'll get the bottom of this(hopefully by end of today) so 
that we can be sure there is no issue. 

 

 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7841) Parallelize offline snapshot in DisabledTableSnapshotHandler


[ 
https://issues.apache.org/jira/browse/HBASE-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592494#comment-13592494
 ] 

Matteo Bertozzi commented on HBASE-7841:


Why not just using an ExecutorCompletionService with a Callable as we have in 
other places like: 
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L433

 Parallelize offline snapshot in DisabledTableSnapshotHandler
 

 Key: HBASE-7841
 URL: https://issues.apache.org/jira/browse/HBASE-7841
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu
 Attachments: 7841.txt


 In DisabledTableSnapshotHandler, there is TODO:
 {code}
   // TODO consider parallelizing these operations since they are independent. 
 Right now its just
   // easier to keep them serial though
   @Override
   public void snapshotRegions(ListPairHRegionInfo, ServerName 
 regionsAndLocations) throws IOException,
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk


[ 
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592500#comment-13592500
 ] 

James Kinley commented on HBASE-7482:
-

In addition to the TableMapReduceUtil change, 'hbase.cluster.id' has to be 
reset in HConnectionImplementation to ensure that it's HBaseClient instance has 
the correct 'clusterId' set and therefore the correct token is selected when 
setting up the Connection.

 Port HBASE-7442 HBase remote CopyTable not working when security enabled to 
 trunk
 -

 Key: HBASE-7482
 URL: https://issues.apache.org/jira/browse/HBASE-7482
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: James Kinley
Priority: Critical
 Fix For: 0.95.0

 Attachments: HBASE-7482-trunk.patch


 Excerpt about the choice of solution from :
 The first option was actually quite messy to implement. {{clusterId}} and 
 {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by 
 *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass 
 the different cluster {{confs}} up through *{{HConnectionManager}}* and 
 *{{HBaseRPC}}* in order to override the clusterId in 
 *{{SecureClient#SecureConnection}}*.
 I've gone with the second option of creating and caching different 
 *{{SecureClients}}* for the local and remote clusters in 
 *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default 
 *{{SocketFactory}}*. I think this is a cleaner solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592501#comment-13592501
 ] 

Lars Hofhansl commented on HBASE-7824:
--

Thanks [~jeffreyz]!

 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7791) Compaction USER_PRIORITY is slightly broken


[ 
https://issues.apache.org/jira/browse/HBASE-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592503#comment-13592503
 ] 

Nick Dimiduk commented on HBASE-7791:
-

bq. I wonder if we can trust all of them to ensure that.

Make it an explicit part of the API (documented in the Interface) and then it's 
not your problem :)

 Compaction USER_PRIORITY is slightly broken
 ---

 Key: HBASE-7791
 URL: https://issues.apache.org/jira/browse/HBASE-7791
 Project: HBase
  Issue Type: Bug
  Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HBASE-7791-v0.patch


 The code to get compaction priority is as such:
 {code}   public int getCompactPriority(int priority) {
  // If this is a user-requested compaction, leave this at the highest 
 priority
  if(priority == Store.PRIORITY_USER) {
return Store.PRIORITY_USER;
  } else {
return this.blockingStoreFileCount - this.storefiles.size();
  }
}
 {code}.
 PRIORITY_USER is 1.
 The priorities are compared as numbers in HRegion, so compactions of blocking 
 stores will override user priority (probably intended); also, if you have 
 blockingFiles minus one, your priority is suddenly PRIORITY_USER, which may 
 cause at least this:
 LOG.debug(Warning, compacting more than  + 
 comConf.getMaxFilesToCompact() +
  files because of a user-requested major compaction);
 as well as some misleading logging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7948) client doesn't need to refresh meta while the region is opening


 [ 
https://issues.apache.org/jira/browse/HBASE-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7948:


Attachment: HBASE-7948-v2.patch

Updating timeout, also some feedback from original JIRA.
There's no findbugs report in latest testResults, I will take a look after this 
one is picked up.

 client doesn't need to refresh meta while the region is opening
 ---

 Key: HBASE-7948
 URL: https://issues.apache.org/jira/browse/HBASE-7948
 Project: HBase
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HBASE-7948-v0.patch, HBASE-7948-v1.patch, 
 HBASE-7948-v1.patch, HBASE-7948-v2.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592505#comment-13592505
 ] 

Ted Yu commented on HBASE-7990:
---

Integrated to 0.94

Thanks for the confirmation, Lars.

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reassigned HBASE-7990:
-

Assignee: Ted Yu

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Work started] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-7990 started by Ted Yu.

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7990:
--

Attachment: 7990.txt

What I integrated.

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6

 Attachments: 7990.txt


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7990:
--

Hadoop Flags: Reviewed
  Status: Patch Available  (was: In Progress)

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6

 Attachments: 7990.txt


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7990) Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of malformed rowkey' to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7990:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Backport HBASE-5837 'hbase shell deleteall to .META. allows insertion of 
 malformed rowkey' to 0.94
 --

 Key: HBASE-7990
 URL: https://issues.apache.org/jira/browse/HBASE-7990
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.94.6

 Attachments: 7990.txt


 This was brought up in discussion entitled 'HRegionInfo was null or empty in 
 Meta errors'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7992) provide pre/post offile region hooks for HMaster.offlineRegion().

2013-03-04 Thread rajeshbabu (JIRA)

rajeshbabu created HBASE-7992:
-

 Summary: provide pre/post offile region hooks for 
HMaster.offlineRegion(). 
 Key: HBASE-7992
 URL: https://issues.apache.org/jira/browse/HBASE-7992
 Project: HBase
  Issue Type: Bug
  Components: Coprocessors
Affects Versions: 0.95.0
Reporter: rajeshbabu
Assignee: rajeshbabu
 Fix For: 0.98.0


presently no hooks to provide access control to offline region in master.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7993) add access control to HMaster.offline region.

2013-03-04 Thread rajeshbabu (JIRA)

rajeshbabu created HBASE-7993:
-

 Summary: add access control to HMaster.offline region.
 Key: HBASE-7993
 URL: https://issues.apache.org/jira/browse/HBASE-7993
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.95.0
Reporter: rajeshbabu
Assignee: rajeshbabu
 Fix For: 0.98.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7808) Refactor Store to use HRegionFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592513#comment-13592513
 ] 

Sergey Shelukhin commented on HBASE-7808:
-

are tests above unrelated? +1 provided that

 Refactor Store to use HRegionFileSystem
 ---

 Key: HBASE-7808
 URL: https://issues.apache.org/jira/browse/HBASE-7808
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: HBASE-7808-v0.patch, HBASE-7808-v1.patch, 
 HBASE-7808-v2.patch, HBASE-7808-v3.patch, HBASE-7808-v4.patch


 Move HStore fs related code inside the HRegionFileSystem. 
 File creation and retrival should be handled there, and the use will be 
 something like.
 {code}
 HRegionFileSystem fs = new HRegionFileSystem
 file = fs.createFile()
 fs.commitFile(family, file);
 for (storeFile: fs.getStoreFiles(family))
...
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7809) Refactor Split/Merge to use HRegionFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592515#comment-13592515
 ] 

Sergey Shelukhin commented on HBASE-7809:
-

left minor general feedback, may review in more detail later... btw, you can 
attach combined patches to jira to run tests even before the prerequisites are 
ready

 Refactor Split/Merge to use HRegionFileSystem
 -

 Key: HBASE-7809
 URL: https://issues.apache.org/jira/browse/HBASE-7809
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor

 Use the HRegionFileSystem for the fs operations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7935) make policy and compactor in default store engine separately pluggable (for things like tier-based, and default policy experiments with permutations)


[ 
https://issues.apache.org/jira/browse/HBASE-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592516#comment-13592516
 ] 

Sergey Shelukhin commented on HBASE-7935:
-

ping? thanks

 make policy and compactor in default store engine separately pluggable (for 
 things like tier-based, and default policy experiments with permutations)
 -

 Key: HBASE-7935
 URL: https://issues.apache.org/jira/browse/HBASE-7935
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HBASE-7935-v0.patch, HBASE-7935-v0-with-7843.patch, 
 HBASE-7935-v1.patch


 Technically, StoreEngine can be used to achieve any permutations of things, 
 but to make it more convenient to replace compaction policy/compator in 
 standard schemes like tier-based, we can add separate hooks in 
 DefaultStoreEngine (as long as custom ones conform to its default 
 expectations e.g. flat list of sorted files, etc.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7902) deletes may be removed during minor compaction, in non-standard compaction schemes


[ 
https://issues.apache.org/jira/browse/HBASE-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592521#comment-13592521
 ] 

Sergey Shelukhin commented on HBASE-7902:
-

resolved as it was committed to trunk and 95

 deletes may be removed during minor compaction,  in non-standard compaction 
 schemes
 ---

 Key: HBASE-7902
 URL: https://issues.apache.org/jira/browse/HBASE-7902
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Attachments: HBASE-7902-v0.patch, HBASE-7902-v0-with-7843.patch, 
 HBASE-7902-v1-.patch, HBASE-7902-v1.patch


 Deletes are only removed during major compaction now. However, in presence of 
 file ordering, deletes can be removed during minor compaction too, as long as 
 there's no file that is not being compacted that is older than the files that 
 are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work


[ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592522#comment-13592522
 ] 

Ted Yu commented on HBASE-7824:
---

Talked with Jeffrey.

I reverted the patch for now.

Jeffrey would provide his suggestion on how to make TestMasterFailover more 
reliable, along with his changes.

 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7992) provide pre/post offile region hooks for HMaster.offlineRegion().

2013-03-04 Thread rajeshbabu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592525#comment-13592525
 ] 

rajeshbabu commented on HBASE-7992:
---

I will provide patch for HBASE-7992  HBASE-7993

 provide pre/post offile region hooks for HMaster.offlineRegion(). 
 --

 Key: HBASE-7992
 URL: https://issues.apache.org/jira/browse/HBASE-7992
 Project: HBase
  Issue Type: Bug
  Components: Coprocessors
Affects Versions: 0.95.0
Reporter: rajeshbabu
Assignee: rajeshbabu
 Fix For: 0.98.0


 presently no hooks to provide access control to offline region in master.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().


 [ 
https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7992:
--

Summary: provide pre/post region offline hooks for HMaster.offlineRegion(). 
  (was: provide pre/post offile region hooks for HMaster.offlineRegion(). )

 provide pre/post region offline hooks for HMaster.offlineRegion(). 
 ---

 Key: HBASE-7992
 URL: https://issues.apache.org/jira/browse/HBASE-7992
 Project: HBase
  Issue Type: Bug
  Components: Coprocessors
Affects Versions: 0.95.0
Reporter: rajeshbabu
Assignee: rajeshbabu
 Fix For: 0.98.0


 presently no hooks to provide access control to offline region in master.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HBASE-7824) Improve master start up time when there is log splitting work


 [ 
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl reopened HBASE-7824:
--


 Improve master start up time when there is log splitting work
 -

 Key: HBASE-7824
 URL: https://issues.apache.org/jira/browse/HBASE-7824
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.94.6

 Attachments: hbase-7824.patch, hbase-7824_v2.patch


 When there is log split work going on, master start up waits till all log 
 split work completes even though the log split has nothing to do with meta 
 region servers.
 It's a bad behavior considering a master node can run when log split is 
 happening while its start up is blocking by log split work. 
 Since master is kind of single point of failure, we should start it ASAP.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7824) Improve master start up time when there is log splitting work