[jira] [Updated] (HBASE-5706) Dropping fs latency stats since buffer is full spam
[
https://issues.apache.org/jira/browse/HBASE-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-5706:
--
Comment: was deleted
(was:
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via
Tom White)
)
Dropping fs latency stats since buffer is full spam
-
Key: HBASE-5706
URL: https://issues.apache.org/jira/browse/HBASE-5706
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Shaneal Manek
Priority: Minor
Fix For: 0.94.0, 0.96.0
I see tons of this while running tests (note that it's a WARN):
{noformat}
2012-04-03 18:54:47,172 WARN org.apache.hadoop.hbase.io.hfile.HFile: Dropping
fs latency stats since buffer is full
{noformat}
While the code says this:
{noformat}
// we don't want to fill up the logs with this message, so only log it
// once every 30 seconds at most
// I also want to avoid locks on the 'critical path' (the common case will
be
// uncontended) - hence the CAS
private static void logDroppedLatencyStat() {
{noformat}
It doesn't seem like this message is actionnable and even though it's printed
only every 30 seconds it's still very spammy.
We should get rid of it or make it more useful (I don't know which).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5480) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+
[ https://issues.apache.org/jira/browse/HBASE-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5480: -- Attachment: HBASE-5480.patch Fixups to MultithreadedTableMapper for Hadoop 0.23.2+ - Key: HBASE-5480 URL: https://issues.apache.org/jira/browse/HBASE-5480 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Andrew Purtell Priority: Critical Attachments: HBASE-5480.patch There are two issues: - StatusReporter has a new method getProgress() - Mapper and reducer context objects can no longer be directly instantiated. See attached patch. I'm not thrilled with the added reflection but it was the minimally intrusive change. Raised the priority to critical because compilation fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5480) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+
[ https://issues.apache.org/jira/browse/HBASE-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5480: -- Attachment: (was: HBASE-5480.patch) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+ - Key: HBASE-5480 URL: https://issues.apache.org/jira/browse/HBASE-5480 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Andrew Purtell Priority: Critical Attachments: HBASE-5480.patch There are two issues: - StatusReporter has a new method getProgress() - Mapper and reducer context objects can no longer be directly instantiated. See attached patch. I'm not thrilled with the added reflection but it was the minimally intrusive change. Raised the priority to critical because compilation fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5480) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+
[ https://issues.apache.org/jira/browse/HBASE-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5480: -- Attachment: HBASE-5480.patch Corrected patch with --no-prefix Fixups to MultithreadedTableMapper for Hadoop 0.23.2+ - Key: HBASE-5480 URL: https://issues.apache.org/jira/browse/HBASE-5480 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Andrew Purtell Priority: Critical Attachments: HBASE-5480.patch There are two issues: - StatusReporter has a new method getProgress() - Mapper and reducer context objects can no longer be directly instantiated. See attached patch. I'm not thrilled with the added reflection but it was the minimally intrusive change. Raised the priority to critical because compilation fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5228: -- Attachment: HBASE-5228.patch [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5228.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5228: -- Fix Version/s: 0.90.6 0.92.1 0.94.0 @Stack: Ok @Ted: Thanks for the patch review, will fix on commit if you think this is good to go in otherwise. [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5228: -- Attachment: (was: HBASE-5228.patch) [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5228: -- Attachment: HBASE-5228-trunk.patch [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5228: -- Attachment: HBASE-5228-0.92.patch [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5210) HFiles are missing from an incremental load
[ https://issues.apache.org/jira/browse/HBASE-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5210: -- Attachment: HBASE-5210-crazy-new-getRandomFilename.patch Patch for discussion. HFiles are missing from an incremental load --- Key: HBASE-5210 URL: https://issues.apache.org/jira/browse/HBASE-5210 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.2 Environment: HBase 0.90.2 with Hadoop-0.20.2 (with durable sync). RHEL 2.6.18-164.15.1.el5. 4 node cluster (1 master, 3 slaves) Reporter: Lawrence Simpson Attachments: HBASE-5210-crazy-new-getRandomFilename.patch We run an overnight map/reduce job that loads data from an external source and adds that data to an existing HBase table. The input files have been loaded into hdfs. The map/reduce job uses the HFileOutputFormat (and the TotalOrderPartitioner) to create HFiles which are subsequently added to the HBase table. On at least two separate occasions (that we know of), a range of output would be missing for a given day. The range of keys for the missing values corresponded to those of a particular region. This implied that a complete HFile somehow went missing from the job. Further investigation revealed the following: * Two different reducers (running in separate JVMs and thus separate class loaders) * in the same server can end up using the same file names for their * HFiles. The scenario is as follows: *1. Both reducers start near the same time. *2. The first reducer reaches the point where it wants to write its first file. *3. It uses the StoreFile class which contains a static Random object *which is initialized by default using a timestamp. *4. The file name is generated using the random number generator. *5. The file name is checked against other existing files. *6. The file is written into temporary files in a directory named *after the reducer attempt. *7. The second reduce task reaches the same point, but its StoreClass *(which is now in the file system's cache) gets loaded within the *time resolution of the OS and thus initializes its Random() *object with the same seed as the first task. *8. The second task also checks for an existing file with the name *generated by the random number generator and finds no conflict *because each task is writing files in its own temporary folder. *9. The first task finishes and gets its temporary files committed *to the real folder specified for output of the HFiles. * 10.The second task then reaches its own conclusion and commits its *files (moveTaskOutputs). The released Hadoop code just overwrites *any files with the same name. No warning messages or anything. *The first task's HFiles just go missing. * * Note: The reducers here are NOT different attempts at the same *reduce task. They are different reduce tasks so data is *really lost. I am currently testing a fix in which I have added code to the Hadoop FileOutputCommitter.moveTaskOutputs method to check for a conflict with an existing file in the final output folder and to rename the HFile if needed. This may not be appropriate for all uses of FileOutputFormat. So I have put this into a new class which is then used by a subclass of HFileOutputFormat. Subclassing of FileOutputCommitter itself was a bit more of a problem due to private declarations. I don't know if my approach is the best fix for the problem. If someone more knowledgeable than myself deems that it is, I will be happy to share what I have done and by that time I may have some information on the results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5195) [Coprocessors] preGet hook does not allow overriding or wrapping filter on incoming Get
[ https://issues.apache.org/jira/browse/HBASE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5195: -- Fix Version/s: (was: 0.92.0) 0.92.1 The patch resolves this issue in a test cluster, but I'm going to add a unit test. Marking for 0.92.1. Change it if you feel strongly it should be in 0.92.0. [Coprocessors] preGet hook does not allow overriding or wrapping filter on incoming Get --- Key: HBASE-5195 URL: https://issues.apache.org/jira/browse/HBASE-5195 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5195.patch Without the ability to wrap the internal Scan on the Get, we can't override (or protect, in the case of access control) Gets as we can Scans. The result is inconsistent behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5195) [Coprocessors] preGet hook does not allow overriding or wrapping filter on incoming Get
[ https://issues.apache.org/jira/browse/HBASE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5195: -- Attachment: HBASE-5195.patch Trying out the attached patch. [Coprocessors] preGet hook does not allow overriding or wrapping filter on incoming Get --- Key: HBASE-5195 URL: https://issues.apache.org/jira/browse/HBASE-5195 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5195.patch Without the ability to wrap the internal Scan on the Get, we can't override (or protect, in the case of access control) Gets as we can Scans. The result is inconsistent behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5131) HBug
[ https://issues.apache.org/jira/browse/HBASE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5131: -- Attachment: hbug-idea-1.txt One idea, pardon the fevered rambling. HBug Key: HBASE-5131 URL: https://issues.apache.org/jira/browse/HBASE-5131 Project: HBase Issue Type: Brainstorming Reporter: Andrew Purtell Attachments: hbug-idea-1.txt A long lived full-system test case, for verifying releases, migration, integration with other tools. Could be filed under BigTop, but this should also be useful to the HBase project as part of its infrastructure. Goals: - Provide a live bug reporting system for the HBase project. - Provide (arguably) useful analysis of collected bug reports, and a framework for adding more over time. - Maintain a project owned application backed by a Hadoop+HBase cluster that can be used to try out release candidates as part of the release process, insuring rolling restart is possible. - Evaluate, and possibly improve, HBase-Pig integration motivated by a application use case. (Auto-mapping of Pig data structures into HBase tables and vice versa? Some mix of online schema modification and use of Avro or protobufs perhaps.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5131) HBug
[ https://issues.apache.org/jira/browse/HBASE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5131: -- Attachment: (was: hbug-idea-1.txt) HBug Key: HBASE-5131 URL: https://issues.apache.org/jira/browse/HBASE-5131 Project: HBase Issue Type: Brainstorming Reporter: Andrew Purtell Attachments: hbug-idea-1.txt A long lived full-system test case, for verifying releases, migration, integration with other tools. Could be filed under BigTop, but this should also be useful to the HBase project as part of its infrastructure. Goals: - Provide a live bug reporting system for the HBase project. - Provide (arguably) useful analysis of collected bug reports, and a framework for adding more over time. - Maintain a project owned application backed by a Hadoop+HBase cluster that can be used to try out release candidates as part of the release process, insuring rolling restart is possible. - Evaluate, and possibly improve, HBase-Pig integration motivated by a application use case. (Auto-mapping of Pig data structures into HBase tables and vice versa? Some mix of online schema modification and use of Avro or protobufs perhaps.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5131) HBug
[ https://issues.apache.org/jira/browse/HBASE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5131: -- Attachment: hbug-idea-1.txt HBug Key: HBASE-5131 URL: https://issues.apache.org/jira/browse/HBASE-5131 Project: HBase Issue Type: Brainstorming Reporter: Andrew Purtell Attachments: hbug-idea-1.txt A long lived full-system test case, for verifying releases, migration, integration with other tools. Could be filed under BigTop, but this should also be useful to the HBase project as part of its infrastructure. Goals: - Provide a live bug reporting system for the HBase project. - Provide (arguably) useful analysis of collected bug reports, and a framework for adding more over time. - Maintain a project owned application backed by a Hadoop+HBase cluster that can be used to try out release candidates as part of the release process, insuring rolling restart is possible. - Evaluate, and possibly improve, HBase-Pig integration motivated by a application use case. (Auto-mapping of Pig data structures into HBase tables and vice versa? Some mix of online schema modification and use of Avro or protobufs perhaps.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5061) HFileLocalityChecker
[ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5061: -- Attachment: StoreFileLocalityChecker.java HFileLocalityChecker Key: HBASE-5061 URL: https://issues.apache.org/jira/browse/HBASE-5061 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Attachments: StoreFileLocalityChecker.java org.apache.hadoop.hbase.HFileLocalityChecker [options] A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage. Where options are: |-f file|Analyze a store file| |-r region|Analyze all store files for the region| |-t table|Analyze all store files for regions of the table served by the local regionserver| |-h host|Consider host local, defaults to the local host| |-v|Verbose operation| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker
[ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5061: -- Summary: StoreFileLocalityChecker (was: HFileLocalityChecker) StoreFileLocalityChecker Key: HBASE-5061 URL: https://issues.apache.org/jira/browse/HBASE-5061 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Attachments: StoreFileLocalityChecker.java org.apache.hadoop.hbase.HFileLocalityChecker [options] A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage. Where options are: |-f file|Analyze a store file| |-r region|Analyze all store files for the region| |-t table|Analyze all store files for regions of the table served by the local regionserver| |-h host|Consider host local, defaults to the local host| |-v|Verbose operation| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Component/s: thrift security rest Description: Somehow the attached changes are missing from the security integration. (was: Somehow the attached change is missing from the security integration. The corresponding change to the Thrift server is in place.) Summary: Missing logons if security is enabled (was: [rest] Missing logon if security is enabled) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Attachment: HBASE-5062.patch Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Status: Patch Available (was: Open) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Attachment: HBASE-5062-v2.patch v2 patch: Since REST and Thrift are not authenticating for HDFS access, only HBase secure RPC, additionally check if _HBase_ authentication is set to kerberos. Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507816/HBASE-5062.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/532//console This message is automatically generated.) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507819/HBASE-5062-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/533//console This message is automatically generated.) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4990) Document secure HBase setup
[ https://issues.apache.org/jira/browse/HBASE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4990: -- Issue Type: Sub-task (was: Task) Parent: HBASE-4376 Document secure HBase setup --- Key: HBASE-4990 URL: https://issues.apache.org/jira/browse/HBASE-4990 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4376) Document login configuration when running on top of secure Hadoop with Kerberos auth enabled
[ https://issues.apache.org/jira/browse/HBASE-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4376: -- Affects Version/s: (was: 0.90.4) 0.94.0 0.92.0 HBASE-4376 should be committed to 0.92 branch also? Document login configuration when running on top of secure Hadoop with Kerberos auth enabled Key: HBASE-4376 URL: https://issues.apache.org/jira/browse/HBASE-4376 Project: HBase Issue Type: Task Components: documentation, security Affects Versions: 0.92.0, 0.94.0 Reporter: Gary Helmling We provide basic support for HBase to run on top of kerberos-authenticated Hadoop, by providing configuration options to have HMaster and HRegionServer login from a keytab on startup. But this isn't documented anywhere outside of hbase-default.xml. We need to provide some basic guidance on setup in the HBase docs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4990) Document secure HBase setup
[ https://issues.apache.org/jira/browse/HBASE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4990: -- Attachment: 4990.txt What I have so far. Can use an editing pass. Missing is description of the new shell commands and examples for using them. More soon. Document secure HBase setup --- Key: HBASE-4990 URL: https://issues.apache.org/jira/browse/HBASE-4990 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: 4990.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles
[ https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4944: -- Resolution: Fixed Fix Version/s: 0.90.5 0.94.0 0.92.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Optionally verify bulk loaded HFiles Key: HBASE-4944 URL: https://issues.apache.org/jira/browse/HBASE-4944 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.92.0, 0.94.0, 0.90.5 Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch We rely on users to produce properly formatted HFiles for bulk import. Attached patch adds an optional code path, toggled by a configuration property, that verifies the HFile under consideration for import is properly sorted. The default maintains the current behavior, which does not scan the file for correctness. Patch is against trunk but can apply against all active branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles
[ https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4944: -- Fix Version/s: (was: 0.90.5) (was: 0.94.0) (was: 0.92.0) Affects Version/s: 0.90.5 0.94.0 0.92.0 Status: Patch Available (was: Open) Optionally verify bulk loaded HFiles Key: HBASE-4944 URL: https://issues.apache.org/jira/browse/HBASE-4944 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Priority: Minor We rely on users to produce properly formatted HFiles for bulk import. Attached patch adds an optional code path, toggled by a configuration property, that verifies the HFile under consideration for import is properly sorted. The default maintains the current behavior, which does not scan the file for correctness. Patch is against trunk but can apply against all active branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles
[ https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4944: -- Attachment: HBASE-4944-v2.patch Rebased patch addressing Ted's comments. Optionally verify bulk loaded HFiles Key: HBASE-4944 URL: https://issues.apache.org/jira/browse/HBASE-4944 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Priority: Minor Attachments: 4944.txt, HBASE-4944-v2.patch We rely on users to produce properly formatted HFiles for bulk import. Attached patch adds an optional code path, toggled by a configuration property, that verifies the HFile under consideration for import is properly sorted. The default maintains the current behavior, which does not scan the file for correctness. Patch is against trunk but can apply against all active branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles
[ https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4944: -- Attachment: (was: 4944.txt) Optionally verify bulk loaded HFiles Key: HBASE-4944 URL: https://issues.apache.org/jira/browse/HBASE-4944 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Priority: Minor Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch We rely on users to produce properly formatted HFiles for bulk import. Attached patch adds an optional code path, toggled by a configuration property, that verifies the HFile under consideration for import is properly sorted. The default maintains the current behavior, which does not scan the file for correctness. Patch is against trunk but can apply against all active branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles
[ https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4944: -- Attachment: HBASE-4944-v3.patch Sorry, v3 patch restores the default to false (current behavior) Optionally verify bulk loaded HFiles Key: HBASE-4944 URL: https://issues.apache.org/jira/browse/HBASE-4944 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Priority: Minor Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch We rely on users to produce properly formatted HFiles for bulk import. Attached patch adds an optional code path, toggled by a configuration property, that verifies the HFile under consideration for import is properly sorted. The default maintains the current behavior, which does not scan the file for correctness. Patch is against trunk but can apply against all active branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)
[
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4945:
--
Attachment: HBASE-4945.patch
NPE in HRegion.bulkLoadHFiles(...)
--
Key: HBASE-4945
URL: https://issues.apache.org/jira/browse/HBASE-4945
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.92.0, 0.94.0
Reporter: Lars Hofhansl
Priority: Minor
Attachments: HBASE-4945.patch
Was playing with completebulkload, and ran into an NPE.
The problem is here (HRegion.bulkLoadHFiles(...)).
{code}
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
}
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
{code}
This should be
{code}
Store store = getStore(familyName);
if (store == null) {
...
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
...
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)
[
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4945:
--
Affects Version/s: 0.92.0
Status: Patch Available (was: Open)
NPE in HRegion.bulkLoadHFiles(...)
--
Key: HBASE-4945
URL: https://issues.apache.org/jira/browse/HBASE-4945
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.92.0, 0.94.0
Reporter: Lars Hofhansl
Priority: Minor
Attachments: HBASE-4945.patch
Was playing with completebulkload, and ran into an NPE.
The problem is here (HRegion.bulkLoadHFiles(...)).
{code}
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
}
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
{code}
This should be
{code}
Store store = getStore(familyName);
if (store == null) {
...
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
...
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4857) Recursive loop on KeeperException in AuthenticationTokenSecretManager/ZKLeaderManager
[
https://issues.apache.org/jira/browse/HBASE-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4857:
--
Priority: Critical (was: Major)
Nice catch. +1 on commit and raise to Critical.
Recursive loop on KeeperException in
AuthenticationTokenSecretManager/ZKLeaderManager
-
Key: HBASE-4857
URL: https://issues.apache.org/jira/browse/HBASE-4857
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.92.0, 0.94.0
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Critical
Fix For: 0.92.0
Attachments: HBASE-4857.patch
Looking through stack traces for {{TestMasterFailover}}, I see a case where
the leader {{AuthenticationTokenSecretManager}} can get into a recursive loop
when a {{KeeperException}} is encountered:
{noformat}
Thread-1-EventThread daemon prio=10 tid=0x7f9fb47b2800 nid=0x77f6
waiting on condition [0x7f9fab376000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at java.lang.Thread.sleep(Thread.java:302)
at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:328)
at
org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry(RetryCounter.java:55)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:206)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:891)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:161)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.init(ZooKeeperWatcher.java:154)
at
org.apache.hadoop.hbase.master.HMaster.tryRecoveringExpiredZKSession(HMaster.java:1397)
at org.apache.hadoop.hbase.master.HMaster.abortNow(HMaster.java:1435)
at org.apache.hadoop.hbase.master.HMaster.abort(HMaster.java:1374)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.abort(ZooKeeperWatcher.java:450)
at
org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.stepDownAsLeader(ZKLeaderManager.java:166)
at
org.apache.hadoop.hbase.security.token.AuthenticationTokenSecretManager$LeaderElector.stop(AuthenticationTokenSecretManager.java:293)
at
org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.stepDownAsLeader(ZKLeaderManager.java:167)
at
org.apache.hadoop.hbase.security.token.AuthenticationTokenSecretManager$LeaderElector.stop(AuthenticationTokenSecretManager.java:293)
at
org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.stepDownAsLeader(ZKLeaderManager.java:167)
at
org.apache.hadoop.hbase.security.token.AuthenticationTokenSecretManager$LeaderElector.stop(AuthenticationTokenSecretManager.java:293)
at
org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.handleLeaderChange(ZKLeaderManager.java:96)
at
org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.nodeDeleted(ZKLeaderManager.java:78)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:286)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
{noformat}
The {{KeeperException}} causes {{ZKLeaderManager}} to call
{{AuthenticationTokenSecretManager$LeaderElector.stop()}}, which calls
{{ZKLeaderManager.stepDownAsLeader()}}, which will encounter another
{{KeeperException}}, and so on...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4835) CME out of ZKConfig.makeZKProps
[
https://issues.apache.org/jira/browse/HBASE-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4835:
--
Attachment: HBASE-4835.patch
I think the simplest course of action is to make a shallow copy of the
Configuration in the ZKW constructor.
CME out of ZKConfig.makeZKProps
---
Key: HBASE-4835
URL: https://issues.apache.org/jira/browse/HBASE-4835
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: HBASE-4835.patch
Mikhail reported this from a five-node, three-RS cluster test:
{code}
2011-11-21 01:30:15,188 FATAL
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
machine_name,60020,1321867814890: Initialization of RS failed. Hence
aborting RS.
java.util.ConcurrentModificationException
at java.util.Hashtable$Enumerator.next(Hashtable.java:1031)
at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1042)
at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:75)
at
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:245)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.init(ZooKeeperWatcher.java:144)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.init(ZooKeeperWatcher.java:124)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.init(HConnectionManager.java:559)
at
org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183)
at
org.apache.hadoop.hbase.catalog.CatalogTracker.init(CatalogTracker.java:177)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:575)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:534)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:642)
at java.lang.Thread.run(Thread.java:619)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[
https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-2418:
--
Resolution: Fixed
Fix Version/s: 0.94.0
Release Note:
This adds support for protecting the state of HBase znodes on a multi-tenant
ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to
HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).
SASL authentication of ZooKeeper clients with the quorum is handled in the ZK
client independently of HBase concerns. To enable strong ZK authentication, one
must create a suitable JaaS configuration, for example:
Server {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab=/etc/hbase/conf/hbase.keytab
storeKey=true
useTicketCache=false
principal=zookeeper/$HOSTNAME;
};
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
useTicketCache=false
keyTab=/etc/hbase/conf/hbase.keytab
principal=hbase/$HOSTNAME;
};
and then configure both the client and server processes to use it, for example
in hbase-site.xml:
HBASE_OPTS=${HBASE_OPTS}
-Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf
HBASE_OPTS=${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true
HBASE_OPTS=${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true
HBase will then secure all znodes but for a few world-readable read-only ones
needed for clients to look up region locations. All internal cluster operations
will be protected from unauthenticated ZK clients, or clients not authenticated
to the HBase principal. Presumably the only ZK clients authenticated to the
HBase principal will be those embedded in the master and regionservers.
We will pull in a Hadoop artifact patched with HADOOP-7070 if building under
the security profile (-P security). 0.20.205 does not yet include HADOOP-7070.
Without it, the JAAS configuration required for secure operation of the
ZooKeeper client will be ignored.
Status: Resolved (was: Patch Available)
Committed to trunk and 0.92.
TestZooKeeperACL passes with and without '-P security' locally. Does not break
the build if '-P security' is not specified. Test failures found by HudsonQA
are not directly related to this change.
add support for ZooKeeper authentication
Key: HBASE-2418
URL: https://issues.apache.org/jira/browse/HBASE-2418
Project: HBase
Issue Type: Improvement
Components: master, regionserver
Reporter: Patrick Hunt
Assignee: Eugene Koontz
Priority: Critical
Labels: security, zookeeper
Fix For: 0.92.0, 0.94.0
Attachments: HBASE-2418-6.patch, HBASE-2418-6.patch
Some users may run a ZooKeeper cluster in multi tenant mode meaning that
more than one client service would
like to share a single ZooKeeper service instance (cluster). In this case the
client services typically want to protect
their data (ZK znodes) from access by other services (tenants) on the
cluster. Say you are running HBase and Solr
and Neo4j, or multiple HBase instances, etc... having
authentication/authorization on the znodes is important for both
security and helping to ensure that services don't interact negatively (touch
each other's data).
Today HBase does not have support for authentication or authorization. This
should be added to the HBase clients
that are accessing the ZK cluster. In general it means calling addAuthInfo
once after a session is established:
http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String,
byte[])
with a user specific credential, often times this is a shared secret or
certificate. You may be able to statically configure this
in some cases (config string or file to read from), however in my case in
particular you may need to access it programmatically,
which adds complexity as the end user may need to load code into HBase for
accessing the credential.
Secondly you need to specify a non world ACL when interacting with znodes
(create primarily):
http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html
http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html
Feel free to ping the ZooKeeper team if you have questions. It might also be
good to discuss with some
potential end users - in particular regarding how the end user can specify
the credential.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Open (was: Patch Available) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch, HBASE-2418-5.patch, HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: HBASE-2418-6.patch v6 patch with above described change. add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Patch Available (was: Open) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: (was: HBASE-2418-5.patch) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: (was: HBASE-2418-5.patch) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: (was: HBASE-2418-5.patch) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Open (was: Patch Available) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: HBASE-2418-6.patch Rebased patch on latest trunk. add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch, HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Patch Available (was: Open) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch, HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4827) TestAdmin should clean up resources after tests
[ https://issues.apache.org/jira/browse/HBASE-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4827: -- Attachment: HBASE-4827.patch TestAdmin should clean up resources after tests --- Key: HBASE-4827 URL: https://issues.apache.org/jira/browse/HBASE-4827 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-4827.patch TestAdmin creates a new HBaseAdmin for each test case but does not clean up resources afterward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4827) TestAdmin should clean up resources after tests
[ https://issues.apache.org/jira/browse/HBASE-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4827: -- Attachment: HBASE-4827.patch Actual committed change. TestAdmin should clean up resources after tests --- Key: HBASE-4827 URL: https://issues.apache.org/jira/browse/HBASE-4827 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.92.0, 0.94.0, 0.90.5 Attachments: HBASE-4827.patch, HBASE-4827.patch TestAdmin creates a new HBaseAdmin for each test case but does not clean up resources afterward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Open (was: Patch Available) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch, HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Patch Available (was: Open) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-6.patch, HBASE-2418-6.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: HBASE-2418-5.patch add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Hadoop Flags: Reviewed Status: Patch Available (was: Open) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: HBASE-2418-5.patch Missing 'return' add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch, HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Attachment: HBASE-2418-5.patch This time again with --no-prefix add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch, HBASE-2418-5.patch, HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Open (was: Patch Available) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch, HBASE-2418-5.patch, HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2418) add support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2418: -- Status: Patch Available (was: Open) add support for ZooKeeper authentication Key: HBASE-2418 URL: https://issues.apache.org/jira/browse/HBASE-2418 Project: HBase Issue Type: Improvement Components: master, regionserver Reporter: Patrick Hunt Assignee: Eugene Koontz Priority: Critical Labels: security, zookeeper Fix For: 0.92.0 Attachments: HBASE-2418-5.patch, HBASE-2418-5.patch, HBASE-2418-5.patch Some users may run a ZooKeeper cluster in multi tenant mode meaning that more than one client service would like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both security and helping to ensure that services don't interact negatively (touch each other's data). Today HBase does not have support for authentication or authorization. This should be added to the HBase clients that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[]) with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential. Secondly you need to specify a non world ACL when interacting with znodes (create primarily): http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some potential end users - in particular regarding how the end user can specify the credential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4755) HBase based block placement in DFS
[ https://issues.apache.org/jira/browse/HBASE-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4755: -- Affects Version/s: 0.94.0 Proposing for 0.94. HBase based block placement in DFS -- Key: HBASE-4755 URL: https://issues.apache.org/jira/browse/HBASE-4755 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Karthik Ranganathan Assignee: Christopher Gist The feature as is only useful for HBase clusters that care about data locality on regionservers, but this feature can also enable a lot of nice features down the road. The basic idea is as follows: instead of letting HDFS determine where to replicate data (r=3) by place blocks on various regions, it is better to let HBase do so by providing hints to HDFS through the DFS client. That way instead of replicating data at a blocks level, we can replicate data at a per-region level (each region owned by a promary, a secondary and a tertiary regionserver). This is better for 2 things: - Can make region failover faster on clusters which benefit from data affinity - On large clusters with random block placement policy, this helps reduce the probability of data loss The algo is as follows: - Each region in META will have 3 columns which are the preferred regionservers for that region (primary, secondary and tertiary) - Preferred assignment can be controlled by a config knob - Upon cluster start, HMaster will enter a mapping from each region to 3 regionservers (random hash, could use current locality, etc) - The load balancer would assign out regions preferring region assignments to primary over secondary over tertiary over any other node - Periodically (say weekly, configurable) the HMaster would run a locality checked and make sure the map it has for region to regionservers is optimal. Down the road, this can be enhanced to control region placement in the following cases: - Mixed hardware SKU where some regionservers can hold fewer regions - Load balancing across tables where we dont want multiple regions of a table to get assigned to the same regionservers - Multi-tenancy, where we can restrict the assignment of the regions of some table to a subset of regionservers, so an abusive app cannot take down the whole HBase cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4554) Allow set/unset coprocessor table attributes from shell.
[
https://issues.apache.org/jira/browse/HBASE-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4554:
--
Resolution: Fixed
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed latest version of patch. TestShell passes. I slightly edited the help
text English so it is more clear.
Allow set/unset coprocessor table attributes from shell.
Key: HBASE-4554
URL: https://issues.apache.org/jira/browse/HBASE-4554
Project: HBase
Issue Type: Improvement
Components: coprocessors
Reporter: Mingjie Lai
Assignee: Mingjie Lai
Fix For: 0.92.0, 0.94.0
Table/region level coprocessor -- RegionObserver -- can be configured by
setting a HTD's attribute which matches Coprocessor$*.
Current shell -- alter -- cannot support to set/unset a table's arbitrary
attribute. We need it in order to configure region level coprocessors to a
table.
Proposed new shell:
{code}
hbase shell alter 't1', METHOD = 'table_att', COPROCESSOR$1 =
'hdfs://cp/foo.jar|org.apache.hadoop.hbase.sample|1|'
hbase shell describe 't1'
{NAME = 't1', COPROCESSOR$1 =
'hdfs://cp/foo.jar|org.apache.hadoop.hbase.sample|1|', MAX_FILESIZE =
'134217728', ...}
hbase shell alter 't1', METHOD = 'table_att_unset', COPROCESSOR$1
hbase shell describe 't1'
{NAME = 't1', MAX_FILESIZE = '134217728', ...}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4745) LRU Statistics thread should be daemon
[
https://issues.apache.org/jira/browse/HBASE-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4745:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and branch. Thanks Jon.
LRU Statistics thread should be daemon
--
Key: HBASE-4745
URL: https://issues.apache.org/jira/browse/HBASE-4745
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Andrew Purtell
Fix For: 0.92.0, 0.94.0
Attachments: HBASE-4745.patch
Here was from 'HBase 0.92/Hadoop 0.22 test results' discussion on dev@hbase
{code}
LRU Statistics #0 prio=10 tid=0x7f4edc7dd800 nid=0x211a waiting
on condition [0x7f4e631e2000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x7f4e88acc968 (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
{code}
We should make this thread daemon thread.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4684) REST server is leaking ZK connections in 0.90
[ https://issues.apache.org/jira/browse/HBASE-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4684: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch. REST server is leaking ZK connections in 0.90 - Key: HBASE-4684 URL: https://issues.apache.org/jira/browse/HBASE-4684 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: Andrew Purtell Priority: Critical Fix For: 0.90.5 Attachments: HBASE-4684-0.90.patch, HBASE-4684-v2-0.90.patch As reported a month ago, http://search-hadoop.com/m/FD6gmKzrxY1, the REST server is leak ZK connections. Upon investigation I see that TableResource.scanTransformAttrs creates a new HBA per minute per table (when the server is getting requests) but never deletes the connection created in there. There are a bunch of other places where HBAs are created but not cleaned after like SchemaResource, StorageClusterStatusResource, StorageClusterVersionResource, ExistsResource, etc. Those places shouldn't be as leaky under normal circumstances tho. Thanks to Jack Levin for bringing up this issue again when he tried to upgrade. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4740) [bulk load] the HBASE-4552 API can't tell if errors on region server is recoverable or unrecoverable error.
[ https://issues.apache.org/jira/browse/HBASE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4740: -- Priority: Blocker (was: Critical) bq. This is an RPC change, so it would be really good to get this api right before the final 0.92 goes out. Raise to blocker, either fix or undo for release. +1 on fix. [bulk load] the HBASE-4552 API can't tell if errors on region server is recoverable or unrecoverable error. Key: HBASE-4740 URL: https://issues.apache.org/jira/browse/HBASE-4740 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Blocker Fix For: 0.92.0 Running TestHFileOutputFormat more frequently seems to show that it has become flaky. It is difficult to tell if this is because of a unrecoverable failure or a recoverable failure. To make this visiable from test and for users, we need to make a change to bulkload call's interface on HRegionServer. The change should make successful rpcs return true, recoverable failures return false, and unrecoverable failure throw an IOException. This is an RPC change, so it would be really good to get this api right before the final 0.92 goes out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4745) LRU Statistics thread should be daemon
[
https://issues.apache.org/jira/browse/HBASE-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4745:
--
Attachment: HBASE-4745.patch
LRU Statistics thread should be daemon
--
Key: HBASE-4745
URL: https://issues.apache.org/jira/browse/HBASE-4745
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Fix For: 0.92.0
Attachments: HBASE-4745.patch
Here was from 'HBase 0.92/Hadoop 0.22 test results' discussion on dev@hbase
{code}
LRU Statistics #0 prio=10 tid=0x7f4edc7dd800 nid=0x211a waiting
on condition [0x7f4e631e2000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x7f4e88acc968 (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
{code}
We should make this thread daemon thread.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4745) LRU Statistics thread should be daemon
[
https://issues.apache.org/jira/browse/HBASE-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4745:
--
Fix Version/s: 0.94.0
Status: Patch Available (was: Open)
LRU Statistics thread should be daemon
--
Key: HBASE-4745
URL: https://issues.apache.org/jira/browse/HBASE-4745
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Andrew Purtell
Fix For: 0.92.0, 0.94.0
Attachments: HBASE-4745.patch
Here was from 'HBase 0.92/Hadoop 0.22 test results' discussion on dev@hbase
{code}
LRU Statistics #0 prio=10 tid=0x7f4edc7dd800 nid=0x211a waiting
on condition [0x7f4e631e2000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x7f4e88acc968 (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
{code}
We should make this thread daemon thread.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4684) REST server is leaking ZK connections in 0.90
[ https://issues.apache.org/jira/browse/HBASE-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4684: -- Status: Patch Available (was: Open) REST server is leaking ZK connections in 0.90 - Key: HBASE-4684 URL: https://issues.apache.org/jira/browse/HBASE-4684 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: Andrew Purtell Priority: Critical Fix For: 0.90.5 Attachments: HBASE-4684-0.90.patch, HBASE-4684-v2-0.90.patch As reported a month ago, http://search-hadoop.com/m/FD6gmKzrxY1, the REST server is leak ZK connections. Upon investigation I see that TableResource.scanTransformAttrs creates a new HBA per minute per table (when the server is getting requests) but never deletes the connection created in there. There are a bunch of other places where HBAs are created but not cleaned after like SchemaResource, StorageClusterStatusResource, StorageClusterVersionResource, ExistsResource, etc. Those places shouldn't be as leaky under normal circumstances tho. Thanks to Jack Levin for bringing up this issue again when he tried to upgrade. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4684) REST server is leaking ZK connections in 0.90
[
https://issues.apache.org/jira/browse/HBASE-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-4684:
--
Attachment: HBASE-4684-v2-0.90.patch
Attached patch v2 that shares a single HBA for all needs.
{quote}
bq. In its current form, the patch will need some modifications to work with
0.90 because HBA doesn't have a close method.
I'm almost positive I based this patch on the head of 0.90 branch.
{quote}
HBASE-4508 backported changes to make HBA and others closeable from HBASE-3777.
REST server is leaking ZK connections in 0.90
-
Key: HBASE-4684
URL: https://issues.apache.org/jira/browse/HBASE-4684
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Assignee: Andrew Purtell
Priority: Critical
Fix For: 0.90.5
Attachments: HBASE-4684-0.90.patch, HBASE-4684-v2-0.90.patch
As reported a month ago, http://search-hadoop.com/m/FD6gmKzrxY1, the REST
server is leak ZK connections. Upon investigation I see that
TableResource.scanTransformAttrs creates a new HBA per minute per table (when
the server is getting requests) but never deletes the connection created in
there.
There are a bunch of other places where HBAs are created but not cleaned
after like SchemaResource, StorageClusterStatusResource,
StorageClusterVersionResource, ExistsResource, etc. Those places shouldn't be
as leaky under normal circumstances tho.
Thanks to Jack Levin for bringing up this issue again when he tried to
upgrade.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4733) Rest client does not close zookeeper sessions (leaking sessions for each GET or PUT)
[ https://issues.apache.org/jira/browse/HBASE-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4733: -- Resolution: Duplicate Status: Resolved (was: Patch Available) Dup of HBASE-4684. Rest client does not close zookeeper sessions (leaking sessions for each GET or PUT) Key: HBASE-4733 URL: https://issues.apache.org/jira/browse/HBASE-4733 Project: HBase Issue Type: Bug Components: rest Affects Versions: 0.90.4 Environment: Fedora 13. Reporter: jack levin Labels: newbie Fix For: 0.90.4 Attachments: my.patch This will appear in the log once the zookeeper connection/session leaking will grow to 2000, zookeeper won't be able to accept any more connections causing REST RPC calls to fail, here is the log when the problem is in progress: 2011-10-26 01:35:49,270 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.3 - max is 2000 2011-10-26 01:35:49,300 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.4 - max is 2000 2011-10-26 01:35:49,317 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.4 - max is 2000 2011-10-26 01:35:49,321 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.3 - max is 2000 2011-10-26 01:35:49,323 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.101.255.3 - max is 2000 2011-10-26 01:35:49,367 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.101.255.2 - max is 2000 2011-10-26 01:35:49,375 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.4 - max is 2000 2011-10-26 01:35:49,382 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.2 - max is 2000 2011-10-26 01:35:49,404 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.101.255.2 - max is 2000 2011-10-26 01:35:49,429 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.2 - max is 2000 2011-10-26 01:35:49,439 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.3 - max is 2000 2011-10-26 01:35:49,469 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.3 - max is 2000 2011-10-26 01:35:49,489 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.101.255.3 - max is 2000 2011-10-26 01:35:49,501 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.101.255.2 - max is 2000 2011-10-26 01:35:49,584 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections from /10.102.255.2 - max is 2000 After the fix, the log looks much better, and we can observe zookeeper connections closing after every RPC call: 2011-11-02 15:50:14,339 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection from /10.101.255.2:37225 2011-11-02 15:50:14,339 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting to establish new session at /10.101.255.2:37225 2011-11-02 15:50:14,340 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session 0x3352857cb1f19f with negotiated timeout 18 for client /10.101.255.2:37225 2011-11-02 15:50:14,363 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /10.101.255.2:37225 which had sessionid 0x3352857cb1f19f 2011-11-02 15:50:14,723 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection from /10.101.255.2:38089 2011-11-02 15:50:14,723 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting to establish new session at /10.101.255.2:38089 2011-11-02 15:50:14,725 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session 0x3352857cb1f1a0 with negotiated timeout 18 for client /10.101.255.2:38089 2011-11-02 15:50:14,771 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /10.101.255.2:38089 which had sessionid 0x3352857cb1f1a0 2011-11-02 15:50:16,326 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection from /10.101.255.3:34085 2011-11-02 15:50:16,326 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting to establish new session at /10.101.255.3:34085 2011-11-02 15:50:16,328 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session 0x3352857cb1f1a1 with negotiated timeout 18 for client /10.101.255.3:34085 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators:
[jira] [Updated] (HBASE-4070) [Coprocessors] Improve region server metrics to report loaded coprocessors to master
[ https://issues.apache.org/jira/browse/HBASE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4070: -- Resolution: Fixed Fix Version/s: 0.94.0 0.92.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.92 [Coprocessors] Improve region server metrics to report loaded coprocessors to master Key: HBASE-4070 URL: https://issues.apache.org/jira/browse/HBASE-4070 Project: HBase Issue Type: Improvement Affects Versions: 0.90.3 Reporter: Mingjie Lai Assignee: Eugene Koontz Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4070.patch, HBASE-4070.patch, HBASE-4070.patch, HBASE-4070.patch, master-web-ui.jpg, rs-status-web-ui.jpg HBASE-3512 is about listing loaded cp classes at shell. To make it more generic, we need a way to report this piece of information from region to master (or just at region server level). So later on, we can display the loaded class names at shell as well as web console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3512) Coprocessors: Shell support for listing currently loaded coprocessor set
[ https://issues.apache.org/jira/browse/HBASE-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-3512: -- Resolution: Fixed Fix Version/s: 0.94.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.92 branch. Coprocessors: Shell support for listing currently loaded coprocessor set Key: HBASE-3512 URL: https://issues.apache.org/jira/browse/HBASE-3512 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: Andrew Purtell Assignee: Eugene Koontz Fix For: 0.92.0, 0.94.0 Attachments: HBASE-3512-only.patch, HBASE-3512-only.patch, HBASE-3512.patch, HBASE-3512.patch, HBASE-3512.patch, hbase-shell-session.txt Add support to the shell for listing the coprocessors loaded globally on the regionserver and those loaded on a per-table basis. Perhaps by extending the 'status' command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-4477: -- Attachment: coprocessorPut2.txt +1 Attached updated patch 'coprocessorPut2.txt which removes some unused locals. I'm curious if passing the cluster ID to internalPut and internalDelete is necessary? They can be retrieved from the Put or Delete, or the Put or Delete can be modified. I don't know enough about replication there: The put or delete gets written to the WAL with one UUID but internally contains another? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
