[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473034#comment-13473034 ] Lars Hofhansl commented on HBASE-6942: -- Maybe you could a client side code snippet to the Javadoc of the endpoint? If not, the test is documentation enough about how to use it, I think. > Endpoint implementation for bulk delete > --- > > Key: HBASE-6942 > URL: https://issues.apache.org/jira/browse/HBASE-6942 > Project: HBase > Issue Type: Improvement > Components: Coprocessors, Performance >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.94.3, 0.96.0 > > Attachments: HBASE-6942.patch > > > We can provide an end point implementation for doing a bulk delete (based on > a scan) at the server side. This can reduce the time taken for such an > operation as right now it need to do a scan to client and issue delete(s) > using rowkeys. > Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-6942: -- Fix Version/s: 0.96.0 0.94.3 > Endpoint implementation for bulk delete > --- > > Key: HBASE-6942 > URL: https://issues.apache.org/jira/browse/HBASE-6942 > Project: HBase > Issue Type: Improvement > Components: Coprocessors, Performance >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.94.3, 0.96.0 > > Attachments: HBASE-6942.patch > > > We can provide an end point implementation for doing a bulk delete (based on > a scan) at the server side. This can reduce the time taken for such an > operation as right now it need to do a scan to client and issue delete(s) > using rowkeys. > Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473032#comment-13473032 ] Anoop Sam John commented on HBASE-6942: --- Thanks Lars and Ted for having a look at the patch @Ted You mean when the deletion in one region fails after some batches, that count wont get returned and so wont get added up to the total count. Correct? Yes I realized that. As per Lars suggestion may be wont be giving a client class also. Just the endpoint implementation. Still Endpoint need to return how many rows it is deleted. Also IOE if any while deletion. Will change. > Endpoint implementation for bulk delete > --- > > Key: HBASE-6942 > URL: https://issues.apache.org/jira/browse/HBASE-6942 > Project: HBase > Issue Type: Improvement > Components: Coprocessors, Performance >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-6942.patch > > > We can provide an end point implementation for doing a bulk delete (based on > a scan) at the server side. This can reduce the time taken for such an > operation as right now it need to do a scan to client and issue delete(s) > using rowkeys. > Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473027#comment-13473027
]
Lars Hofhansl commented on HBASE-6968:
--
Looks like the first two changes are (in one form or the other) in 0.94+
already.
The 0.94 MemStoreFlusher.reclaimMemStoreMemory looks bad, though. We are taking
a lock and are awaiting a condition *inside* a synchronized method... It seems
we can remove the synchronized there. The code inside the method does not need
it and there are no other synchronized methods (or synchronized(this) blocks)
in the class (which is also why it is not noticeably bad, because nothing is
locked out).
> Several HBase write perf improvement
>
>
> Key: HBASE-6968
> URL: https://issues.apache.org/jira/browse/HBASE-6968
> Project: HBase
> Issue Type: Improvement
>Reporter: Liyin Tang
>
> Here are 2 hbase write performance improvements recently:
> 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
> creating a HBaseConfiguraiton object, it would parse the xml configuration
> files from disk, which is not cheap operation.
> In HLog.java:
> orig:
> {code:title=HLog.java}
> newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
> {code}
> new:
> {code}
> newWriter = createWriter(fs, newPath, conf);
> {code}
> 2) Change 2 hotspot synchronized functions into double locking pattern. So it
> shall remove the synchronization overhead in the normal case.
> orig:
> {code:title=HBaseRpcMetrics.java}
> public synchronized void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> m = create(name);
> }
> m.inc(amt);
> }
> {code}
> new:
> {code}
> public void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> synchronized (this) {
> if ((m = get(name)) == null) {
> m = create(name);
> }
> }
> }
> m.inc(amt);
> }
> {code}
> =
> orig:
> {code:title=MemStoreFlusher.java}
> public synchronized void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> {code}
> new:
> {code}
> public void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> private synchronized void flushSomeRegions() {
> if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
> return; // double check the global memstore size inside of the
> synchronized block.
> }
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473029#comment-13473029 ] Lars Hofhansl commented on HBASE-6733: -- I think we want this in 0.94 as well. > [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] > --- > > Key: HBASE-6733 > URL: https://issues.apache.org/jira/browse/HBASE-6733 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6733-1.patch, 6733-2.patch, 6733-3.patch > > > The failure is in TestReplication.queueFailover (fails due to unreplicated > rows). I have come across two problems: > 1. The sleepMultiplier is not properly reset when the currentPath is changed > (in ReplicationSource.java). > 2. ReplicationExecutor sometime removes files to replicate from the queue too > early, resulting in corresponding edits missing. Here the problem is due to > the fact the log-file length that the replication executor finds is not the > most updated one, and hence it doesn't read anything from there, and > ultimately, when there is a log roll, the replication-queue gets a new entry, > and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-6785: --- Attachment: 6785-simplified-pb1.patch Passing the patch through hadoopqa (this is the same patch that I just uploaded on RB). > Convert AggregateProtocol to protobuf defined coprocessor service > - > > Key: HBASE-6785 > URL: https://issues.apache.org/jira/browse/HBASE-6785 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Gary Helmling >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6785-simplified-pb1.patch, Aggregate.proto, > Aggregate.proto > > > With coprocessor endpoints now exposed as protobuf defined services, we > should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-6785: --- Status: Patch Available (was: Open) > Convert AggregateProtocol to protobuf defined coprocessor service > - > > Key: HBASE-6785 > URL: https://issues.apache.org/jira/browse/HBASE-6785 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Gary Helmling >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6785-simplified-pb1.patch, Aggregate.proto, > Aggregate.proto > > > With coprocessor endpoints now exposed as protobuf defined services, we > should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473012#comment-13473012 ] Hudson commented on HBASE-6733: --- Integrated in HBase-TRUNK #3440 (See [https://builds.apache.org/job/HBase-TRUNK/3440/]) HBASE-6733 TestReplication.queueFailover occasionally fails [Part-2] (Devaraj Das via JD) (Revision 1396463) Result = SUCCESS jdcryans : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java > [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] > --- > > Key: HBASE-6733 > URL: https://issues.apache.org/jira/browse/HBASE-6733 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6733-1.patch, 6733-2.patch, 6733-3.patch > > > The failure is in TestReplication.queueFailover (fails due to unreplicated > rows). I have come across two problems: > 1. The sleepMultiplier is not properly reset when the currentPath is changed > (in ReplicationSource.java). > 2. ReplicationExecutor sometime removes files to replicate from the queue too > early, resulting in corresponding edits missing. Here the problem is due to > the fact the log-file length that the replication executor finds is not the > most updated one, and hence it doesn't read anything from there, and > ultimately, when there is a log roll, the replication-queue gets a new entry, > and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
[ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473005#comment-13473005 ] Devaraj Das commented on HBASE-6758: Hey [~jdcryans], this patch doesn't change that behavior at all (new log is put up in ZK before the log is being written to, and blocks talking to ZK..). This patch only changes the postLogRoll placement and that deterministically ensures the previous log file is really closed before enqueuing the new log for replication. The code changes in the replicator thread (ReplicationSource.java) makes sure that the entire iteration of the loop "sees" a closed log file at least once (and hence takes care of the problem reported in the jira). > [replication] The replication-executor should make sure the file that it is > replicating is closed before declaring success on that file > --- > > Key: HBASE-6758 > URL: https://issues.apache.org/jira/browse/HBASE-6758 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, > 6758-trunk-1.patch, 6758-trunk-2.patch, 6758-trunk-3.patch, > TEST-org.apache.hadoop.hbase.replication.TestReplication.xml > > > I have seen cases where the replication-executor would lose data to replicate > since the file hasn't been closed yet. Upon closing, the new data becomes > visible. Before that happens the ZK node shouldn't be deleted in > ReplicationSourceManager.logPositionAndCleanOldLogs. Changes need to be made > in ReplicationSource.processEndOfFile as well (currentPath related). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
[ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472982#comment-13472982 ] Jean-Daniel Cryans commented on HBASE-6758: --- The last time I played around postLogRoll in HBASE-3515, I found that we must ensure that we have that log up in ZK before we start writing to it because it would be possible for writers to append and at the same time not be able to add the log in ZK because the session timed out. The current code blocks talking to ZK. > [replication] The replication-executor should make sure the file that it is > replicating is closed before declaring success on that file > --- > > Key: HBASE-6758 > URL: https://issues.apache.org/jira/browse/HBASE-6758 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, > 6758-trunk-1.patch, 6758-trunk-2.patch, 6758-trunk-3.patch, > TEST-org.apache.hadoop.hbase.replication.TestReplication.xml > > > I have seen cases where the replication-executor would lose data to replicate > since the file hasn't been closed yet. Upon closing, the new data becomes > visible. Before that happens the ZK node shouldn't be deleted in > ReplicationSourceManager.logPositionAndCleanOldLogs. Changes need to be made > in ReplicationSource.processEndOfFile as well (currentPath related). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472978#comment-13472978
]
ramkrishna.s.vasudevan commented on HBASE-6968:
---
Nice one.
> Several HBase write perf improvement
>
>
> Key: HBASE-6968
> URL: https://issues.apache.org/jira/browse/HBASE-6968
> Project: HBase
> Issue Type: Improvement
>Reporter: Liyin Tang
>
> Here are 2 hbase write performance improvements recently:
> 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
> creating a HBaseConfiguraiton object, it would parse the xml configuration
> files from disk, which is not cheap operation.
> In HLog.java:
> orig:
> {code:title=HLog.java}
> newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
> {code}
> new:
> {code}
> newWriter = createWriter(fs, newPath, conf);
> {code}
> 2) Change 2 hotspot synchronized functions into double locking pattern. So it
> shall remove the synchronization overhead in the normal case.
> orig:
> {code:title=HBaseRpcMetrics.java}
> public synchronized void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> m = create(name);
> }
> m.inc(amt);
> }
> {code}
> new:
> {code}
> public void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> synchronized (this) {
> if ((m = get(name)) == null) {
> m = create(name);
> }
> }
> }
> m.inc(amt);
> }
> {code}
> =
> orig:
> {code:title=MemStoreFlusher.java}
> public synchronized void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> {code}
> new:
> {code}
> public void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> private synchronized void flushSomeRegions() {
> if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
> return; // double check the global memstore size inside of the
> synchronized block.
> }
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6971) .META. directory does not contain .tableinfo serialization
Enis Soztutar created HBASE-6971: Summary: .META. directory does not contain .tableinfo serialization Key: HBASE-6971 URL: https://issues.apache.org/jira/browse/HBASE-6971 Project: HBase Issue Type: Bug Components: Filesystem Integration, master Affects Versions: 0.94.3, 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar This is not very critical, but I've noticed that we do not serialize the HTableDescriptor to /.META./.tableinfo, although we do it for ROOT. MasterFileSystem.createRootTableInfo() should probably have an META equivalent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472977#comment-13472977
]
ramkrishna.s.vasudevan commented on HBASE-6853:
---
Ok, I shall commit this later today with a wait time on test.
@Lars/Stack
You mind committing it?
> IllegalArgument Exception is thrown when an empty region is spliitted.
> --
>
> Key: HBASE-6853
> URL: https://issues.apache.org/jira/browse/HBASE-6853
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: Priyadarshini
> Fix For: 0.94.2, 0.96.0
>
> Attachments: HBASE-6853_2_splitsuccess.patch,
> HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
>
>
> This is w.r.t a mail sent in the dev mail list.
> Empty region split should be handled gracefully. Either we should not allow
> the split to happen if we know that the region is empty or we should allow
> the split to happen by setting the no of threads to the thread pool executor
> as 1.
> {code}
> int nbFiles = hstoreFilesToSplit.size();
> ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
> builder.setNameFormat("StoreFileSplitter-%1$d");
> ThreadFactory factory = builder.build();
> ThreadPoolExecutor threadPool =
> (ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
> List> futures = new ArrayList>(nbFiles);
> {code}
> Here the nbFiles needs to be a non zero positive value.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-6733: -- Resolution: Fixed Fix Version/s: (was: 0.92.3) 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk, thanks for your patch and your patience Devaraj. > [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] > --- > > Key: HBASE-6733 > URL: https://issues.apache.org/jira/browse/HBASE-6733 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6733-1.patch, 6733-2.patch, 6733-3.patch > > > The failure is in TestReplication.queueFailover (fails due to unreplicated > rows). I have come across two problems: > 1. The sleepMultiplier is not properly reset when the currentPath is changed > (in ReplicationSource.java). > 2. ReplicationExecutor sometime removes files to replicate from the queue too > early, resulting in corresponding edits missing. Here the problem is due to > the fact the log-file length that the replication executor finds is not the > most updated one, and hence it doesn't read anything from there, and > ultimately, when there is a log roll, the replication-queue gets a new entry, > and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6302) Document how to run integration tests
[ https://issues.apache.org/jira/browse/HBASE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472975#comment-13472975 ] stack commented on HBASE-6302: -- Any luck w/ this one Enis? > Document how to run integration tests > - > > Key: HBASE-6302 > URL: https://issues.apache.org/jira/browse/HBASE-6302 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: stack >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.96.0 > > Attachments: HBASE-6302_v1.patch > > > HBASE-6203 has attached the old IT doc with some mods. When we figure how > ITs are to be run, update it and apply the documentation under this issue. > Making a blocker against 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472973#comment-13472973 ] stack commented on HBASE-6969: -- As written, at end of tests, won't your passed in zkcluster be shutdown? That is probably not what you want? Should there be a getMiniZookeeperCluster to answer your added setMiniZookeeperCluster? Is baseZKCluster necessary? Why not just an internal flag which has whether or not HBaseTestingUtility started the zk cluster? If we didn't start it, we shouldn't stop it on the way out? Otherwise, looks like useful functionality to add. Thanks Micah. > Allow for injecting MiniZookeeperCluster instance > - > > Key: HBASE-6969 > URL: https://issues.apache.org/jira/browse/HBASE-6969 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.1 >Reporter: Micah Whitacre > Attachments: hbase-6969.patch > > > While not an official part of the HBase API, it'd be nice if > HBaseTestingUtility allowed for consumer to inject their own instance of the > MiniZookeeperCluster when executing tests. > Currently there is no way to control how the ZK instance is started which > we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). > Or if you want to control which specific port it starts on vs random. > Allowing consumers to inject an instance (unstarted) gives them the freedom > to implement their own without having to fork the entire HBaseTestingUtility > class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472969#comment-13472969
]
stack commented on HBASE-6853:
--
+1 on committing addendum and +1 on adding a wait time on the test.
> IllegalArgument Exception is thrown when an empty region is spliitted.
> --
>
> Key: HBASE-6853
> URL: https://issues.apache.org/jira/browse/HBASE-6853
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: Priyadarshini
> Fix For: 0.94.2, 0.96.0
>
> Attachments: HBASE-6853_2_splitsuccess.patch,
> HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
>
>
> This is w.r.t a mail sent in the dev mail list.
> Empty region split should be handled gracefully. Either we should not allow
> the split to happen if we know that the region is empty or we should allow
> the split to happen by setting the no of threads to the thread pool executor
> as 1.
> {code}
> int nbFiles = hstoreFilesToSplit.size();
> ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
> builder.setNameFormat("StoreFileSplitter-%1$d");
> ThreadFactory factory = builder.build();
> ThreadPoolExecutor threadPool =
> (ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
> List> futures = new ArrayList>(nbFiles);
> {code}
> Here the nbFiles needs to be a non zero positive value.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-6970:
-
Description:
We just ran into a strange issue where could neither start nor stop services
with hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "$@" $startStop > "$logout" 2>&1 < /dev/null &
echo $! > $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has the
wrong pid in it.
Edit: Spelling and more spelling errors.
was:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "$@" $startStop > "$logout" 2>&1 < /dev/null &
echo $! > $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has he wrong
pid in it.
Edit: Spelling
> hbase-deamon.sh creates/updates pid file even when that start failed.
> -
>
> Key: HBASE-6970
> URL: https://issues.apache.org/jira/browse/HBASE-6970
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>
> We just ran into a strange issue where could neither start nor stop services
> with hbase-deamon.sh.
> The problem is this:
> {code}
> nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
> --config "${HBASE_CONF_DIR}" \
> $command "$@" $startStop > "$logout" 2>&1 < /dev/null &
> echo $! > $pid
> {code}
> So the pid file is created or updated even when the start of the service
> failed. The next stop command will then fail, because the pid file has the
> wrong pid in it.
> Edit: Spelling and more spelling errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-6970:
-
Description:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "$@" $startStop > "$logout" 2>&1 < /dev/null &
echo $! > $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has he wrong
pid in it.
Edit: Spelling
was:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "$@" $startStop > "$logout" 2>&1 < /dev/null &
echo $! > $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
> hbase-deamon.sh creates/updates pid file even when that start failed.
> -
>
> Key: HBASE-6970
> URL: https://issues.apache.org/jira/browse/HBASE-6970
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>
> We just ran into a strange issue where could start or stop services with
> hbase-deamon.sh.
> The problem is this:
> {code}
> nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
> --config "${HBASE_CONF_DIR}" \
> $command "$@" $startStop > "$logout" 2>&1 < /dev/null &
> echo $! > $pid
> {code}
> So the pid file is created or updated even when the start of the service
> failed. The next stop command will then fail, because the pid file has he
> wrong pid in it.
> Edit: Spelling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5525) Truncate and preserve region boundaries option
[
https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472967#comment-13472967
]
Hadoop QA commented on HBASE-5525:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548513/HBASE-5525.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSide
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//console
This message is automatically generated.
> Truncate and preserve region boundaries option
> --
>
> Key: HBASE-5525
> URL: https://issues.apache.org/jira/browse/HBASE-5525
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: Kevin Odell
> Labels: newbie, noob
> Fix For: 0.96.0
>
> Attachments: HBASE-5525.patch
>
>
> A tool that would be useful for testing (and maybe in prod too) would be a
> truncate option to keep the current region boundaries. Right now what you
> have to do is completely kill the table and recreate it with the correct
> regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472964#comment-13472964
]
Jean-Daniel Cryans commented on HBASE-6970:
---
lol I'm looking at the same problem and worrying about the same thing. At least
changing this behavior would help things like supervisord.
> hbase-deamon.sh creates/updates pid file even when that start failed.
> -
>
> Key: HBASE-6970
> URL: https://issues.apache.org/jira/browse/HBASE-6970
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>
> We just ran into a strange issue where could start or stop services with
> hbase-deamon.sh.
> The problem is this:
> {code}
> nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
> --config "${HBASE_CONF_DIR}" \
> $command "$@" $startStop > "$logout" 2>&1 < /dev/null &
> echo $! > $pid
> {code}
> So the pid file is create or update even the start of the service failed. The
> next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472959#comment-13472959
]
Lars Hofhansl commented on HBASE-6970:
--
Also we found that the stop command does not return a failed exit code (<> 0),
if the stop fails because the running process was not found.
I am not sure how risky changing this behavior is for current installation that
might rely on the this behavior.
> hbase-deamon.sh creates/updates pid file even when that start failed.
> -
>
> Key: HBASE-6970
> URL: https://issues.apache.org/jira/browse/HBASE-6970
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>
> We just ran into a strange issue where could start or stop services with
> hbase-deamon.sh.
> The problem is this:
> {code}
> nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
> --config "${HBASE_CONF_DIR}" \
> $command "$@" $startStop > "$logout" 2>&1 < /dev/null &
> echo $! > $pid
> {code}
> So the pid file is create or update even the start of the service failed. The
> next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
Lars Hofhansl created HBASE-6970:
Summary: hbase-deamon.sh creates/updates pid file even when that
start failed.
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "$@" $startStop > "$logout" 2>&1 < /dev/null &
echo $! > $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472958#comment-13472958
]
Lars Hofhansl commented on HBASE-6853:
--
Should we commit the addendum? (Also should we limit the waiting, or just wait
until the test is killed?)
> IllegalArgument Exception is thrown when an empty region is spliitted.
> --
>
> Key: HBASE-6853
> URL: https://issues.apache.org/jira/browse/HBASE-6853
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: Priyadarshini
> Fix For: 0.94.2, 0.96.0
>
> Attachments: HBASE-6853_2_splitsuccess.patch,
> HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
>
>
> This is w.r.t a mail sent in the dev mail list.
> Empty region split should be handled gracefully. Either we should not allow
> the split to happen if we know that the region is empty or we should allow
> the split to happen by setting the no of threads to the thread pool executor
> as 1.
> {code}
> int nbFiles = hstoreFilesToSplit.size();
> ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
> builder.setNameFormat("StoreFileSplitter-%1$d");
> ThreadFactory factory = builder.build();
> ThreadPoolExecutor threadPool =
> (ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
> List> futures = new ArrayList>(nbFiles);
> {code}
> Here the nbFiles needs to be a non zero positive value.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472946#comment-13472946 ] Micah Whitacre commented on HBASE-6969: --- It should be noted I made the patch based off of trunk but I would like it patched back to 0.92.1 (the version I'm currently on). Trunk has a number of improvements (e.g configuration for the port) but I'm on CDH4.1 which is 0.92.1. > Allow for injecting MiniZookeeperCluster instance > - > > Key: HBASE-6969 > URL: https://issues.apache.org/jira/browse/HBASE-6969 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.1 >Reporter: Micah Whitacre > Attachments: hbase-6969.patch > > > While not an official part of the HBase API, it'd be nice if > HBaseTestingUtility allowed for consumer to inject their own instance of the > MiniZookeeperCluster when executing tests. > Currently there is no way to control how the ZK instance is started which > we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). > Or if you want to control which specific port it starts on vs random. > Allowing consumers to inject an instance (unstarted) gives them the freedom > to implement their own without having to fork the entire HBaseTestingUtility > class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell updated HBASE-5525: --- Fix Version/s: 0.96.0 Affects Version/s: 0.96.0 Status: Patch Available (was: Open) > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.96.0 >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Labels: newbie, noob > Fix For: 0.96.0 > > Attachments: HBASE-5525.patch > > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell updated HBASE-5525: --- Attachment: HBASE-5525.patch So I added a new command truncate_preserve, it seemed to fit with the flow of the other commands: disable_all disable_peer enable_all enable_peer alter alter_async alter_status I am not too attached to the name so I am open for suggestions. I tested this out by altering the ruby code on my box and it worked. Let me know if you see anything strange. > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Labels: newbie, noob > Attachments: HBASE-5525.patch > > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micah Whitacre updated HBASE-6969: -- Attachment: hbase-6969.patch Simple patch file that just allows the HBaseTestingUtility to access a base MiniZKCluster instance. > Allow for injecting MiniZookeeperCluster instance > - > > Key: HBASE-6969 > URL: https://issues.apache.org/jira/browse/HBASE-6969 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.1 >Reporter: Micah Whitacre > Attachments: hbase-6969.patch > > > While not an official part of the HBase API, it'd be nice if > HBaseTestingUtility allowed for consumer to inject their own instance of the > MiniZookeeperCluster when executing tests. > Currently there is no way to control how the ZK instance is started which > we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). > Or if you want to control which specific port it starts on vs random. > Allowing consumers to inject an instance (unstarted) gives them the freedom > to implement their own without having to fork the entire HBaseTestingUtility > class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
Micah Whitacre created HBASE-6969: - Summary: Allow for injecting MiniZookeeperCluster instance Key: HBASE-6969 URL: https://issues.apache.org/jira/browse/HBASE-6969 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.1 Reporter: Micah Whitacre While not an official part of the HBase API, it'd be nice if HBaseTestingUtility allowed for consumer to inject their own instance of the MiniZookeeperCluster when executing tests. Currently there is no way to control how the ZK instance is started which we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). Or if you want to control which specific port it starts on vs random. Allowing consumers to inject an instance (unstarted) gives them the freedom to implement their own without having to fork the entire HBaseTestingUtility class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472937#comment-13472937 ] Hudson commented on HBASE-6316: --- Integrated in HBase-TRUNK #3439 (See [https://builds.apache.org/job/HBase-TRUNK/3439/]) HBASE-6316 Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting (Revision 1396428) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/Reference.java * /hbase/trunk/hbase-server/src/test/data/a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestReference.java > Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting > - > > Key: HBASE-6316 > URL: https://issues.apache.org/jira/browse/HBASE-6316 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, > a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c > > > Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 > to start over data written by a 0.94. Need to fix it so don't have to do > this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472932#comment-13472932
]
Enis Soztutar commented on HBASE-6929:
--
@Stack, I think -Dclassifier causes the problem.
{code}
mvn clean install -Dhadoop.profile=2.0
{code}
Works for me on 0.94. There should be a pom.xml change for deploying the local
jar with a classifier. I'll look into that.
@Lars, I did not see an umbrella issue, but I think we have to compile a list,
and see what comes out of it. If I have some time by the end of week, I'll try
to do that.
> Publish Hbase 0.94 artifacts build against hadoop-2.0
> -
>
> Key: HBASE-6929
> URL: https://issues.apache.org/jira/browse/HBASE-6929
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 0.94.2
>Reporter: Enis Soztutar
>
> Downstream projects (flume, hive, pig, etc) depends on hbase, but since the
> hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot
> depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should
> also push hbase jars build with hadoop2.0 profile into maven, possibly with
> version string like 0.94.2-hadoop2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472902#comment-13472902 ] Hudson commented on HBASE-6940: --- Integrated in HBase-TRUNK #3438 (See [https://builds.apache.org/job/HBase-TRUNK/3438/]) HBASE-6940 Enable GC logging by default (Revision 1396399) Result = FAILURE tedyu : Files : * /hbase/trunk/conf/hbase-env.sh > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Assignee: Ted Yu >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472901#comment-13472901 ] Hudson commented on HBASE-6953: --- Integrated in HBase-TRUNK #3438 (See [https://builds.apache.org/job/HBase-TRUNK/3438/]) HBASE-6953 Incorrect javadoc description of HFileOutputFormat regarding multiple column families (Revision 1396383) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java > Incorrect javadoc description of HFileOutputFormat regarding multiple column > families > - > > Key: HBASE-6953 > URL: https://issues.apache.org/jira/browse/HBASE-6953 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Gabriel Reid >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6953.patch > > > The javadoc for HFileOutputFormat states that the class does not support > writing multiple column families; however, this hasn't been the case since > HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472897#comment-13472897 ] Hudson commented on HBASE-6953: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6953 Incorrect javadoc description of HFileOutputFormat regarding multiple column families (Revision 1396383) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java > Incorrect javadoc description of HFileOutputFormat regarding multiple column > families > - > > Key: HBASE-6953 > URL: https://issues.apache.org/jira/browse/HBASE-6953 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Gabriel Reid >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6953.patch > > > The javadoc for HFileOutputFormat states that the class does not support > writing multiple column families; however, this hasn't been the case since > HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472898#comment-13472898 ] Hudson commented on HBASE-6940: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6940 Enable GC logging by default (Revision 1396399) Result = FAILURE tedyu : Files : * /hbase/trunk/conf/hbase-env.sh > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Assignee: Ted Yu >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472895#comment-13472895 ] Hudson commented on HBASE-6961: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6961 In place scripts fail if mvn install hasn't been run (Revision 1396307) Result = FAILURE eclark : Files : * /hbase/trunk/bin/hbase > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472896#comment-13472896 ] Hudson commented on HBASE-6316: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6316 Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting (Revision 1396428) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/Reference.java * /hbase/trunk/hbase-server/src/test/data/a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestReference.java > Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting > - > > Key: HBASE-6316 > URL: https://issues.apache.org/jira/browse/HBASE-6316 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, > a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c > > > Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 > to start over data written by a 0.94. Need to fix it so don't have to do > this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6316. -- Resolution: Fixed Assignee: stack Hadoop Flags: Reviewed Committed to trunk > Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting > - > > Key: HBASE-6316 > URL: https://issues.apache.org/jira/browse/HBASE-6316 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, > a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c > > > Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 > to start over data written by a 0.94. Need to fix it so don't have to do > this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472887#comment-13472887 ] stack commented on HBASE-6945: -- [~nkeywal] You should be able to admin now (I added you as administrator) [~kumarr] Is the OSMXBean missing from your patch? You forgot to svn/git add it? > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: ResourceChecker.patch > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HBASE-6968:
--
Description:
Here are 2 hbase write performance improvements recently:
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
was:
Here are 2 hbase write performance improvements recently found out.
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
> Several HBase write perf improvement
>
>
> Key: HBASE-6968
> URL: https://issues.apache.org/jira/browse/HBASE-6968
> Project: HBase
> Issue Type: Improvement
>Reporter: Liyin Tang
>
> Here are 2 hbase write performance improvements recently:
> 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
> creating a HBaseConfiguraiton object, it would parse the xml configuration
> files from disk, which is not cheap operation.
> In HLog.java:
> orig:
> {code:title=HLog.java}
> newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
> {code}
> new:
> {code}
> newWriter = createWriter(fs, newPath, conf);
> {code}
> 2) Change 2 hotspot synchronized functions into double locking pattern. So it
> shall remove the synchronization overhead in the normal case.
> orig:
> {code:title=HBaseRpcMetrics.java}
> public synchronized void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> m = cr
[jira] [Updated] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6316: - Attachment: 6316v3.txt Here is what I committed. Thanks for the review Ted. The else was in previous code but I fixed it (Added WARNING about return in middle of a method). > Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting > - > > Key: HBASE-6316 > URL: https://issues.apache.org/jira/browse/HBASE-6316 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, > a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c > > > Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 > to start over data written by a 0.94. Need to fix it so don't have to do > this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HBASE-6968:
--
Description:
Here are 2 hbase write performance improvements recently found out.
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
was:
There are two improvements in this jira:
1) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
2) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
> Several HBase write perf improvement
>
>
> Key: HBASE-6968
> URL: https://issues.apache.org/jira/browse/HBASE-6968
> Project: HBase
> Issue Type: Improvement
>Reporter: Liyin Tang
>
> Here are 2 hbase write performance improvements recently found out.
> 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
> creating a HBaseConfiguraiton object, it would parse the xml configuration
> files from disk, which is not cheap operation.
> In HLog.java:
> orig:
> {code:title=HLog.java}
> newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
> {code}
> new:
> {code}
> newWriter = createWriter(fs, newPath, conf);
> {code}
> 2) Change 2 hotspot synchronized functions into double locking pattern. So it
> shall remove the synchronization overhead in the normal case.
> orig:
> {code:title=HBaseRpcMetrics.java}
> public synchronized void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> m = create(name);
> }
> m.inc(amt);
> }
> {code}
> new:
> {code}
> public void inc(String name, int amt) {
> MetricsTimeVaryingRate m = get(name);
> if (m == null) {
> synchronized (this) {
> if ((m = get(name)) == null) {
> m = create(name);
> }
> }
> }
> m.inc(amt);
> }
> {code}
> =
> orig:
> {code:title=MemStoreFlusher.java}
> public synchronized void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> {code}
> new:
> {code}
> public void reclaimMemStoreMemory() {
> if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {
> flushSomeRegions();
> }
> }
> private synchronized void flushSomeRegions() {
> if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {
> return; // double check the global memstore size inside of the
> synchronized block.
> }
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6968) Several HBase write perf improvement
Liyin Tang created HBASE-6968: - Summary: Several HBase write perf improvement Key: HBASE-6968 URL: https://issues.apache.org/jira/browse/HBASE-6968 Project: HBase Issue Type: Improvement Reporter: Liyin Tang There are two improvements in this jira: 1) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove the synchronization overhead in the normal case. 2) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is not cheap operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6967) Introduce a hook in StoreScanner to allow aggregation of KeyValues in a single row.
Adela Maznikar created HBASE-6967: - Summary: Introduce a hook in StoreScanner to allow aggregation of KeyValues in a single row. Key: HBASE-6967 URL: https://issues.apache.org/jira/browse/HBASE-6967 Project: HBase Issue Type: New Feature Reporter: Adela Maznikar Assignee: Adela Maznikar Priority: Minor Introduce an interface for enabling hooks in StoreScanner -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Assignee: Ted Yu >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472849#comment-13472849 ] Ted Yu commented on HBASE-6940: --- Integrated to trunk. Thanks for the review, Stack. > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Assignee: Ted Yu >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-6940: - Assignee: Ted Yu > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Assignee: Ted Yu >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[
https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472845#comment-13472845
]
Ted Yu commented on HBASE-6316:
---
{code}
+return convert(FSProtos.Reference.parseFrom(in));
+ } else {
{code}
nit: keyword else can be omitted.
You may want to submit for QA run after including
a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c in the patch.
> Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
> -
>
> Key: HBASE-6316
> URL: https://issues.apache.org/jira/browse/HBASE-6316
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 6316.txt, 6316v2.txt, 6316v3.txt,
> a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c
>
>
> Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96
> to start over data written by a 0.94. Need to fix it so don't have to do
> this -- that zk state left over gets auto-migrated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472826#comment-13472826 ] Hudson commented on HBASE-6961: --- Integrated in HBase-TRUNK #3437 (See [https://builds.apache.org/job/HBase-TRUNK/3437/]) HBASE-6961 In place scripts fail if mvn install hasn't been run (Revision 1396307) Result = SUCCESS eclark : Files : * /hbase/trunk/bin/hbase > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472810#comment-13472810 ] stack commented on HBASE-6316: -- Can I get a review for this blocker 0.96 issue? Thanks. > Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting > - > > Key: HBASE-6316 > URL: https://issues.apache.org/jira/browse/HBASE-6316 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, > a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c > > > Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 > to start over data written by a 0.94. Need to fix it so don't have to do > this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6953. -- Resolution: Fixed Fix Version/s: 0.96.0 Committed to trunk. Thanks for the patch Gabriel. > Incorrect javadoc description of HFileOutputFormat regarding multiple column > families > - > > Key: HBASE-6953 > URL: https://issues.apache.org/jira/browse/HBASE-6953 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Gabriel Reid >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6953.patch > > > The javadoc for HFileOutputFormat states that the class does not support > writing multiple column families; however, this hasn't been the case since > HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3661) Handle empty qualifier better in shell for increments
[
https://issues.apache.org/jira/browse/HBASE-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472801#comment-13472801
]
stack commented on HBASE-3661:
--
Patch looks good to me Michael. Can you mark it as patch available so it runs
by hadoopqa (I can't for some reason)
> Handle empty qualifier better in shell for increments
> -
>
> Key: HBASE-3661
> URL: https://issues.apache.org/jira/browse/HBASE-3661
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.92.0
>Reporter: Lars George
>Assignee: Michael Drzal
>Priority: Minor
> Attachments: HBASE-3661.patch
>
>
> When trying to increment a counter using the examples, which specify no
> *explicit* qualifier you get an error:
> {code}
> hbase(main):014:0> incr 'testtable', 'cnt1', 'colfam1', 1
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact region server 10.0.0.57:51640 for region
> testtable,,1300267113942.cd2e7925140eb414d519621e384fb654., row 'cnt1', but
> failed after 7 attempts.
> Exceptions:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.ColumnCount.(ColumnCount.java:47)
> at
> org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.(ExplicitColumnTracker.java:69)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.(ScanQueryMatcher.java:93)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:65)
> at
> org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1436)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.(HRegion.java:2412)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateInternalScanner(HRegion.java:1185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1171)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1155)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getLastIncrement(HRegion.java:3087)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.incrementColumnValue(HRegion.java:3312)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.incrementColumnValue(HRegionServer.java:2570)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:309)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1060)
> Here is some help for this command:
> Increments a cell 'value' at specified table/row/column coordinates.
> To increment a cell value in table 't1' at row 'r1' under column
> 'c1' by 1 (can be omitted) or 10 do:
> hbase> incr 't1', 'r1', 'c1'
> hbase> incr 't1', 'r1', 'c1', 1
> hbase> incr 't1', 'r1', 'c1', 10
> {code}
> Handle this more gracefully (printing 5 stacktraces is ugly), improve the
> help to specify what is needed more clearly. Or fix the server side to
> support this, if this makes sense, and therefore never triggering this issue.
> Adding a qualifier makes it work:
> {code}
> hbase(main):015:0> incr 'testtable', 'cnt1', 'colfam1:test', 1
> COUNTER VALUE = 1
> hbase(main):016:0> incr 'testtable', 'cnt1', 'colfam1:test', 1
> COUNTER VALUE = 2
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[
https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472800#comment-13472800
]
Hadoop QA commented on HBASE-6940:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548465/6940-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//console
This message is automatically generated.
> Enable GC logging by default
>
>
> Key: HBASE-6940
> URL: https://issues.apache.org/jira/browse/HBASE-6940
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Reporter: stack
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6940-trunk.txt, 6940-v2.txt
>
>
> I think we should enable gc by default. Its pretty frictionless apparently
> and could help in the case where folks are getting off the ground.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3646) When mapper writes multiple values for a key keep chronological order of values
[ https://issues.apache.org/jira/browse/HBASE-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-3646. -- Resolution: Incomplete Closing for now. Open a new issue when a patch Bob. Thanks. > When mapper writes multiple values for a key keep chronological order of > values > --- > > Key: HBASE-3646 > URL: https://issues.apache.org/jira/browse/HBASE-3646 > Project: HBase > Issue Type: New Feature > Components: Client >Affects Versions: 0.90.1 > Environment: Cloudera 3.5 VM > TableMapper > TableReducer >Reporter: Bob Cummins >Priority: Minor > > When mapper writes multiple values for a key, the underlying collection class > maps each of the values to the key, but not always in chronological order. If > chronological order were guaranteed each of the values mapped to the key, > each of the values could be understood as specific and different parameters > between the mapper and the reducer. > I've done little tricks like having the mapper flag one a the values by > making it a negative number, which the reducer recognizes and can write it to > hbase as a unique column value.This is a kluge workaround which it would be > nice to not have to do. > Used to formulate this suggestion: > TableMapper > TableReducer -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails in trunk
[
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472792#comment-13472792
]
Jimmy Xiang commented on HBASE-6958:
I can take a look next week if it is not urgent.
> TestAssignmentManager sometimes fails in trunk
> --
>
> Key: HBASE-6958
> URL: https://issues.apache.org/jira/browse/HBASE-6958
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
> Fix For: 0.96.0
>
>
> From
> https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
> :
> {code}
> Stacktrace
> java.lang.Exception: test timed out after 5000 milliseconds
> at java.lang.System.arraycopy(Native Method)
> at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
> at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
> at java.lang.Thread.exit(Thread.java:732)
> ...
> 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0]
> zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s)
> of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set
> watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.,
> state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509,
> payload.length=0
> 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0]
> executor.EventHandler(205): Caught throwable while processing event
> RS_ZK_REGION_CLOSED
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
> at
> org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
> at
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread]
> master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE,
> server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state
> from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.
> state=OFFLINE, ts=1349484372508, server=null}
> {code}
> Looks like NPE happened on this line:
> {code}
> this.gate.set(true);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6951) Allow the master info server to be started in a read only mode.
[ https://issues.apache.org/jira/browse/HBASE-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472787#comment-13472787 ] stack commented on HBASE-6951: -- +1 on patch > Allow the master info server to be started in a read only mode. > --- > > Key: HBASE-6951 > URL: https://issues.apache.org/jira/browse/HBASE-6951 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Labels: noob > Attachments: HBASE-6951-trunk-0.patch > > > There are some cases that a user could want a web ui to be accessible but > might not want the split and compact functionality to be usable. > Allowing the web ui to start in a readOnly mode would be good. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections
[
https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472779#comment-13472779
]
stack commented on HBASE-6956:
--
You have a patch Igor?
> Do not return back to HTablePool closed connections
> ---
>
> Key: HBASE-6956
> URL: https://issues.apache.org/jira/browse/HBASE-6956
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.90.6
>Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
> closed
> org.apache.hadoop.hbase.client.ClosedConnectionException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
> closed
> {code}
> After investigation we assumed that it occurs because closed connection
> returns back into HTablePool.
> For our opinion best solution is check whether the table is closed in method
> HTablePool.putTable and if true don't add it into the queue and release such
> HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through
> HTableInterface
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5355) Compressed RPC's for HBase
[ https://issues.apache.org/jira/browse/HBASE-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472776#comment-13472776 ] Devaraj Das commented on HBASE-5355: Raised HBASE-6966 for the forward port to trunk. > Compressed RPC's for HBase > -- > > Key: HBASE-5355 > URL: https://issues.apache.org/jira/browse/HBASE-5355 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Affects Versions: 0.89.20100924 >Reporter: Karthik Ranganathan >Assignee: Karthik Ranganathan > Attachments: HBASE-5355-0.94.patch > > > Some application need ability to do large batched writes and reads from a > remote MR cluster. These eventually get bottlenecked on the network. These > results are also pretty compressible sometimes. > The aim here is to add the ability to do compressed calls to the server on > both the send and receive paths. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of logging to a more human-readable level
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472774#comment-13472774 ] stack commented on HBASE-6804: -- +1 on commit. Can this go back into 0.94 too? > [replication] lower the amount of logging to a more human-readable level > > > Key: HBASE-6804 > URL: https://issues.apache.org/jira/browse/HBASE-6804 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans >Priority: Critical > Fix For: 0.96.0 > > Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch > > > We need stop logging every time replication decides to do something. It used > to be extremely useful when the code base was younger but now it should be > possible to bring it down while keeping it relevant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6966) "Compressed RPCs for HBase" (HBASE-5355) port to trunk
Devaraj Das created HBASE-6966: -- Summary: "Compressed RPCs for HBase" (HBASE-5355) port to trunk Key: HBASE-6966 URL: https://issues.apache.org/jira/browse/HBASE-6966 Project: HBase Issue Type: Improvement Components: IPC/RPC Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 This jira will address the port of the compressed RPC implementation to trunk. I am expecting the patch to be significantly different due to the PB stuff in trunk, and hence filed a separate jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[
https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472770#comment-13472770
]
stack commented on HBASE-6940:
--
+1
On commit put this line:
{code}
+export HBASE_USE_GC_LOGFILE=true
{code}
... before the uncommented HBASE_OPTS
> Enable GC logging by default
>
>
> Key: HBASE-6940
> URL: https://issues.apache.org/jira/browse/HBASE-6940
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Reporter: stack
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6940-trunk.txt, 6940-v2.txt
>
>
> I think we should enable gc by default. Its pretty frictionless apparently
> and could help in the case where folks are getting off the ground.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472764#comment-13472764 ] Kumar Ravi commented on HBASE-6945: --- Ted, Thanks for your comment. Attaching a patch file.. > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: ResourceChecker.patch > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6945: -- Attachment: ResourceChecker.patch > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: ResourceChecker.patch > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472762#comment-13472762 ] Kumar Ravi commented on HBASE-6945: --- A build with the above patch along with the new class in https://issues.apache.org/jira/browse/HBASE-6965 were built and tested on Revision 1396305: /hbase/branches/0.94 with IBM JDK 7, openJDK 6 as well as Oracle JDK 6. Builds were successful and the junit tests (runDevTests) ran with all three JDKs without any errors or failures. > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Status: Patch Available (was: Open) > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Attachment: 6940-v2.txt > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt, 6940-v2.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6957) TestRowCounter consistently fails against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472754#comment-13472754
]
stack commented on HBASE-6957:
--
Any chance of a patch Ted?
> TestRowCounter consistently fails against hadoop-2.0
>
>
> Key: HBASE-6957
> URL: https://issues.apache.org/jira/browse/HBASE-6957
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 0.96.0
>
>
> In
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/210/testReport/org.apache.hadoop.hbase.mapreduce/TestRowCounter/testRowCounterHiddenColumn/
> , we can see:
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at
> org.apache.hadoop.hbase.mapreduce.TestRowCounter.runRowCount(TestRowCounter.java:135)
> at
> org.apache.hadoop.hbase.mapreduce.TestRowCounter.testRowCounterHiddenColumn(TestRowCounter.java:118)
> ...
> 2012-10-05 11:24:17,355 WARN [ContainersLauncher #1]
> launcher.ContainerLaunch(246): Failed to launch container.
> java.lang.ArithmeticException: / by zero
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:355)
> at
> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
> at
> org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLogPathForWrite(LocalDirsHandlerService.java:268)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:126)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2012-10-05 11:24:17,356 WARN [DeletionService #1]
> nodemanager.DefaultContainerExecutor(276): delete returned false for path:
> [/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-2.0.0/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-localDir-nm-1_0/usercache/jenkins/appcache/application_1349436189156_0003/container_1349436189156_0003_01_02]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails in trunk
[
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472751#comment-13472751
]
stack commented on HBASE-6958:
--
Any chance of a patch Ted?
> TestAssignmentManager sometimes fails in trunk
> --
>
> Key: HBASE-6958
> URL: https://issues.apache.org/jira/browse/HBASE-6958
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
> Fix For: 0.96.0
>
>
> From
> https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
> :
> {code}
> Stacktrace
> java.lang.Exception: test timed out after 5000 milliseconds
> at java.lang.System.arraycopy(Native Method)
> at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
> at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
> at java.lang.Thread.exit(Thread.java:732)
> ...
> 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0]
> zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s)
> of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set
> watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.,
> state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509,
> payload.length=0
> 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0]
> executor.EventHandler(205): Caught throwable while processing event
> RS_ZK_REGION_CLOSED
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
> at
> org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
> at
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread]
> master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE,
> server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state
> from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.
> state=OFFLINE, ts=1349484372508, server=null}
> {code}
> Looks like NPE happened on this line:
> {code}
> this.gate.set(true);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472749#comment-13472749 ] stack commented on HBASE-6940: -- That would work but fix the comments. They don't make sense now the default is these environment variables are on by default. > Enable GC logging by default > > > Key: HBASE-6940 > URL: https://issues.apache.org/jira/browse/HBASE-6940 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: stack >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6940-trunk.txt > > > I think we should enable gc by default. Its pretty frictionless apparently > and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472746#comment-13472746 ] Ted Yu commented on HBASE-6945: --- Hadoop QA only runs patch through trunk test suite. Can you produce a patch file for hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java and attach the patch to this JIRA ? Thanks > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6945: -- Status: Open (was: Patch Available) > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472744#comment-13472744 ] Ted Yu commented on HBASE-6965: --- Hi, You may want to put this file under hbase-server/src/main/java/org/apache/hadoop/hbase/util and produce a patch for trunk. Thanks > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: OSMXBean.java > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[
https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumar Ravi updated HBASE-6945:
--
Status: Patch Available (was: Open)
--- curr_hbase/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
2012-10-04 12:59:14.0 -0400
+++ new_hbase/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
2012-10-08 16:32:00.0 -0400
@@ -20,13 +20,11 @@
package org.apache.hadoop.hbase;
-import com.sun.management.UnixOperatingSystemMXBean;
+import org.apache.hadoop.hbase.util.OSMXBean;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.hbase.client.HConnectionTestingUtility;
-import java.lang.management.ManagementFactory;
-import java.lang.management.OperatingSystemMXBean;
import java.util.*;
@@ -42,42 +40,33 @@
* On unix, we know how to get the number of open file descriptor
*/
private static class ResourceAnalyzer {
-private static final OperatingSystemMXBean osStats;
-private static final UnixOperatingSystemMXBean unixOsStats;
public long getThreadsCount() {
return Thread.getAllStackTraces().size();
}
public long getOpenFileDescriptorCount() {
- if (unixOsStats == null) {
-return 0;
- } else {
-return unixOsStats.getOpenFileDescriptorCount();
- }
+ OSMXBean osMbean = new OSMXBean();
+ if (osMbean != null && osMbean.getUnix() == true) {
+return osMbean.getOpenFileDescriptorCount();
+ }
+else
+return 0;
}
public long getMaxFileDescriptorCount() {
- if (unixOsStats == null) {
-return 0;
- } else {
-return unixOsStats.getMaxFileDescriptorCount();
+ OSMXBean osMbean = new OSMXBean();
+ if (osMbean != null && osMbean.getUnix() == true) {
+ return osMbean.getMaxFileDescriptorCount();
}
+ else
+ return 0;
}
public long getConnectionCount(){
return HConnectionTestingUtility.getConnectionCount();
}
-static {
- osStats =
-ManagementFactory.getOperatingSystemMXBean();
- if (osStats instanceof UnixOperatingSystemMXBean) {
-unixOsStats = (UnixOperatingSystemMXBean) osStats;
- } else {
-unixOsStats = null;
- }
-}
}
private static final ResourceAnalyzer rc = new ResourceAnalyzer();
> Compilation errors when using non-Sun JDKs to build HBase-0.94
> --
>
> Key: HBASE-6945
> URL: https://issues.apache.org/jira/browse/HBASE-6945
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.94.1
> Environment: RHEL 6.3, IBM Java 7
>Reporter: Kumar Ravi
>Assignee: Kumar Ravi
> Fix For: 0.94.3
>
>
> When using IBM Java 7 to build HBase-0.94.1, the following comilation error
> is seen.
> [INFO] -
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25]
> error: package com.sun.management does not exist
> [ERROR]
> /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25]
> error: cannot find symbol
> [ERROR] symbol: class UnixOperatingSystemMXBean
> location: class ResourceAnalyzer
> /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29]
> error: cannot find symbol
> [ERROR] symbol: class UnixOperatingSystemMXBean
> location: class ResourceAnalyzer
> /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23]
> error: cannot find symbol
> [INFO] 4 errors
> [INFO] -
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> I have a patch available which should work for all JDKs including Sun.
> I am in the process of testing this patch. Preliminary tests indicate the
> build is working fine with this patch. I will post this patch when I am done
> testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472741#comment-13472741 ] Kumar Ravi commented on HBASE-6945: --- I have created another JIRA https://issues.apache.org/jira/browse/HBASE-6965 to be used along with the patch I plan to submit here. This new class called OSMXBean is used by the patch here. > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-6965 started by Kumar Ravi. > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: OSMXBean.java > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Attachment: OSMXBean.java Here's the source file for OSMXBean.java. This would reside in the src/main/java/org/apache/hadoop/hbase/util directory. > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > Attachments: OSMXBean.java > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Status: Patch Available (was: Open) Here's the source file in it's entirety. > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Status: Open (was: Patch Available) > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Fix For: 0.94.3 > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
Kumar Ravi created HBASE-6965: - Summary: Generic MXBean Utility class to support all JDK vendors Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6961: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review Stack. Integrated into trunk on revision 1396307 > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6964) Add more instrumentation for log roll
Amitanand Aiyer created HBASE-6964: -- Summary: Add more instrumentation for log roll Key: HBASE-6964 URL: https://issues.apache.org/jira/browse/HBASE-6964 Project: HBase Issue Type: Sub-task Reporter: Amitanand Aiyer Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472729#comment-13472729
]
Hadoop QA commented on HBASE-6963:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548448/trunk_6963.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//console
This message is automatically generated.
> unable to run hbck on a secure cluster
> --
>
> Key: HBASE-6963
> URL: https://issues.apache.org/jira/browse/HBASE-6963
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk_6963.patch
>
>
> {noformat}
> 12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
> preCheckPermission
> org.apache.hadoop.security.AccessControlException: Permission denied:
> action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
> at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[ https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472723#comment-13472723 ] Lars Hofhansl commented on HBASE-6929: -- We were discussing this internally, and there'd be interest for us to have 0.94 working with Hadoop-2. Is there a top-level jira that I can check out to see the magnitude of the changes needed? > Publish Hbase 0.94 artifacts build against hadoop-2.0 > - > > Key: HBASE-6929 > URL: https://issues.apache.org/jira/browse/HBASE-6929 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 0.94.2 >Reporter: Enis Soztutar > > Downstream projects (flume, hive, pig, etc) depends on hbase, but since the > hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot > depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should > also push hbase jars build with hadoop2.0 profile into maven, possibly with > version string like 0.94.2-hadoop2.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472720#comment-13472720 ] stack commented on HBASE-6961: -- +1 on 1. and 2. then; i.e. +1 on committing the attached patch. > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell reassigned HBASE-5525: -- Assignee: Kevin Odell > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Labels: newbie, noob > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-6963:
---
Status: Patch Available (was: Open)
> unable to run hbck on a secure cluster
> --
>
> Key: HBASE-6963
> URL: https://issues.apache.org/jira/browse/HBASE-6963
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk_6963.patch
>
>
> {noformat}
> 12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
> preCheckPermission
> org.apache.hadoop.security.AccessControlException: Permission denied:
> action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
> at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-6963:
---
Attachment: trunk_6963.patch
> unable to run hbck on a secure cluster
> --
>
> Key: HBASE-6963
> URL: https://issues.apache.org/jira/browse/HBASE-6963
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk_6963.patch
>
>
> {noformat}
> 12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
> preCheckPermission
> org.apache.hadoop.security.AccessControlException: Permission denied:
> action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
> at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6963) unable to run hbck on a secure cluster
Jimmy Xiang created HBASE-6963:
--
Summary: unable to run hbck on a secure cluster
Key: HBASE-6963
URL: https://issues.apache.org/jira/browse/HBASE-6963
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
{noformat}
12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
at
org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472644#comment-13472644 ] Elliott Clark commented on HBASE-6961: -- No maven spew. It's all piped to /dev/null. The computation is cached in target/cached_classpath.txt as long as that file remains this won't get re-computed. So a mvn clean or a rm target/cached_classpath.txt will force a re-run of mvn package -DskipTests dependency:build-classpath but other than that the start up shouldn't change. > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472645#comment-13472645
]
stack commented on HBASE-6929:
--
You know how to fix this [~enis]?
{code}
Here is what I ran:
stack:0.94 stack$ mvn -DskipTests -Dhadoop.profile=2.0 -Dclassifier=hadoop2
install
...and I got this:
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 1:12.527s
[INFO] Finished at: Tue Oct 09 12:11:41 PDT 2012
[INFO] Final Memory: 28M/81M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:2.1:build-classpath
(create-mrapp-generated-classpath) on project hbase: not found in any
repository: aopalliance:aopalliance:java-source:hadoop2:1.0: Could not find
artifact aopalliance:aopalliance:jar:hadoop2:1.0 in apache release
(https://repository.apache.org/content/repositories/releases/)
[ERROR]
[ERROR] Try downloading the file manually from the project website.
[ERROR]
[ERROR] Then, install it using the command:
[ERROR] mvn install:install-file -DgroupId=aopalliance -DartifactId=aopalliance
-Dversion=1.0 -Dclassifier=hadoop2 -Dpackaging=java-source -Dfile=/path/to/file
[ERROR]
[ERROR] Alternatively, if you host your own repository you can deploy the file
there:
[ERROR] mvn deploy:deploy-file -DgroupId=aopalliance -DartifactId=aopalliance
-Dversion=1.0 -Dclassifier=hadoop2 -Dpackaging=java-source -Dfile=/path/to/file
-Durl=[url] -DrepositoryId=[id]
[ERROR]
[ERROR]
[ERROR] aopalliance:aopalliance:java-source:1.0
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] apache release
(https://repository.apache.org/content/repositories/releases/, releases=true,
snapshots=true),
[ERROR] apache non-releases (http://people.apache.org/~stack/m2/repository,
releases=true, snapshots=false),
[ERROR] java.net (http://download.java.net/maven/2/, releases=true,
snapshots=false),
[ERROR] codehaus (http://repository.codehaus.org/, releases=true,
snapshots=false),
[ERROR] repository.jboss.org
(http://repository.jboss.org/nexus/content/groups/public-jboss/, releases=true,
snapshots=false),
[ERROR] ghelmling.testing (http://people.apache.org/~garyh/mvn/, releases=true,
snapshots=true),
[ERROR] apache.snapshots (http://repository.apache.org/snapshots,
releases=false, snapshots=true),
[ERROR] central (http://repo1.maven.org/maven2, releases=true, snapshots=false)
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
{code}
> Publish Hbase 0.94 artifacts build against hadoop-2.0
> -
>
> Key: HBASE-6929
> URL: https://issues.apache.org/jira/browse/HBASE-6929
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 0.94.2
>Reporter: Enis Soztutar
>
> Downstream projects (flume, hive, pig, etc) depends on hbase, but since the
> hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot
> depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should
> also push hbase jars build with hadoop2.0 profile into maven, possibly with
> version string like 0.94.2-hadoop2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[ https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472607#comment-13472607 ] stack commented on HBASE-6929: -- [~lhofhansl] If you do mvn package, it will install the build into your local mvn repository (see under ~/.m2/repository). Notice how the suffix is -SNAPSHOT. The suggestion is to push one of these snapshots up to apache (Apache maintains an area where you can publish mvn snapshots; see https://repository.apache.org/content/groups/snapshots/org/apache/hbase/hbase/). > Publish Hbase 0.94 artifacts build against hadoop-2.0 > - > > Key: HBASE-6929 > URL: https://issues.apache.org/jira/browse/HBASE-6929 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 0.94.2 >Reporter: Enis Soztutar > > Downstream projects (flume, hive, pig, etc) depends on hbase, but since the > hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot > depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should > also push hbase jars build with hadoop2.0 profile into maven, possibly with > version string like 0.94.2-hadoop2.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472604#comment-13472604 ] stack commented on HBASE-6961: -- On 1., it will show mvn spew everytime I run bin/start-hbase (even if just dev context)? And slow startup everytime? That seems a little obnoxious. Can we not improve the message folks get if they fail to package first? On 2., the reorder of cp, that is good (I actually misread the ordering you were trying to achieve -- pardon me). > In place scripts fail if mvn install hasn't been run > > > Key: HBASE-6961 > URL: https://issues.apache.org/jira/browse/HBASE-6961 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-6961-0.patch > > > bin/hbase tries to get dependencies of the project however it fails if mvn > install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472516#comment-13472516 ] Ted Yu commented on HBASE-6942: --- @Anoop: Can you add a test case where deletion fails in one of the regions of the table ? Once you add that test, you would discover a small flaw in determining row count. Thanks > Endpoint implementation for bulk delete > --- > > Key: HBASE-6942 > URL: https://issues.apache.org/jira/browse/HBASE-6942 > Project: HBase > Issue Type: Improvement > Components: Coprocessors, Performance >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-6942.patch > > > We can provide an end point implementation for doing a bulk delete (based on > a scan) at the server side. This can reduce the time taken for such an > operation as right now it need to do a scan to client and issue delete(s) > using rowkeys. > Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472509#comment-13472509 ] Lars Hofhansl commented on HBASE-6942: -- Patch looks good over all (but have not looked in detail). Since this is a specific use case (we're issuing family deletes always), I'd say we should only provide the EndPoint and don't add a public API into HTableInterface. > Endpoint implementation for bulk delete > --- > > Key: HBASE-6942 > URL: https://issues.apache.org/jira/browse/HBASE-6942 > Project: HBase > Issue Type: Improvement > Components: Coprocessors, Performance >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-6942.patch > > > We can provide an end point implementation for doing a bulk delete (based on > a scan) at the server side. This can reduce the time taken for such an > operation as right now it need to do a scan to client and issue delete(s) > using rowkeys. > Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-643) Rename tables
[ https://issues.apache.org/jira/browse/HBASE-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472458#comment-13472458 ] stack commented on HBASE-643: - @Devin No. What is attached here has rotted. It would need work to update it. Even then, the script was a band-aid. What is needed is our keeping an id rather than the table name everywhere so a rename requires our changing the tablename once in a table to id map rather than in a few places in the filesystem as well as in the metadata kept by each table region. > Rename tables > - > > Key: HBASE-643 > URL: https://issues.apache.org/jira/browse/HBASE-643 > Project: HBase > Issue Type: New Feature >Reporter: Michael Bieniosek > Attachments: copy_table.rb, rename_table.rb > > > It would be nice to be able to rename tables, if this is possible. Some of > our internal users are doing things like: upload table mytable -> realize > they screwed up -> upload table mytable_2 -> decide mytable_2 looks better -> > have to go on using mytable_2 instead of originally desired table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6962) Upgrade hadoop 1 dependency to hadoop 1.1
[ https://issues.apache.org/jira/browse/HBASE-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6962: -- Summary: Upgrade hadoop 1 dependency to hadoop 1.1 (was: Upgrade to hadoop 1.1) > Upgrade hadoop 1 dependency to hadoop 1.1 > - > > Key: HBASE-6962 > URL: https://issues.apache.org/jira/browse/HBASE-6962 > Project: HBase > Issue Type: Bug > Environment: hadoop 1.1 contains multiple important fixes, including > HDFS-3703 >Reporter: Ted Yu > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6962) Upgrade to hadoop 1.1
Ted Yu created HBASE-6962: - Summary: Upgrade to hadoop 1.1 Key: HBASE-6962 URL: https://issues.apache.org/jira/browse/HBASE-6962 Project: HBase Issue Type: Bug Environment: hadoop 1.1 contains multiple important fixes, including HDFS-3703 Reporter: Ted Yu -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472406#comment-13472406 ] Ted Yu commented on HBASE-6785: --- Minor comment: 6758-simplified-pb.patch should have been named 6785-simplified-pb.patch (JIRA number was wrong) review comments to follow. > Convert AggregateProtocol to protobuf defined coprocessor service > - > > Key: HBASE-6785 > URL: https://issues.apache.org/jira/browse/HBASE-6785 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Gary Helmling >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: Aggregate.proto, Aggregate.proto > > > With coprocessor endpoints now exposed as protobuf defined services, we > should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-643) Rename tables
[ https://issues.apache.org/jira/browse/HBASE-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472387#comment-13472387 ] Devin Bayer commented on HBASE-643: --- Hi. Is it safe to use the rename_table.rb script attached to this bug? Which versions does it work with? > Rename tables > - > > Key: HBASE-643 > URL: https://issues.apache.org/jira/browse/HBASE-643 > Project: HBase > Issue Type: New Feature >Reporter: Michael Bieniosek > Attachments: copy_table.rb, rename_table.rb > > > It would be nice to be able to rename tables, if this is possible. Some of > our internal users are doing things like: upload table mytable -> realize > they screwed up -> upload table mytable_2 -> decide mytable_2 looks better -> > have to go on using mytable_2 instead of originally desired table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5498) Secure Bulk Load
[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472217#comment-13472217
]
Hadoop QA commented on HBASE-5498:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548374/HBASE-5498_trunk_2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//console
This message is automatically generated.
> Secure Bulk Load
>
>
> Key: HBASE-5498
> URL: https://issues.apache.org/jira/browse/HBASE-5498
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Francis Liu
>Assignee: Francis Liu
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-5498_94_2.patch, HBASE-5498_94.patch,
> HBASE-5498_94.patch, HBASE-5498_draft_94.patch, HBASE-5498_draft.patch,
> HBASE-5498_trunk_2.patch, HBASE-5498_trunk.patch
>
>
> Design doc:
> https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load
> Short summary:
> Security as it stands does not cover the bulkLoadHFiles() feature. Users
> calling this method will bypass ACLs. Also loading is made more cumbersome in
> a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data
> from user's directory to the hbase directory, which would require certain
> write access privileges set.
> Our solution is to create a coprocessor which makes use of AuthManager to
> verify if a user has write access to the table. If so, launches a MR job as
> the hbase user to do the importing (ie rewrite from text to hfiles). One
> tricky part this job will have to do is impersonate the calling user when
> reading the input files. We can do this by expecting the user to pass an hdfs
> delegation token as part of the secureBulkLoad() coprocessor call and extend
> an inputformat to make use of that token. The output is written to a
> temporary directory accessible only by hbase and then bulkloadHFiles() is
> called.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
