[jira] [Commented] (HBASE-7305) ZK based Read/Write locks for table operations
[ https://issues.apache.org/jira/browse/HBASE-7305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584100#comment-13584100 ] ramkrishna.s.vasudevan commented on HBASE-7305: --- Going thro the patch, currenlty we have use case only with WriteLock? So when we do region assignments thro balancing then we should create a readlock on the path /znode/table-lock for the given table. Now when this process is going on we will stop the user from Disabling or Modifying the table. Am i right? That part is yet to be implemented? ZK based Read/Write locks for table operations -- Key: HBASE-7305 URL: https://issues.apache.org/jira/browse/HBASE-7305 Project: HBase Issue Type: Bug Components: Client, master, Zookeeper Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: 7305-v11.txt, hbase-7305_v0.patch, hbase-7305_v10.patch, hbase-7305_v13.patch, hbase-7305_v14.patch, hbase-7305_v15.patch, hbase-7305_v1-based-on-curator.patch, hbase-7305_v2.patch, hbase-7305_v4.patch, hbase-7305_v9.patch This has started as forward porting of HBASE-5494 and HBASE-5991 from the 89-fb branch to trunk, but diverged enough to have it's own issue. The idea is to implement a zk based read/write lock per table. Master initiated operations should get the write lock, and region operations (region split, moving, balance?, etc) acquire a shared read lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7906) OfflineCallBack in bulk assignment does not work as expected
ramkrishna.s.vasudevan created HBASE-7906: - Summary: OfflineCallBack in bulk assignment does not work as expected Key: HBASE-7906 URL: https://issues.apache.org/jira/browse/HBASE-7906 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan This is as part of discussion in HBASE-7799. The OfflineCallBack does not bother if the znode already exists or not. Infact the catch block AM.asyncSetOfflineInZooKeeper() is a dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restart
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584107#comment-13584107
]
ramkrishna.s.vasudevan commented on HBASE-6469:
---
Understood the patch now.
bq.LOG.error(Error trying to enable the table + tableEntry.getKey(), e);
Change the error msg.
Should there be sync mechanism between the place where we remove the tableEntry
and the place where we add it in Enable/DisableTable handler?
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restart
-
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Enis Soztutar
Assignee: Nick Dimiduk
Fix For: 0.96.0, 0.94.6
Attachments: 6469-expose-force-r3.patch, HBASE-6469.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7790) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block
[ https://issues.apache.org/jira/browse/HBASE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7790: -- Status: Patch Available (was: Open) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block --- Key: HBASE-7790 URL: https://issues.apache.org/jira/browse/HBASE-7790 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-7790.patch This is based on discussion in HBASE-7698. Jimmy suggested this improvment. Look at https://issues.apache.org/jira/browse/HBASE-7698?focusedCommentId=13572736page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13572736 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7790) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block
[ https://issues.apache.org/jira/browse/HBASE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584179#comment-13584179 ] ramkrishna.s.vasudevan commented on HBASE-7790: --- Ran the testsuite. org.apache.hadoop.hbase.regionserver.TestRegionServerNoMaster failed. Found the reason and corrected the patch accordingly. Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block --- Key: HBASE-7790 URL: https://issues.apache.org/jira/browse/HBASE-7790 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-7790.patch This is based on discussion in HBASE-7698. Jimmy suggested this improvment. Look at https://issues.apache.org/jira/browse/HBASE-7698?focusedCommentId=13572736page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13572736 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7790) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block
[ https://issues.apache.org/jira/browse/HBASE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7790: -- Attachment: HBASE-7790.patch Patch for trunk. Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block --- Key: HBASE-7790 URL: https://issues.apache.org/jira/browse/HBASE-7790 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-7790.patch This is based on discussion in HBASE-7698. Jimmy suggested this improvment. Look at https://issues.apache.org/jira/browse/HBASE-7698?focusedCommentId=13572736page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13572736 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7839) Add a script to simulate a dead machine during the integration tests
[ https://issues.apache.org/jira/browse/HBASE-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584186#comment-13584186 ] nkeywal commented on HBASE-7839: [~enis]No, it's not allowed as it's considered as a security issue. The wrapper doesn't fix the security issue, but workarounds the OS protection. Add a script to simulate a dead machine during the integration tests Key: HBASE-7839 URL: https://issues.apache.org/jira/browse/HBASE-7839 Project: HBase Issue Type: New Feature Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 7839.v1.patch A kill -9 cannot be used to simulate a dead box. With a kill -9: 1) the sockets are closed by the OS 2) the next calls will get immediately a connection refused. Physically unplugging the cable does work. To automate this, the best solution seems to be using a firewall config. It's a little bit complicated however, because we need to refuse the incoming packets. If we block the out connection, it's detected by the OS and there is no wait on the socket. Moreover you need to be root, or use a program with a sticky bit to launch the script that changes the firewall config. This will be usable from the integration tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7906) OfflineCallBack in bulk assignment does not work as expected
[ https://issues.apache.org/jira/browse/HBASE-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584198#comment-13584198 ] ramkrishna.s.vasudevan commented on HBASE-7906: --- So if we want async behaviour then we cannot find if the znode exists or not. Even if it exists we should take some action on that. May be we should forcefully set the state to OFFLINE or we may have to abort the server. But for doing it we may have to read the data of the znode. So can we introduce a config to say if it is ok to check the data and then asynchronously create znode in OFFLINE state. Because as Jimmy pointed out in HBASE-7799 this piece of code is aimed at faster execution and hence the async nature. Any other suggestions on this? OfflineCallBack in bulk assignment does not work as expected Key: HBASE-7906 URL: https://issues.apache.org/jira/browse/HBASE-7906 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan This is as part of discussion in HBASE-7799. The OfflineCallBack does not bother if the znode already exists or not. Infact the catch block AM.asyncSetOfflineInZooKeeper() is a dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7838) HBase regionserver never stops when running `hbase-daemon.sh stop regionserver`
[ https://issues.apache.org/jira/browse/HBASE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584210#comment-13584210 ] nkeywal commented on HBASE-7838: committed addendum to trunk. Will wait for 0.94. HBase regionserver never stops when running `hbase-daemon.sh stop regionserver` --- Key: HBASE-7838 URL: https://issues.apache.org/jira/browse/HBASE-7838 Project: HBase Issue Type: Bug Components: regionserver, scripts, shell Affects Versions: 0.92.1, 0.96.0, 0.94.4 Environment: Using CDH4.1.2 Reporter: Damien Hardy Assignee: Damien Hardy Labels: newbie Fix For: 0.96.0 Attachments: 7838.addendum.v1.patch, HBASE-7838-0.94.patch, HBASE-7838-trunk.patch Currently running full stack (Hadoop + Hbase + zookeeper) on a VM I want to reboot, every services stopped but HBase regionserver. For 2 days now, it's aligning dots on the console ... I propose a timeout (default 600 overidable by $HBASE_STOP_TIMEOUT) to force kill -9 on the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7865) HBase regionserver never stops when running `bin/stop-hbase.sh` on master
[ https://issues.apache.org/jira/browse/HBASE-7865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584211#comment-13584211 ] nkeywal commented on HBASE-7865: It's up to you JM. If you think you can have more logs or more info on how to reproduce it, keeping this jira open is useful. If not, it can be closed as wontfix or not reproduced... I've committed HBASE-7838 to trunk, if you want to add a stack trace dump to it, feel free to open a jira. HBase regionserver never stops when running `bin/stop-hbase.sh` on master - Key: HBASE-7865 URL: https://issues.apache.org/jira/browse/HBASE-7865 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Attachments: hbase-root-master-hbasetest.log, hbase-root-master-hbasetest.out.4, jstack_hbasetest.txt, jstack_node1.txt, jstack_node3.txt, jstack_node7.txt I faced 3 regions (out of 8) never stopping today. This is pretty bad because the script is supposed to wait until all the RS stopped to re-start everything, therefor, servers are never going back online. HBASE-7838 will help with that and will kill the RSs. But that will not really solve the root cause. Attached are the jstack for the 3 servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7790) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block
[
https://issues.apache.org/jira/browse/HBASE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584216#comment-13584216
]
Hadoop QA commented on HBASE-7790:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570457/HBASE-7790.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4496//console
This message is automatically generated.
Refactor OpenRegionHandler so that the cleanup happens in one place - the
finally block
---
Key: HBASE-7790
URL: https://issues.apache.org/jira/browse/HBASE-7790
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-7790.patch
This is based on discussion in HBASE-7698. Jimmy suggested this improvment.
Look at
https://issues.apache.org/jira/browse/HBASE-7698?focusedCommentId=13572736page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13572736
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7865) HBase regionserver never stops when running `bin/stop-hbase.sh` on master
[ https://issues.apache.org/jira/browse/HBASE-7865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584225#comment-13584225 ] Jean-Marc Spaggiari commented on HBASE-7865: Let's close it as not reproduced. I will update the start script to add the jstack into it and submit it via another JIRA. That way I will just have to take a look from time to time and see if there is any jstack log created. If any, I will open another ticket. Thanks. HBase regionserver never stops when running `bin/stop-hbase.sh` on master - Key: HBASE-7865 URL: https://issues.apache.org/jira/browse/HBASE-7865 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Attachments: hbase-root-master-hbasetest.log, hbase-root-master-hbasetest.out.4, jstack_hbasetest.txt, jstack_node1.txt, jstack_node3.txt, jstack_node7.txt I faced 3 regions (out of 8) never stopping today. This is pretty bad because the script is supposed to wait until all the RS stopped to re-start everything, therefor, servers are never going back online. HBASE-7838 will help with that and will kill the RSs. But that will not really solve the root cause. Attached are the jstack for the 3 servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5154) Can't put small timestamp after delete the column
[
https://issues.apache.org/jira/browse/HBASE-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584254#comment-13584254
]
Guido Serra aka Zeph commented on HBASE-5154:
-
seems that someone is trying to fix it HBASE-5241
Can't put small timestamp after delete the column
-
Key: HBASE-5154
URL: https://issues.apache.org/jira/browse/HBASE-5154
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3
Environment: OS: Linux 2.6.32-33-server #70-Ubuntu SMP
JRE: Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
Hadoop: Version: 0.20-append-r1056497, r1056491
Hbase run on 4 HRegion + 1 HMaster cluster.
Reporter: robi
Priority: Critical
1. Call put to insert some value in column 'fm:a' like:
Put.add('fm', 'a', 1000, 'abc'), here timestamp = 1000.
2. Delete the column 'fm:a'
3. Try to do #1 again.(it doesn't work, but can insert put which use
timestamp 1000)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restart
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584290#comment-13584290
]
rajeshbabu commented on HBASE-6469:
---
@Ram, Thanks for review.
bq. Change the error msg.
I will change.
bq.Should there be sync mechanism between the place where we remove the
tableEntry and the place where we add it in Enable/DisableTable handler?
Its needed. I will prepare new patch with some test cases also.
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restart
-
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Enis Soztutar
Assignee: Nick Dimiduk
Fix For: 0.96.0, 0.94.6
Attachments: 6469-expose-force-r3.patch, HBASE-6469.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restart
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reassigned HBASE-6469:
---
Assignee: rajeshbabu (was: Nick Dimiduk)
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restart
-
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.96.0, 0.94.6
Attachments: 6469-expose-force-r3.patch, HBASE-6469.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7865) HBase regionserver never stops when running `bin/stop-hbase.sh` on master
[ https://issues.apache.org/jira/browse/HBASE-7865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-7865. --- Resolution: Cannot Reproduce HBase regionserver never stops when running `bin/stop-hbase.sh` on master - Key: HBASE-7865 URL: https://issues.apache.org/jira/browse/HBASE-7865 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Attachments: hbase-root-master-hbasetest.log, hbase-root-master-hbasetest.out.4, jstack_hbasetest.txt, jstack_node1.txt, jstack_node3.txt, jstack_node7.txt I faced 3 regions (out of 8) never stopping today. This is pretty bad because the script is supposed to wait until all the RS stopped to re-start everything, therefor, servers are never going back online. HBASE-7838 will help with that and will kill the RSs. But that will not really solve the root cause. Attached are the jstack for the 3 servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584323#comment-13584323 ] Ted Yu commented on HBASE-7878: --- There is no assignee for HDFS-2296. The change in this JIRA is similar to how Accumulo handles lease recovery. Since hdfs API change would take some time, I prefer putting in better safe safeguard through this JIRA. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v1.txt, 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584323#comment-13584323 ] Ted Yu edited comment on HBASE-7878 at 2/22/13 2:59 PM: There is no assignee for HDFS-2296. The change in this JIRA is similar to how Accumulo handles lease recovery. Since hdfs API change would take some time, I prefer putting in better safeguard through this JIRA. was (Author: yuzhih...@gmail.com): There is no assignee for HDFS-2296. The change in this JIRA is similar to how Accumulo handles lease recovery. Since hdfs API change would take some time, I prefer putting in better safe safeguard through this JIRA. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v1.txt, 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4155) the problem in hbase thrift client when scan/get rows by timestamp
[
https://issues.apache.org/jira/browse/HBASE-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584331#comment-13584331
]
Guido Serra aka Zeph commented on HBASE-4155:
-
similar issue...
{code}
hbase(main):007:0 scan AAA_customer, {TIMERANGE = [0, 1360032970]}
ROW COLUMN+CELL
0 row(s) in 1.5590 seconds
hbase(main):008:0 scan AAA_customer
ROW COLUMN+CELL
1 column=mysql:birthday,
timestamp=1360292144, value=1999-01-01
{code}
the problem in hbase thrift client when scan/get rows by timestamp
--
Key: HBASE-4155
URL: https://issues.apache.org/jira/browse/HBASE-4155
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.90.0
Reporter: zezhou
Attachments: 4155.txt, patch.txt, patch.txt.svn
Original Estimate: 1m
Remaining Estimate: 1m
I want to scan rows by specified timestamp. I use following hbase shell
command :
scan 'testcrawl',{TIMESTAMP=1312268202071}
ROW COLUMN+CELL
put1.com column=crawl:data,
timestamp=1312268202071, value=htmlput1/html
put1.com column=crawl:type,
timestamp=1312268202071, value=html
put1.com column=links:outlinks,
timestamp=1312268202071, value=www.163.com;www.sina.com
As I expected, I can get the rows which timestamp is 1312268202071.
But when I use thift client to do the same thing ,the return data is the rows
which time before specified timestamp , not the same as hbase
shell.following is timestamp of return data:
131217917
1312268202059
I look up the source in
hbase/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java, it use
following code to set time parameter .
scan.setTimeRange(Long.MIN_VALUE, timestamp);
This cause thrift client return rows before specified row ,not the rows
timestamp specified.
But in hbase client and avro client ，it use following code to set time
parameter.
scan.setTimeStamp(timestamp);
this will return rows timestamp specified.
Is this a feature or a bug in thrift client ?
if this is a feature, which method in thrift client can get the rows by
specified timestamp?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4155) the problem in hbase thrift client when scan/get rows by timestamp
[
https://issues.apache.org/jira/browse/HBASE-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584334#comment-13584334
]
Guido Serra aka Zeph commented on HBASE-4155:
-
while, this other works
{code}
hbase(main):009:0 scan AAA_customer, {TIMESTAMP = 1360032970}
ROW COLUMN+CELL
0 row(s) in 1.3960 seconds
hbase(main):010:0 scan AAA_customer, {TIMESTAMP = 1360292144}
ROW COLUMN+CELL
1 column=mysql:birthday,
timestamp=1360292144, 1999-01-01
{code}
the problem in hbase thrift client when scan/get rows by timestamp
--
Key: HBASE-4155
URL: https://issues.apache.org/jira/browse/HBASE-4155
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.90.0
Reporter: zezhou
Attachments: 4155.txt, patch.txt, patch.txt.svn
Original Estimate: 1m
Remaining Estimate: 1m
I want to scan rows by specified timestamp. I use following hbase shell
command :
scan 'testcrawl',{TIMESTAMP=1312268202071}
ROW COLUMN+CELL
put1.com column=crawl:data,
timestamp=1312268202071, value=htmlput1/html
put1.com column=crawl:type,
timestamp=1312268202071, value=html
put1.com column=links:outlinks,
timestamp=1312268202071, value=www.163.com;www.sina.com
As I expected, I can get the rows which timestamp is 1312268202071.
But when I use thift client to do the same thing ,the return data is the rows
which time before specified timestamp , not the same as hbase
shell.following is timestamp of return data:
131217917
1312268202059
I look up the source in
hbase/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java, it use
following code to set time parameter .
scan.setTimeRange(Long.MIN_VALUE, timestamp);
This cause thrift client return rows before specified row ,not the rows
timestamp specified.
But in hbase client and avro client ，it use following code to set time
parameter.
scan.setTimeStamp(timestamp);
this will return rows timestamp specified.
Is this a feature or a bug in thrift client ?
if this is a feature, which method in thrift client can get the rows by
specified timestamp?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7907) time range Scan to be made available via Thrift
Guido Serra aka Zeph created HBASE-7907:
---
Summary: time range Scan to be made available via Thrift
Key: HBASE-7907
URL: https://issues.apache.org/jira/browse/HBASE-7907
Project: HBase
Issue Type: New Feature
Reporter: Guido Serra aka Zeph
this is the mapping of the Scan Object in Thrift as of today at
-
http://svn.apache.org/viewvc/hbase/trunk/hbase-server/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift?view=markup
{code}
132 /**
133 * A Scan object is used to specify scanner parameters when opening a
scanner.
134 */
135 struct TScan {
136 1:optional Text startRow,
137 2:optional Text stopRow,
138 3:optional i64 timestamp,
139 4:optional listText columns,
140 5:optional i32 caching,
141 6:optional Text filterString
142 }
{code}
this is the Scan Object
- http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html
which has:
bq. To only retrieve columns within a specific range of version timestamps,
execute setTimeRange.
and
bq. To only retrieve columns with a specific timestamp, execute setTimestamp.
the second functionality/method is reachable, the first one setTimeRange() is
not (or at least at me) via Thrift
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[ https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584370#comment-13584370 ] Guido Serra aka Zeph commented on HBASE-5694: - confirmed on Version 0.92.1-cdh4.1.2, even worse, without specifying the columns, given a timestamp it behaves like a range filter from 0 (epoch) to timestamp -1 (basically an .. until, excluded) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly --- Key: HBASE-5694 URL: https://issues.apache.org/jira/browse/HBASE-5694 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.1 Reporter: Wouter Bolsterlee Fix For: 0.94.0 Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch The getRowsWithColumnsTs() method in the Thrift interface only applies the timestamp if columns are explicitly specified. However, this method also allows for columns to be unspecified (this is even used internally to implement e.g. getRows()). The cause of the bug is a minor scoping issue: the time range is set inside a wrong if statement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[ https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584386#comment-13584386 ] Wouter Bolsterlee commented on HBASE-5694: -- Guido, I think this patch has never made it into 0.92.x series, only to trunk at the time of committing (ended up in 0.94+ I think), so your observation that this bug is still present in 0.92.1 is correct. getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly --- Key: HBASE-5694 URL: https://issues.apache.org/jira/browse/HBASE-5694 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.1 Reporter: Wouter Bolsterlee Fix For: 0.94.0 Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch The getRowsWithColumnsTs() method in the Thrift interface only applies the timestamp if columns are explicitly specified. However, this method also allows for columns to be unspecified (this is even used internally to implement e.g. getRows()). The cause of the bug is a minor scoping issue: the time range is set inside a wrong if statement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7838) HBase regionserver never stops when running `hbase-daemon.sh stop regionserver`
[ https://issues.apache.org/jira/browse/HBASE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584395#comment-13584395 ] Hudson commented on HBASE-7838: --- Integrated in HBase-TRUNK #3891 (See [https://builds.apache.org/job/HBase-TRUNK/3891/]) HBASE-7838 HBase regionserver never stoping when running hbase-daemon.sh stop regionserver (Damien Hardy) - addendum (Revision 1448997) Result = FAILURE nkeywal : Files : * /hbase/trunk/bin/hbase-common.sh * /hbase/trunk/bin/hbase-daemon.sh * /hbase/trunk/bin/stop-hbase.sh HBase regionserver never stops when running `hbase-daemon.sh stop regionserver` --- Key: HBASE-7838 URL: https://issues.apache.org/jira/browse/HBASE-7838 Project: HBase Issue Type: Bug Components: regionserver, scripts, shell Affects Versions: 0.92.1, 0.96.0, 0.94.4 Environment: Using CDH4.1.2 Reporter: Damien Hardy Assignee: Damien Hardy Labels: newbie Fix For: 0.96.0 Attachments: 7838.addendum.v1.patch, HBASE-7838-0.94.patch, HBASE-7838-trunk.patch Currently running full stack (Hadoop + Hbase + zookeeper) on a VM I want to reboot, every services stopped but HBase regionserver. For 2 days now, it's aligning dots on the console ... I propose a timeout (default 600 overidable by $HBASE_STOP_TIMEOUT) to force kill -9 on the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7866) TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK failed 3 times in a row
[
https://issues.apache.org/jira/browse/HBASE-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-7866:
--
Attachment: HBASE-7866_0.94_1.patch
This is what i committed to 0.94.
Committed to trunk also.
Thanks for the review Ted and Lars.
TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK failed 3
times in a row
-
Key: HBASE-7866
URL: https://issues.apache.org/jira/browse/HBASE-7866
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0, 0.94.6
Attachments: HBASE-7866_0.94_1.patch, HBASE-7866_0.94.patch,
HBASE-7866_trunk_1.patch, HBASE-7866_trunk.patch, HBASE-7866_trunk.patch
Looks like the jenkins machines are flaky/slow again, causing this test to
fail. Same stacktrace all three times:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK(TestSplitTransactionOnCluster.java:656)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK(TestSplitTransactionOnCluster.java:608)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7866) TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK failed 3 times in a row
[
https://issues.apache.org/jira/browse/HBASE-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-7866:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK failed 3
times in a row
-
Key: HBASE-7866
URL: https://issues.apache.org/jira/browse/HBASE-7866
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0, 0.94.6
Attachments: HBASE-7866_0.94_1.patch, HBASE-7866_0.94.patch,
HBASE-7866_trunk_1.patch, HBASE-7866_trunk.patch, HBASE-7866_trunk.patch
Looks like the jenkins machines are flaky/slow again, causing this test to
fail. Same stacktrace all three times:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK(TestSplitTransactionOnCluster.java:656)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSplitBeforeSettingSplittingInZK(TestSplitTransactionOnCluster.java:608)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[
https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584412#comment-13584412
]
Guido Serra aka Zeph commented on HBASE-5694:
-
[~uws] if this is actually the patch (taken from above)
{code}
--- ThriftServer.java.orig 2012-04-01 23:41:16.881172406 +0200
+++ ThriftServer.java 2012-04-01 23:41:30.177238337 +0200
@@ -477,8 +477,8 @@
get.addColumn(famAndQf[0], famAndQf[1]);
}
}
-get.setTimeRange(Long.MIN_VALUE, timestamp);
}
+ get.setTimeRange(Long.MIN_VALUE, timestamp);
gets.add(get);
}
Result[] result = table.get(gets);
{code}
it is the wrong behavior that I'm getting, as it is inconsistent with the
scannerOpenWithScan
we shall not use the setTimeRange but the setTimestamp... as the signature in
Thrift states:
{code}
471 * Get the specified columns for the specified table and rows at the
specified
472 * timestamp. Returns an empty list if no rows exist.
{code}
and not a range scan from Long.MIN_VALUE to timestamp as implemented above
getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
---
Key: HBASE-5694
URL: https://issues.apache.org/jira/browse/HBASE-5694
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.92.1
Reporter: Wouter Bolsterlee
Fix For: 0.94.0
Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch
The getRowsWithColumnsTs() method in the Thrift interface only applies the
timestamp if columns are explicitly specified. However, this method also
allows for columns to be unspecified (this is even used internally to
implement e.g. getRows()). The cause of the bug is a minor scoping issue: the
time range is set inside a wrong if statement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7327) Assignment Timeouts: Remove the code from the master
[ https://issues.apache.org/jira/browse/HBASE-7327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584415#comment-13584415 ] ramkrishna.s.vasudevan commented on HBASE-7327: --- @N The following cases could cause some problem. -What if two assigns comes parallely and both of them fail. -What happens when two unassign call goes together and one among them is for disabling table. Another reason why things are working better is that the SSH side has been strengthened after HBASE-7521(HBASE-6060) has gone into both trunk and 0.94. Now Master clearly knows how to handle assign and the assign that happens thro SSH. One suggestion could be, For some time can we just change the code in such a way that TOM thread is not started at all. Later we can delete the code. What you feel? Assignment Timeouts: Remove the code from the master Key: HBASE-7327 URL: https://issues.apache.org/jira/browse/HBASE-7327 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7327.v1.uncomplete.patch, 7327.v2.patch As per HBASE-7247... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584420#comment-13584420 ] ramkrishna.s.vasudevan commented on HBASE-7878: --- Ok Ted. So now as per the patch if the recoverLease explicitly fails then we go and use append() and make recoverd true. If recoverLease() returns false, we still retry, this assumption is next time recoverLease would be successful right? May be better to add a configured number of retries and throw exception if retries are exhausted. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v1.txt, 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.3-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584436#comment-13584436 ] Ted Yu commented on HBASE-7904: --- I ran the failed tests locally and they passed. Integrated to trunk. Thanks for the review, Ram. Upgrade hadoop 2.0 dependency to 2.0.3-alpha Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.96.0 Attachments: 7904.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount
[ https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584437#comment-13584437 ] Tianying Chang commented on HBASE-7818: --- can someone review this patch for 0.94? Since there is no need to generate a trunk for this anymore. add region level metrics readReqeustCount and writeRequestCount Key: HBASE-7818 URL: https://issues.apache.org/jira/browse/HBASE-7818 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.94.4 Reporter: Tianying Chang Assignee: Tianying Chang Priority: Minor Fix For: 0.94.6 Attachments: HBASE-7818_1.patch, HBASE-7818.patch Request rate at region server level can help identify the hot region server. But it will be good if we can further identify the hot regions on that region server. That way, we can easily find out unbalanced regions problem. Currently, readRequestCount and writeReqeustCount per region is exposed at webUI. It will be more useful to expose it through hadoop metrics framework and/or JMX, so that people can see the history when the region is hot. I am exposing the existing readRequestCount/writeRequestCount into the dynamic region level metrics framework. I am not changing/exposing it as rate because our openTSDB is taking the raw data of read/write count, and apply rate function to display the rate already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7901) TestStore::testDeleteExpiredStoreFiles has a hidden issue
[ https://issues.apache.org/jira/browse/HBASE-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7901: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Integrated to trunk. Thanks for the patch, Sergey. TestStore::testDeleteExpiredStoreFiles has a hidden issue - Key: HBASE-7901 URL: https://issues.apache.org/jira/browse/HBASE-7901 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7901-v0.patch TestStore::testDeleteExpiredStoreFiles has what looks like a mistake in the j-loop, where i is checked in the 2nd part of for. It looks like the loop never executes, as if it did it would never terminate. If that is fixed, test fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7901) TestStore::testDeleteExpiredStoreFiles has a hidden issue
[ https://issues.apache.org/jira/browse/HBASE-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7901: -- Resolution: Fixed Status: Resolved (was: Patch Available) TestStore::testDeleteExpiredStoreFiles has a hidden issue - Key: HBASE-7901 URL: https://issues.apache.org/jira/browse/HBASE-7901 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7901-v0.patch TestStore::testDeleteExpiredStoreFiles has what looks like a mistake in the j-loop, where i is checked in the 2nd part of for. It looks like the loop never executes, as if it did it would never terminate. If that is fixed, test fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7878: -- Attachment: (was: 7878-trunk-v1.txt) recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584457#comment-13584457 ] Ted Yu commented on HBASE-7878: --- In my opinion, retry limit can be delegated to hdfs layer. Meaning, lease recovery shouldn't hang in hdfs layer. So I don't see a strong need for adding retry count. Thanks recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianying Chang updated HBASE-7896: -- Attachment: rename_table.rb 1. disable original table 'origin_name' 2. run the script: hbase org.jruby.Main rename_Table.rb origin_name new_name 3. disable table 'new_name' 4. enable table 'new_name' make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[
https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guido Serra aka Zeph updated HBASE-5694:
Attachment: setTimestamp.patch
up to me the correct patch shall be setTimestamp.patch that I computed
against origin/0.92.0rc4 from the github repository
{code}
index 231a564..4c46a4f 100644
--- a/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
+++ b/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
@@ -413,7 +413,7 @@ public class ThriftServer {
HTable table = getTable(tableName);
if (columns == null) {
Get get = new Get(getBytes(row));
- get.setTimeRange(Long.MIN_VALUE, timestamp);
+ get.setTimestamp(timestamp);
Result result = table.get(get);
return ThriftUtilities.rowResultFromHBase(result);
}
@@ -426,7 +426,7 @@ public class ThriftServer {
get.addColumn(famAndQf[0], famAndQf[1]);
}
}
-get.setTimeRange(Long.MIN_VALUE, timestamp);
+get.setTimestamp(timestamp);
Result result = table.get(get);
return ThriftUtilities.rowResultFromHBase(result);
} catch (IOException e) {
{code}
getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
---
Key: HBASE-5694
URL: https://issues.apache.org/jira/browse/HBASE-5694
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.92.1
Reporter: Wouter Bolsterlee
Fix For: 0.94.0
Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch,
setTimestamp.patch
The getRowsWithColumnsTs() method in the Thrift interface only applies the
timestamp if columns are explicitly specified. However, this method also
allows for columns to be unspecified (this is even used internally to
implement e.g. getRows()). The cause of the bug is a minor scoping issue: the
time range is set inside a wrong if statement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[ https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584467#comment-13584467 ] Ted Yu commented on HBASE-5694: --- @Guido: This issue has been resolved for 10 months. Mind opening a new JIRA and attach your patch there ? getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly --- Key: HBASE-5694 URL: https://issues.apache.org/jira/browse/HBASE-5694 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.1 Reporter: Wouter Bolsterlee Fix For: 0.94.0 Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch, setTimestamp.patch The getRowsWithColumnsTs() method in the Thrift interface only applies the timestamp if columns are explicitly specified. However, this method also allows for columns to be unspecified (this is even used internally to implement e.g. getRows()). The cause of the bug is a minor scoping issue: the time range is set inside a wrong if statement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[
https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584468#comment-13584468
]
Guido Serra aka Zeph commented on HBASE-5694:
-
argh... leave it... all of this is just WRONG...
{code}
if (tScan.isSetTimestamp()) {
scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp());
}
{code}
instead of exposing the setTimeRange on the Thrift interface someone decided to
hide it this way
getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
---
Key: HBASE-5694
URL: https://issues.apache.org/jira/browse/HBASE-5694
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.92.1
Reporter: Wouter Bolsterlee
Fix For: 0.94.0
Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch,
setTimestamp.patch
The getRowsWithColumnsTs() method in the Thrift interface only applies the
timestamp if columns are explicitly specified. However, this method also
allows for columns to be unspecified (this is even used internally to
implement e.g. getRows()). The cause of the bug is a minor scoping issue: the
time range is set inside a wrong if statement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[ https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584472#comment-13584472 ] Guido Serra aka Zeph commented on HBASE-5694: - [~yuzhih...@gmail.com] I will (sorry for this thread, I'd better go home and enjoy the weekend) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly --- Key: HBASE-5694 URL: https://issues.apache.org/jira/browse/HBASE-5694 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.1 Reporter: Wouter Bolsterlee Fix For: 0.94.0 Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch, setTimestamp.patch The getRowsWithColumnsTs() method in the Thrift interface only applies the timestamp if columns are explicitly specified. However, this method also allows for columns to be unspecified (this is even used internally to implement e.g. getRows()). The cause of the bug is a minor scoping issue: the time range is set inside a wrong if statement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount
[
https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584477#comment-13584477
]
Ted Yu commented on HBASE-7818:
---
For RegionMetricsStorage.java :
{code}
+import org.apache.hadoop.hbase.regionserver.HRegion;
{code}
The above import doesn't seem to be used.
Other than this, the patch is ready.
add region level metrics readReqeustCount and writeRequestCount
Key: HBASE-7818
URL: https://issues.apache.org/jira/browse/HBASE-7818
Project: HBase
Issue Type: Improvement
Components: metrics
Affects Versions: 0.94.4
Reporter: Tianying Chang
Assignee: Tianying Chang
Priority: Minor
Fix For: 0.94.6
Attachments: HBASE-7818_1.patch, HBASE-7818.patch
Request rate at region server level can help identify the hot region server.
But it will be good if we can further identify the hot regions on that region
server. That way, we can easily find out unbalanced regions problem.
Currently, readRequestCount and writeReqeustCount per region is exposed at
webUI. It will be more useful to expose it through hadoop metrics framework
and/or JMX, so that people can see the history when the region is hot.
I am exposing the existing readRequestCount/writeRequestCount into the
dynamic region level metrics framework. I am not changing/exposing it as rate
because our openTSDB is taking the raw data of read/write count, and apply
rate function to display the rate already.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7790) Refactor OpenRegionHandler so that the cleanup happens in one place - the finally block
[
https://issues.apache.org/jira/browse/HBASE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584486#comment-13584486
]
Ted Yu commented on HBASE-7790:
---
{code}
+ if (!openSuccessful) {
+doCleanUpOnFailedOpen(region, transitionedToOpening);
+ }
final Boolean current = this.rsServices.getRegionsInTransitionInRS().
remove(this.regionInfo.getEncodedNameAsBytes());
@@ -191,19 +184,35 @@
// 1) this.rsServices.addToOnlineRegions(region);
// 2) the ZK state.
if (openSuccessful) {
{code}
The variable current is only used in the if block where openSuccessful is true.
Maybe restructure the code such that assignment to current is wrapped in else
block ?
Patch looks good.
Refactor OpenRegionHandler so that the cleanup happens in one place - the
finally block
---
Key: HBASE-7790
URL: https://issues.apache.org/jira/browse/HBASE-7790
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-7790.patch
This is based on discussion in HBASE-7698. Jimmy suggested this improvment.
Look at
https://issues.apache.org/jira/browse/HBASE-7698?focusedCommentId=13572736page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13572736
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.3-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584497#comment-13584497
]
Ted Yu commented on HBASE-7904:
---
Reverted due to the following:
{code}
org.apache.hadoop.yarn.YarnException: Failed to Start
org.apache.hadoop.mapred.MiniMRCluster
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
at
org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
at org.apache.hadoop.mapred.MiniMRCluster.init(MiniMRCluster.java:180)
at org.apache.hadoop.mapred.MiniMRCluster.init(MiniMRCluster.java:170)
at org.apache.hadoop.mapred.MiniMRCluster.init(MiniMRCluster.java:162)
at org.apache.hadoop.mapred.MiniMRCluster.init(MiniMRCluster.java:123)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
...
Caused by: org.apache.hadoop.yarn.YarnException:
java.lang.IllegalStateException: Queue configuration missing child queue names
for root
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
... 38 more
Caused by: java.lang.IllegalStateException: Queue configuration missing child
queue names for root
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:255)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:220)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:161)
{code}
Upgrade hadoop 2.0 dependency to 2.0.3-alpha
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.96.0
Attachments: 7904.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount
[ https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianying Chang updated HBASE-7818: -- Attachment: HBASE-7818_2.patch Ted, you are right. I introduced that when I was having more LOG output for debugging. I have deleted that import and uploaded the new patch. Thanks. add region level metrics readReqeustCount and writeRequestCount Key: HBASE-7818 URL: https://issues.apache.org/jira/browse/HBASE-7818 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.94.4 Reporter: Tianying Chang Assignee: Tianying Chang Priority: Minor Fix For: 0.94.6 Attachments: HBASE-7818_1.patch, HBASE-7818_2.patch, HBASE-7818.patch Request rate at region server level can help identify the hot region server. But it will be good if we can further identify the hot regions on that region server. That way, we can easily find out unbalanced regions problem. Currently, readRequestCount and writeReqeustCount per region is exposed at webUI. It will be more useful to expose it through hadoop metrics framework and/or JMX, so that people can see the history when the region is hot. I am exposing the existing readRequestCount/writeRequestCount into the dynamic region level metrics framework. I am not changing/exposing it as rate because our openTSDB is taking the raw data of read/write count, and apply rate function to display the rate already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount
[ https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584504#comment-13584504 ] Ted Yu commented on HBASE-7818: --- @Tianying: Do you mind running test suite for 0.94 and give us the result ? Thanks add region level metrics readReqeustCount and writeRequestCount Key: HBASE-7818 URL: https://issues.apache.org/jira/browse/HBASE-7818 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.94.4 Reporter: Tianying Chang Assignee: Tianying Chang Priority: Minor Fix For: 0.94.6 Attachments: HBASE-7818_1.patch, HBASE-7818_2.patch, HBASE-7818.patch Request rate at region server level can help identify the hot region server. But it will be good if we can further identify the hot regions on that region server. That way, we can easily find out unbalanced regions problem. Currently, readRequestCount and writeReqeustCount per region is exposed at webUI. It will be more useful to expose it through hadoop metrics framework and/or JMX, so that people can see the history when the region is hot. I am exposing the existing readRequestCount/writeRequestCount into the dynamic region level metrics framework. I am not changing/exposing it as rate because our openTSDB is taking the raw data of read/write count, and apply rate function to display the rate already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4755) HBase based block placement in DFS
[ https://issues.apache.org/jira/browse/HBASE-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-4755: --- Priority: Critical (was: Major) HBase based block placement in DFS -- Key: HBASE-4755 URL: https://issues.apache.org/jira/browse/HBASE-4755 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Karthik Ranganathan Assignee: Christopher Gist Priority: Critical Attachments: 4755-wip-1.patch The feature as is only useful for HBase clusters that care about data locality on regionservers, but this feature can also enable a lot of nice features down the road. The basic idea is as follows: instead of letting HDFS determine where to replicate data (r=3) by place blocks on various regions, it is better to let HBase do so by providing hints to HDFS through the DFS client. That way instead of replicating data at a blocks level, we can replicate data at a per-region level (each region owned by a promary, a secondary and a tertiary regionserver). This is better for 2 things: - Can make region failover faster on clusters which benefit from data affinity - On large clusters with random block placement policy, this helps reduce the probability of data loss The algo is as follows: - Each region in META will have 3 columns which are the preferred regionservers for that region (primary, secondary and tertiary) - Preferred assignment can be controlled by a config knob - Upon cluster start, HMaster will enter a mapping from each region to 3 regionservers (random hash, could use current locality, etc) - The load balancer would assign out regions preferring region assignments to primary over secondary over tertiary over any other node - Periodically (say weekly, configurable) the HMaster would run a locality checked and make sure the map it has for region to regionservers is optimal. Down the road, this can be enhanced to control region placement in the following cases: - Mixed hardware SKU where some regionservers can hold fewer regions - Load balancing across tables where we dont want multiple regions of a table to get assigned to the same regionservers - Multi-tenancy, where we can restrict the assignment of the regions of some table to a subset of regionservers, so an abusive app cannot take down the whole HBase cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-7896 started by Tianying Chang. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work stopped] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-7896 stopped by Tianying Chang. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7818) add region level metrics readReqeustCount and writeRequestCount
[ https://issues.apache.org/jira/browse/HBASE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584527#comment-13584527 ] Tianying Chang commented on HBASE-7818: --- sure, I ran it before. Let me run again to make sure. add region level metrics readReqeustCount and writeRequestCount Key: HBASE-7818 URL: https://issues.apache.org/jira/browse/HBASE-7818 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.94.4 Reporter: Tianying Chang Assignee: Tianying Chang Priority: Minor Fix For: 0.94.6 Attachments: HBASE-7818_1.patch, HBASE-7818_2.patch, HBASE-7818.patch Request rate at region server level can help identify the hot region server. But it will be good if we can further identify the hot regions on that region server. That way, we can easily find out unbalanced regions problem. Currently, readRequestCount and writeReqeustCount per region is exposed at webUI. It will be more useful to expose it through hadoop metrics framework and/or JMX, so that people can see the history when the region is hot. I am exposing the existing readRequestCount/writeRequestCount into the dynamic region level metrics framework. I am not changing/exposing it as rate because our openTSDB is taking the raw data of read/write count, and apply rate function to display the rate already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.3-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7904: -- Issue Type: Task (was: Bug) Upgrade hadoop 2.0 dependency to 2.0.3-alpha Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.96.0 Attachments: 7904.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.3-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584536#comment-13584536
]
Ted Yu commented on HBASE-7904:
---
Looks like MiniMRCluster is deprecated:
{code}
* @deprecated Use {@link org.apache.hadoop.mapred.MiniMRClientClusterFactory}
* instead
*/
@Deprecated
public class MiniMRCluster {
{code}
Upgrade hadoop 2.0 dependency to 2.0.3-alpha
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.96.0
Attachments: 7904.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7879) JUnit dependency in main from htrace
[ https://issues.apache.org/jira/browse/HBASE-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584537#comment-13584537 ] Jonathan Leavitt commented on HBASE-7879: - I did a release of htrace to 1.50 that just has the dependency scope change for junit and no other changes. JUnit dependency in main from htrace Key: HBASE-7879 URL: https://issues.apache.org/jira/browse/HBASE-7879 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal Priority: Minor HTrace main depends on Junit , it should be only test. I created a junit in the github, that's https://github.com/cloudera/htrace/issues/1. If it's not fixed, we will be able to drop it in our pom, but let's wait a little before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5416) Improve performance of scans with some kind of filters.
[ https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584553#comment-13584553 ] Dave Latham commented on HBASE-5416: I have a class that directly implements the Filter interface. This change looks to me like it will prevent me from doing a rolling upgrade to 0.94.5 of region servers while my client is using this filter on scans because the filter will fail to implement the changed interface. Is that correct? Is that acceptable? Improve performance of scans with some kind of filters. --- Key: HBASE-5416 URL: https://issues.apache.org/jira/browse/HBASE-5416 Project: HBase Issue Type: Improvement Components: Filters, Performance, regionserver Affects Versions: 0.90.4 Reporter: Max Lapan Assignee: Sergey Shelukhin Fix For: 0.96.0, 0.94.5 Attachments: 5416-0.94-v1.txt, 5416-0.94-v2.txt, 5416-0.94-v3.txt, 5416-Filtered_scans_v6.patch, 5416-v13.patch, 5416-v14.patch, 5416-v15.patch, 5416-v16.patch, 5416-v5.txt, 5416-v6.txt, Filtered_scans.patch, Filtered_scans_v2.patch, Filtered_scans_v3.patch, Filtered_scans_v4.patch, Filtered_scans_v5.1.patch, Filtered_scans_v5.patch, Filtered_scans_v7.patch, HBASE-5416-v10.patch, HBASE-5416-v11.patch, HBASE-5416-v12.patch, HBASE-5416-v12.patch, HBASE-5416-v7-rebased.patch, HBASE-5416-v8.patch, HBASE-5416-v9.patch, org.apache.hadoop.hbase.regionserver.TestHRegion-output.txt When the scan is performed, whole row is loaded into result list, after that filter (if exists) is applied to detect that row is needed. But when scan is performed on several CFs and filter checks only data from the subset of these CFs, data from CFs, not checked by a filter is not needed on a filter stage. Only when we decided to include current row. And in such case we can significantly reduce amount of IO performed by a scan, by loading only values, actually checked by a filter. For example, we have two CFs: flags and snap. Flags is quite small (bunch of megabytes) and is used to filter large entries from snap. Snap is very large (10s of GB) and it is quite costly to scan it. If we needed only rows with some flag specified, we use SingleColumnValueFilter to limit result to only small subset of region. But current implementation is loading both CFs to perform scan, when only small subset is needed. Attached patch adds one routine to Filter interface to allow filter to specify which CF is needed to it's operation. In HRegion, we separate all scanners into two groups: needed for filter and the rest (joined). When new row is considered, only needed data is loaded, filter applied, and only if filter accepts the row, rest of data is loaded. At our data, this speeds up such kind of scans 30-50 times. Also, this gives us the way to better normalize the data into separate columns by optimizing the scans performed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584558#comment-13584558 ] Tianying Chang commented on HBASE-7896: --- The high level summary of renaming a table is like this: create hdfs dir for the new table; rename the Hfile from old table dir into new table dir; delete the entry for old table from META; insert the entry for the new table into META; manipulate zookeeper to create a znode for the new table. Because this is an un-conventional way of creating a new table, the master is not aware of the new table. That is why I manipulate the zookeeper by explicitly inserting a znode for the new table. But master still does not read in the full information of the new table, so it is not REALLY enabled. By inserting the znode with enable state, we can make it officially enabled by calling disable/enable sequence on this new table without needing to restart master, because it invoked the master to learn of the new table. This is just my quick way to make rename_table.rb work for our production cluster. My next step plan is 1. make master aware of the newly renamed table immediately without explicit znode manipulation and table disable/enable. This will probably need to expose some API from Master 2. make rename_table as 2pc with roll back capability. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Meil updated HBASE-7221:
-
Attachment: hbase-server_hbase_7221_v6.patch
Made RowKeyDataConverter an abstract class
Created RowKeyBytesDataConverter the default implementation (i.e., uses Bytes
utility)
Added Javadoc explanation of the implications of Bytes sort order in
RowKeyBytesDataConverter.
Added reference in RowKeySchema that RowKeyBytesDataConverter is the default
strategy.
RowKey utility class for rowkey construction
Key: HBASE-7221
URL: https://issues.apache.org/jira/browse/HBASE-7221
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
A common question in the dist-lists is how to construct rowkeys, particularly
composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
you to sensibly populate that byte-array, and that's where things tend to go
off the rails.
The intent of this RowKey utility class isn't meant to add functionality into
Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
Example:
{code}
RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
key.addHash(a);
key.add(b);
byte bytes[] = key.getBytes();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-6469:
--
Summary: Failure on enable/disable table will cause table state in zk to be
left as enabling/disabling until master is restarted (was: Failure on
enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restart)
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restarted
---
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.96.0, 0.94.6
Attachments: 6469-expose-force-r3.patch, HBASE-6469.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7507) Make memstore flush be able to retry after exception
[
https://issues.apache.org/jira/browse/HBASE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584563#comment-13584563
]
Lars Hofhansl commented on HBASE-7507:
--
I don't know whether this caused the test failures. At this point I'd say
probably not.
IMHO, this particular fix is only important if we have fixed all other write
attempts for HDFS.
Make memstore flush be able to retry after exception
Key: HBASE-7507
URL: https://issues.apache.org/jira/browse/HBASE-7507
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Critical
Fix For: 0.96.0
Attachments: 7507-94.patch, 7507-trunk v1.patch, 7507-trunk v2.patch,
7507-trunkv3.patch
We will abort regionserver if memstore flush throws exception.
I thinks we could do retry to make regionserver more stable because file
system may be not ok in a transient time. e.g. Switching namenode in the
NamenodeHA environment
{code}
HRegion#internalFlushcache(){
...
try {
...
}catch(Throwable t){
DroppedSnapshotException dse = new DroppedSnapshotException(region: +
Bytes.toStringBinary(getRegionName()));
dse.initCause(t);
throw dse;
}
...
}
MemStoreFlusher#flushRegion(){
...
region.flushcache();
...
try {
}catch(DroppedSnapshotException ex){
server.abort(Replay of HLog required. Forcing server shutdown, ex);
}
...
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584565#comment-13584565
]
Doug Meil commented on HBASE-7221:
--
v6 patch has been added with Javadoc changes, plus making RowKeyDataConverter
and abstract class with RowKeyBytesDataConverter the default implementation
(it's still a pluggable pattern as before though).
RowKey utility class for rowkey construction
Key: HBASE-7221
URL: https://issues.apache.org/jira/browse/HBASE-7221
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
A common question in the dist-lists is how to construct rowkeys, particularly
composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
you to sensibly populate that byte-array, and that's where things tend to go
off the rails.
The intent of this RowKey utility class isn't meant to add functionality into
Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
Example:
{code}
RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
key.addHash(a);
key.add(b);
byte bytes[] = key.getBytes();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7838) HBase regionserver never stops when running `hbase-daemon.sh stop regionserver`
[ https://issues.apache.org/jira/browse/HBASE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584569#comment-13584569 ] Lars Hofhansl commented on HBASE-7838: -- What about JM's suggestion to take a jstack of the process before we kill -9 it? Otherwise we'll diminish our chances to identify the root issue. HBase regionserver never stops when running `hbase-daemon.sh stop regionserver` --- Key: HBASE-7838 URL: https://issues.apache.org/jira/browse/HBASE-7838 Project: HBase Issue Type: Bug Components: regionserver, scripts, shell Affects Versions: 0.92.1, 0.96.0, 0.94.4 Environment: Using CDH4.1.2 Reporter: Damien Hardy Assignee: Damien Hardy Labels: newbie Fix For: 0.96.0 Attachments: 7838.addendum.v1.patch, HBASE-7838-0.94.patch, HBASE-7838-trunk.patch Currently running full stack (Hadoop + Hbase + zookeeper) on a VM I want to reboot, every services stopped but HBase regionserver. For 2 days now, it's aligning dots on the console ... I propose a timeout (default 600 overidable by $HBASE_STOP_TIMEOUT) to force kill -9 on the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584570#comment-13584570 ] Jean-Marc Spaggiari commented on HBASE-7896: Is there anyone working on HBASE-643 to implement a table id - table name link? Because that will definitively fix/simplify the rename logic. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7838) HBase regionserver never stops when running `hbase-daemon.sh stop regionserver`
[ https://issues.apache.org/jira/browse/HBASE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584576#comment-13584576 ] Jean-Marc Spaggiari commented on HBASE-7838: Hi Lars, I'm working on that. I will open another JIRA and submit the patch. I agree that keep the jstack trace each time we have to do a kill -9 will be usefull to track potential issues. HBase regionserver never stops when running `hbase-daemon.sh stop regionserver` --- Key: HBASE-7838 URL: https://issues.apache.org/jira/browse/HBASE-7838 Project: HBase Issue Type: Bug Components: regionserver, scripts, shell Affects Versions: 0.92.1, 0.96.0, 0.94.4 Environment: Using CDH4.1.2 Reporter: Damien Hardy Assignee: Damien Hardy Labels: newbie Fix For: 0.96.0 Attachments: 7838.addendum.v1.patch, HBASE-7838-0.94.patch, HBASE-7838-trunk.patch Currently running full stack (Hadoop + Hbase + zookeeper) on a VM I want to reboot, every services stopped but HBase regionserver. For 2 days now, it's aligning dots on the console ... I propose a timeout (default 600 overidable by $HBASE_STOP_TIMEOUT) to force kill -9 on the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584577#comment-13584577
]
Lars Hofhansl commented on HBASE-7221:
--
What I was trying to say is that in the builder we should be able to indicate
whether a key part is sorting ascending or descending, even though this is not
used by the default converter. We can default it to ascending.
I think without this it is not future proof. Does this make sense?
RowKey utility class for rowkey construction
Key: HBASE-7221
URL: https://issues.apache.org/jira/browse/HBASE-7221
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
A common question in the dist-lists is how to construct rowkeys, particularly
composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
you to sensibly populate that byte-array, and that's where things tend to go
off the rails.
The intent of this RowKey utility class isn't meant to add functionality into
Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
Example:
{code}
RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
key.addHash(a);
key.add(b);
byte bytes[] = key.getBytes();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7305) ZK based Read/Write locks for table operations
[ https://issues.apache.org/jira/browse/HBASE-7305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584581#comment-13584581 ] Enis Soztutar commented on HBASE-7305: -- Thanks Ram for checking this out. Yes in this patch, we only add write locking in most of the table operations (except for offlineRegion, moveRegion, etc). There is a follow up patch, HBASE-7546, which adds read locking for region splits. Do you think that we should acquire a read lock, for balancer, or AM, SSH operations? ZK based Read/Write locks for table operations -- Key: HBASE-7305 URL: https://issues.apache.org/jira/browse/HBASE-7305 Project: HBase Issue Type: Bug Components: Client, master, Zookeeper Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: 7305-v11.txt, hbase-7305_v0.patch, hbase-7305_v10.patch, hbase-7305_v13.patch, hbase-7305_v14.patch, hbase-7305_v15.patch, hbase-7305_v1-based-on-curator.patch, hbase-7305_v2.patch, hbase-7305_v4.patch, hbase-7305_v9.patch This has started as forward porting of HBASE-5494 and HBASE-5991 from the 89-fb branch to trunk, but diverged enough to have it's own issue. The idea is to implement a zk based read/write lock per table. Master initiated operations should get the write lock, and region operations (region split, moving, balance?, etc) acquire a shared read lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7891) Add an index number prior to each table region in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584587#comment-13584587 ] Nick Dimiduk commented on HBASE-7891: - This is potentially confusing to a user who might expect the index number to map uniquely to a region. Huh? Why did the name of region #3 change? I think the label Count would carry less potential for confusion. Add an index number prior to each table region in table.jsp --- Key: HBASE-7891 URL: https://issues.apache.org/jira/browse/HBASE-7891 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.5 Reporter: Maryann Xue Assignee: Maryann Xue Priority: Trivial Attachments: HBASE-7891-0.94.patch Adding an index number for each table region in table.jsp would make it easier to locate a region or to count regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584597#comment-13584597
]
Doug Meil commented on HBASE-7221:
--
I think configurable sort order is a good idea, but I think that is another
ticket. For what this proposal was trying to achieve it addresses the
concerns. Is that reasonable?
RowKey utility class for rowkey construction
Key: HBASE-7221
URL: https://issues.apache.org/jira/browse/HBASE-7221
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
A common question in the dist-lists is how to construct rowkeys, particularly
composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
you to sensibly populate that byte-array, and that's where things tend to go
off the rails.
The intent of this RowKey utility class isn't meant to add functionality into
Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
Example:
{code}
RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
key.addHash(a);
key.add(b);
byte bytes[] = key.getBytes();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7546) Obtain a table read lock on region split operations
[ https://issues.apache.org/jira/browse/HBASE-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7546: - Attachment: hbase-7546_v3.patch Updated patch with Ted's comments Obtain a table read lock on region split operations --- Key: HBASE-7546 URL: https://issues.apache.org/jira/browse/HBASE-7546 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7546_v1.patch, hbase-7546_v2.patch, hbase-7546_v3.patch As discussed in the parent issue HBASE-7305, we should be coordinating between splits and table operations to ensure that they don't happen at the same time. In this issue we will acquire shared read locks for region splits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584601#comment-13584601 ] Enis Soztutar commented on HBASE-7411: -- [~randgalt] This is great news. Let me know if you need any help in Incubator. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584604#comment-13584604 ] Andrew Purtell commented on HBASE-7411: --- bq. FYI - Curator will soon be going into Incubator. Stay tuned.. I'll hope for the best. The second that happens you can consider any objection I have here withdrawn. Use Netflix's Curator zookeeper library --- Key: HBASE-7411 URL: https://issues.apache.org/jira/browse/HBASE-7411 Project: HBase Issue Type: New Feature Components: Zookeeper Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-7411_v0.patch We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere but we can continue the discussion in this. The advantages for the curator lib over ours are the recipes. We have very similar retrying mechanism, and we don't need much of the nice client-API layer. We also have similar Listener interface, etc. I think we can decide on one of the following options: 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some code from there. 2. Replace all of our zk usage / connection management to curator. We may keep the current set of API's as a thin wrapper. 3. Use our own connection management / retry logic, and build a custom CuratorFramework implementation for the curator recipes. This will keep the current zk logic/code intact, and allow us to use curator-recipes as we see fit. 4. Allow both curator and our zk layer to manage the connection. We will still have 1 connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak show? I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. Related issues: HBASE-5547 HBASE-7305 HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7908) Add jstack logs in hbase-common.sh when killing the server.
Jean-Marc Spaggiari created HBASE-7908: -- Summary: Add jstack logs in hbase-common.sh when killing the server. Key: HBASE-7908 URL: https://issues.apache.org/jira/browse/HBASE-7908 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor It might be a good idea to keep the jstack logs when killing -9 never ending servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7908) Add jstack logs in hbase-common.sh when killing the server.
[ https://issues.apache.org/jira/browse/HBASE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7908: --- Status: Patch Available (was: Open) Might be working on 0.94 too.. Add jstack logs in hbase-common.sh when killing the server. --- Key: HBASE-7908 URL: https://issues.apache.org/jira/browse/HBASE-7908 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Attachments: HBASE-7908-v0-trunk.patch It might be a good idea to keep the jstack logs when killing -9 never ending servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7908) Add jstack logs in hbase-common.sh when killing the server.
[ https://issues.apache.org/jira/browse/HBASE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7908: --- Attachment: HBASE-7908-v0-trunk.patch Add jstack logs in hbase-common.sh when killing the server. --- Key: HBASE-7908 URL: https://issues.apache.org/jira/browse/HBASE-7908 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Attachments: HBASE-7908-v0-trunk.patch It might be a good idea to keep the jstack logs when killing -9 never ending servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584634#comment-13584634
]
Hadoop QA commented on HBASE-7221:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570515/hbase-server_hbase_7221_v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4497//console
This message is automatically generated.
RowKey utility class for rowkey construction
Key: HBASE-7221
URL: https://issues.apache.org/jira/browse/HBASE-7221
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
A common question in the dist-lists is how to construct rowkeys, particularly
composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
you to sensibly populate that byte-array, and that's where things tend to go
off the rails.
The intent of this RowKey utility class isn't meant to add functionality into
Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
Example:
{code}
RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
key.addHash(a);
key.add(b);
byte bytes[] = key.getBytes();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584642#comment-13584642 ] Matteo Bertozzi commented on HBASE-7896: [~jmspaggi] having an id can be a quick a dirty fix, but is more painful than what you think. Also this fs layout with the table names hardcoded limits the new features that we can do like deduplication, handling blocks to have a proper compaction without rewriting the whole file co, already snapshot are more like a workaround due to the current fs layout. Anway HBASE-7806 is the beginning of what you're looking for. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584648#comment-13584648 ] Jean-Marc Spaggiari commented on HBASE-7896: Thanks [~mbertozzi]. I will look at HBASE-7806. If you need some hands for it, just let me know. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7843) enable encapsulating compaction policy/compactor/store file manager interaction shennanigans
[ https://issues.apache.org/jira/browse/HBASE-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584675#comment-13584675 ] Matteo Bertozzi commented on HBASE-7843: patch looks good to me, there's an open Major Compaction question (on reviewboard) that can be moved to another jira. enable encapsulating compaction policy/compactor/store file manager interaction shennanigans Key: HBASE-7843 URL: https://issues.apache.org/jira/browse/HBASE-7843 Project: HBase Issue Type: Improvement Components: Compaction Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7843-v0.patch, HBASE-7843-v1.patch To avoid massive casting and/or deciphering of structures traveling between SFM, compaction policy, and compactor in non-trivial compaction schemes like stripe or level, we need to make interaction between themselves hidden. Elsewhere, the changes are being made to coprocessor for compactions that will make CompactionRequest a limited-visibility class for users, with coprocessors being able to subclass and return it. -This seems like a viable solution for the problem at hand too. Policy will (optionally) subclass compaction request and return it. Instead of calling something.compact(req), req.compact() will be called (with something already stored inside req as of now), after which, magic will happen.- After merging that code I actually see that subclassing compactionrequest in both will break the policy, or require bunch of ugly code in coprocessors (a subclass for every policy, and telling them apart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7908) Add jstack logs in hbase-common.sh when killing the server.
[
https://issues.apache.org/jira/browse/HBASE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584684#comment-13584684
]
Hadoop QA commented on HBASE-7908:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570524/HBASE-7908-v0-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestZKProcedureControllers
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4498//console
This message is automatically generated.
Add jstack logs in hbase-common.sh when killing the server.
---
Key: HBASE-7908
URL: https://issues.apache.org/jira/browse/HBASE-7908
Project: HBase
Issue Type: Improvement
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Minor
Attachments: HBASE-7908-v0-trunk.patch
It might be a good idea to keep the jstack logs when killing -9 never ending
servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Open (was: Patch Available) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Attachment: HBASE-7692.v4.patch Patch v4 addresses issues raised in this [thread on hbase-dev|http://mail-archives.apache.org/mod_mbox/hbase-dev/201302.mbox/%3CCANZa=gsdzg-5pptq_vswjdwo+jn8hhusvq1wzgp9q86g4dp...@mail.gmail.com%3E]. Specifically, it moves the library out into its own module: hbase-orderly. It also shortens the package name to org.apache.hadoop.hbase.orderly. Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Patch Available (was: Open) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584701#comment-13584701
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570535/HBASE-7692.v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 74 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4499//console
This message is automatically generated.
Add utility class to generate ordered byte[] serialization
--
Key: HBASE-7692
URL: https://issues.apache.org/jira/browse/HBASE-7692
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
HBASE-7692.v3.patch, HBASE-7692.v4.patch
The current Bytes utility class works, but produces output that does not
maintain the native sort ordering of the input value. This results in, for
example, a negative value that does not necessarily sort before a positive
value. HBase should provide a canonical implementation of such a
serialization format so that third-parties can reliably build on top of
HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584720#comment-13584720
]
Nick Dimiduk commented on HBASE-7692:
-
This implementation is mostly a superset of functionality described/requested
in HBASE-7221. The example provided in that ticket's description is implemented
using orderly like so:
{code}
StructRowKey rowkey = new StructBuilder()
.add(new FixedByteArrayRowKey(16))
.add(new LongRowKey())
.toRowKey();
byte[] bytes = rowkey.serialize(new Object[] { a, b });
{code}
In this case, the hash operation must be supplied by the user.
Consuming the serialized bytes is pretty easy too. Here's an example of basic
iteration:
{code}
rowkey.iterateOver(bytes);
for (Object f : rowkey) {
System.out.println(f);
}
{code}
The attached patches includes a number of
[examples|https://github.com/ndimiduk/hbase/tree/7692-ordered-serialization/hbase-examples/src/main/java/org/apache/hadoop/hbase/orderly].
(cc [~dmeil], [~lhofhansl])
Add utility class to generate ordered byte[] serialization
--
Key: HBASE-7692
URL: https://issues.apache.org/jira/browse/HBASE-7692
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
HBASE-7692.v3.patch, HBASE-7692.v4.patch
The current Bytes utility class works, but produces output that does not
maintain the native sort ordering of the input value. This results in, for
example, a negative value that does not necessarily sort before a positive
value. HBase should provide a canonical implementation of such a
serialization format so that third-parties can reliably build on top of
HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7909) How to figure if a Cell is deep or shallow.
stack created HBASE-7909: Summary: How to figure if a Cell is deep or shallow. Key: HBASE-7909 URL: https://issues.apache.org/jira/browse/HBASE-7909 Project: HBase Issue Type: Task Reporter: stack Priority: Minor The CellScanner interface is how you iterate scanners. It is more bare bones than java Iterator, explicitly so, to minimize the need for retaining references to the current Cell. The Interface currently has get/current to pull the Cell that is currently loaded in the breech. It also has (had) another method getDeepCopy. This latter was removed by hbase-7899 Tools to build cell blocks as suggested by [~mcorgan] (and seconded by other reviewers in that they found it problematic). So, how then to determine if the Cell you have is a deep or shallow copy? On the one hand, should we even be concerned? The whole point of our Cell retrofit, in part, is to force us disconnect from how the Cell is implemented so maybe we should just do away w/ this notion of deepCopy altogether and hope that in action, we don't actually need it and that we our fixation is only because deepCopies is all we ever had when we were exclusively KeyValue. Or, do we need to add a means of asking a Cell Are you deep? or having deepCopies implement a subInterface -- StableCell or StandaloneCell? This issue raises the problem but I do not think it critical we deal with it just now. At least, I do not see imminent need, at least not currently where we are still Cell backed by deepCopy KeyValues. Maybe later when we have different implementations this issue will come to the fore. Until then, am fine leaving it as minor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7649) client retry timeout doesn't need to do x2 fallback when going to different server
[ https://issues.apache.org/jira/browse/HBASE-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7649: Attachment: HBASE-7649-v5.patch Rebase the patch, some changes. Some fixes were now integrated so I will need to rerun the integration tests to see if there are glitches. client retry timeout doesn't need to do x2 fallback when going to different server -- Key: HBASE-7649 URL: https://issues.apache.org/jira/browse/HBASE-7649 Project: HBase Issue Type: Improvement Components: Client Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7649-v0.patch, HBASE-7649-v1.patch, HBASE-7649-v2.patch, HBASE-7649-v2.patch, HBASE-7649-v2.patch, HBASE-7649-v3.patch, HBASE-7649-v4.patch, HBASE-7649-v5.patch See HBASE-7520. When we go to server A, get a bunch of failures, then finally learn the region is on B it doesn't make sense to wait for 30 seconds before going to B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584743#comment-13584743
]
Ted Yu commented on HBASE-6469:
---
Overall, looks quite good.
{code}
+ public class TableStateUpdater extends Chore {
{code}
This class is internal to AssignmentManager and can be declared private.
{code}
+protected void chore() {
+ if (!tablesInPartialState.isEmpty()) {
{code}
If you write the condition differently, you should be able to save some
indentation.
{code}
+} catch (KeeperException e) {
+ LOG.error(Error trying to enable the table +
tableEntry.getKey(), e);
+}
{code}
With the current position of the catch block, you need some flag so that
'enable' and 'disable' can be selected in the log.
{code}
+ if (tableEntry.getValue() == regionStates.getRegionsOfTable(
+ Bytes.toBytes(tableEntry.getKey())).size()) {
{code}
Is it possible that region count gets bigger than tableEntry.getValue() ? If
not, add PreCondition ?
{code}
+MapString, Integer tablesInPartialState = new TreeMapString, Integer();
{code}
nit: if you use byte[] as key (with proper comparator), you can save the
Bytes.toBytes() call.
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restarted
---
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.96.0, 0.94.6
Attachments: 6469-expose-force-r3.patch, HBASE-6469.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7791) Compaction USER_PRIORITY is slightly broken
[
https://issues.apache.org/jira/browse/HBASE-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584746#comment-13584746
]
Sergey Shelukhin commented on HBASE-7791:
-
bq. It should be increase by one?
Higher number means lower pri, so logically we reduce priority.
bq. But if really a compaction is USER requested then what is the priority. And
even in that case we increase the priority?
No, this happens only if the priority passed in is not user, see the
surrounding if
Compaction USER_PRIORITY is slightly broken
---
Key: HBASE-7791
URL: https://issues.apache.org/jira/browse/HBASE-7791
Project: HBase
Issue Type: Bug
Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
Attachments: HBASE-7791-v0.patch
The code to get compaction priority is as such:
{code} public int getCompactPriority(int priority) {
// If this is a user-requested compaction, leave this at the highest
priority
if(priority == Store.PRIORITY_USER) {
return Store.PRIORITY_USER;
} else {
return this.blockingStoreFileCount - this.storefiles.size();
}
}
{code}.
PRIORITY_USER is 1.
The priorities are compared as numbers in HRegion, so compactions of blocking
stores will override user priority (probably intended); also, if you have
blockingFiles minus one, your priority is suddenly PRIORITY_USER, which may
cause at least this:
LOG.debug(Warning, compacting more than +
comConf.getMaxFilesToCompact() +
files because of a user-requested major compaction);
as well as some misleading logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7649) client retry timeout doesn't need to do x2 fallback when going to different server
[ https://issues.apache.org/jira/browse/HBASE-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7649: Status: Patch Available (was: Open) client retry timeout doesn't need to do x2 fallback when going to different server -- Key: HBASE-7649 URL: https://issues.apache.org/jira/browse/HBASE-7649 Project: HBase Issue Type: Improvement Components: Client Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-7649-v0.patch, HBASE-7649-v1.patch, HBASE-7649-v2.patch, HBASE-7649-v2.patch, HBASE-7649-v2.patch, HBASE-7649-v3.patch, HBASE-7649-v4.patch, HBASE-7649-v5.patch See HBASE-7520. When we go to server A, get a bunch of failures, then finally learn the region is on B it doesn't make sense to wait for 30 seconds before going to B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7906) OfflineCallBack in bulk assignment does not work as expected
[ https://issues.apache.org/jira/browse/HBASE-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584747#comment-13584747 ] Sergey Shelukhin commented on HBASE-7906: - Is this a duplicate of HBASE-7859 ? OfflineCallBack in bulk assignment does not work as expected Key: HBASE-7906 URL: https://issues.apache.org/jira/browse/HBASE-7906 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan This is as part of discussion in HBASE-7799. The OfflineCallBack does not bother if the znode already exists or not. Infact the catch block AM.asyncSetOfflineInZooKeeper() is a dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7824) Improve master start up time when there is log split work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-7824: - Attachment: hbase-7824.patch Unblock master starts up when there are non-meta region servers' split work. After the fix, master will only block by meta region server log spliting work. More details as following: 1) Adding support to fast master up 2) Adding a test case for new code 3) Fix test function makehlog to creat rows with keys within assigned region's key space Improve master start up time when there is log split work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.6 Attachments: hbase-7824.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log split work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584762#comment-13584762 ] Jean-Marc Spaggiari commented on HBASE-7824: Hi Jeffrey, Is this for 0.94.x? Or for trunk? If it's for trunk, I think the patch not done on the right directory level. Seems to be missing hbaser-server/ Improve master start up time when there is log split work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.6 Attachments: hbase-7824.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log split work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584765#comment-13584765 ] Jeffrey Zhong commented on HBASE-7824: -- This patch is for 0.94.6. Improve master start up time when there is log split work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.6 Attachments: hbase-7824.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7843) enable encapsulating compaction policy/compactor/store file manager interaction shennanigans
[ https://issues.apache.org/jira/browse/HBASE-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7843: Attachment: HBASE-7843-v2.patch fix javadoc warning, rebase and some small changes. Findbugs warning seems to be bogus judging by files in testReport enable encapsulating compaction policy/compactor/store file manager interaction shennanigans Key: HBASE-7843 URL: https://issues.apache.org/jira/browse/HBASE-7843 Project: HBase Issue Type: Improvement Components: Compaction Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7843-v0.patch, HBASE-7843-v1.patch, HBASE-7843-v2.patch To avoid massive casting and/or deciphering of structures traveling between SFM, compaction policy, and compactor in non-trivial compaction schemes like stripe or level, we need to make interaction between themselves hidden. Elsewhere, the changes are being made to coprocessor for compactions that will make CompactionRequest a limited-visibility class for users, with coprocessors being able to subclass and return it. -This seems like a viable solution for the problem at hand too. Policy will (optionally) subclass compaction request and return it. Instead of calling something.compact(req), req.compact() will be called (with something already stored inside req as of now), after which, magic will happen.- After merging that code I actually see that subclassing compactionrequest in both will break the policy, or require bunch of ugly code in coprocessors (a subclass for every policy, and telling them apart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7910) Dont use reflection for security
Enis Soztutar created HBASE-7910: Summary: Dont use reflection for security Key: HBASE-7910 URL: https://issues.apache.org/jira/browse/HBASE-7910 Project: HBase Issue Type: Improvement Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 security.User class uses reflection so that HBase can work with older Hadoop's not having security. Now that we require 1.x, or 0.23 or 2.x, all Hadoop versions have security code. We can get rid of most of the User class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7911) Remove duplicated code from CreateTableHandler
Matteo Bertozzi created HBASE-7911: -- Summary: Remove duplicated code from CreateTableHandler Key: HBASE-7911 URL: https://issues.apache.org/jira/browse/HBASE-7911 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7911-v0.patch The snapshot merge have left some unused/duplicated code in CreateTableHandler/ModifyRegionUtils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7911) Remove duplicated code from CreateTableHandler
[ https://issues.apache.org/jira/browse/HBASE-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7911: --- Attachment: HBASE-7911-v0.patch Remove duplicated code from CreateTableHandler -- Key: HBASE-7911 URL: https://issues.apache.org/jira/browse/HBASE-7911 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7911-v0.patch The snapshot merge have left some unused/duplicated code in CreateTableHandler/ModifyRegionUtils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7911) Remove duplicated code from CreateTableHandler
[ https://issues.apache.org/jira/browse/HBASE-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7911: --- Status: Patch Available (was: Open) Remove duplicated code from CreateTableHandler -- Key: HBASE-7911 URL: https://issues.apache.org/jira/browse/HBASE-7911 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.96.0 Attachments: HBASE-7911-v0.patch The snapshot merge have left some unused/duplicated code in CreateTableHandler/ModifyRegionUtils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7902) deletes may be removed during minor compaction, depending on file set
[ https://issues.apache.org/jira/browse/HBASE-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7902: Attachment: HBASE-7902-v0-with-7843.patch HBASE-7902-v0.patch Patch is be based on HBASE-7843, as the mechanism to propagate the flag will have to be different before and after that, no sense to redo it twice. Attaching the actual patch and the combined patch Apache will be able to apply. The patch changes ScanType semantics and names for compaction to be more precise, and changes policy to set it based on whether we go from the oldest file. Piggybacked on existing test to verify. deletes may be removed during minor compaction, depending on file set - Key: HBASE-7902 URL: https://issues.apache.org/jira/browse/HBASE-7902 Project: HBase Issue Type: Improvement Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HBASE-7902-v0.patch, HBASE-7902-v0-with-7843.patch Deletes are only removed during major compaction now. However, in presence of file ordering, deletes can be removed during minor compaction too, as long as there's no file that is not being compacted that is older than the files that are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7902) deletes may be removed during minor compaction, depending on file set
[ https://issues.apache.org/jira/browse/HBASE-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7902: Status: Patch Available (was: Open) deletes may be removed during minor compaction, depending on file set - Key: HBASE-7902 URL: https://issues.apache.org/jira/browse/HBASE-7902 Project: HBase Issue Type: Improvement Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HBASE-7902-v0.patch, HBASE-7902-v0-with-7843.patch Deletes are only removed during major compaction now. However, in presence of file ordering, deletes can be removed during minor compaction too, as long as there's no file that is not being compacted that is older than the files that are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584780#comment-13584780 ] Nick Dimiduk commented on HBASE-7692: - I'm not sure what's going on with HadoopQA here. The patch applies with patch -p1 for me. Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Open (was: Patch Available) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Patch Available (was: Open) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
